Host running LXD, Ubuntu 19.04, Open-Vswitch has arp table overflow

We are slowly migrating from vmware ESXi to LXD and one of our Hosts every 5 days runs out of arp cache,
We can increase the value sysctl net.ipv4.neigh.default.gc_thresh3
to address the issue But can’t find the cause. On the arp -an has 2 entries

Can you show us the actual error you are getting?

Also, can you provide more info about the network layout and LXD configuration you are using please.

Actual Error Message
syslog.5.gz:May 7 14:36:48 PRD32-LXD24 kernel: [669472.411419] neighbour: arp_cache: neighbor table overflow!

The network config
We have 2 ports configured as master/slave bond for admin access bond0
We have 2 ports configured as a master/slave for openvswitch cofigured as a bridge
Then we have 3 vlans configured for different containers connected to br0 which the containers connect to

ip addr output (Note: i have removed ip addresses)
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN group default qlen 1000
    link/ether 3e:1a:ff:xx:xx:xx brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 3e:1a:ff:xx:xx:Xx brd ff:ff:ff:ff:ff:ff
4: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether ec:f4:bb:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet6 fe80::eef4:xxxx:xxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
5: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether ec:f4:bb:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet6 fe80::eef4:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 3e:1a:ff:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 10.x.x.24/24 brd 10.x.x.255 scope global bond0
       valid_lft forever preferred_lft forever
    inet6 fe80::3c1a:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
7: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7e:f1:42:xx:xx:xx brd ff:ff:ff:ff:ff:ff
8: vlan1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:99:5b:xx:xx:xx brd ff:ff:ff:ff:ff:ff
9: vlan2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:99:5b:xx:xx:xx brd ff:ff:ff:ff:ff:ff
10: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether ec:f4:bb:xx:xx:xx brd ff:ff:ff:ff:ff:ff
11: vlan3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 7a:99:5b:xx:xx:xx brd ff:ff:ff:ff:ff:ff
29: veth5894b7ab@if28: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether b6:fc:21:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 3
    inet6 fe80::b4fc:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
31: vetha89afdd4@if30: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 12:9d:ba:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 8
    inet6 fe80::109d:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
35: vethf02eb959@if34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 06:da:6d:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 5
    inet6 fe80::4da:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
41: veth1c8ade5e@if40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 62:35:55:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 11
    inet6 fe80::6035:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
45: veth843cfc1c@if44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 3a:0c:c5:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 6
    inet6 fe80::380c:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
47: vethcdc6f273@if46: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether ee:c6:21:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 12
    inet6 fe80::ecc6:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever
49: veth25d2cc33@if48: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 6e:97:bc:xx:xx:xx brd ff:ff:ff:ff:ff:ff link-netnsid 10
    inet6 fe80::6c97:xxxx:xxxx:xxxx/64 scope link
       valid_lft forever preferred_lft forever

Open vswitch

 Bridge "br0"
        Port "veth14f917b5"
            tag: VLAN1
            Interface "veth14f917b5"
                error: "could not open network device veth14f917b5 (No such device)"
        Port "veth40786deb"
            tag: VLAN1
            Interface "veth40786deb"
                error: "could not open network device veth40786deb (No such device)"
        Port "vethb9077afb"
            tag: VLAN1
            Interface "vethb9077afb"
                error: "could not open network device vethb9077afb (No such device)"
        Port "veth261d44ad"
            tag: VLAN1
            Interface "veth261d44ad"
                error: "could not open network device veth261d44ad (No such device)"
        Port "vethe1f14b92"
            tag: VLAN1
            Interface "vethe1f14b92"
                error: "could not open network device vethe1f14b92 (No such device)"
        Port "veth1c8ade5e"
            tag: VLAN1
            Interface "veth1c8ade5e"
        Port "veth5894b7ab"
            tag: VLAN1
            Interface "veth5894b7ab"
        Port "vethf02eb959"
            tag: VLAN1
            Interface "vethf02eb959"
        Port "veth1b041011"
            tag: VLAN1
            Interface "veth1b041011"
                error: "could not open network device veth1b041011 (No such device)"
        Port vlan2name
            tag: VLAN2
            Interface vlan2name
                type: internal
        Port "vethec88dbb1"
            tag: VLAN1
            Interface "vethec88dbb1"
                error: "could not open network device vethec88dbb1 (No such device)"
        Port "veth0d6a2f90"
            tag: VLAN1
            Interface "veth0d6a2f90"
                error: "could not open network device veth0d6a2f90 (No such device)"
        Port "vethdb224cfe"
            tag: VLAN1
            Interface "vethdb224cfe"
                error: "could not open network device vethdb224cfe (No such device)"
        Port "vetha89afdd4"
            tag: VLAN1
            Interface "vetha89afdd4"
        Port "veth843cfc1c"
            tag: VLAN1
            Interface "veth843cfc1c"
        Port "veth85d0c3bc"
            tag: VLAN1
            Interface "veth85d0c3bc"
                error: "could not open network device veth85d0c3bc (No such device)"
        Port "vethe54104d9"
            tag: VLAN1
            Interface "vethe54104d9"
                error: "could not open network device vethe54104d9 (No such device)"
        Port "veth497f3c74"
            tag: VLAN3
            Interface "veth497f3c74"
                error: "could not open network device veth497f3c74 (No such device)"
        Port "vethd773292f"
            tag: VLAN1
            Interface "vethd773292f"
                error: "could not open network device vethd773292f (No such device)"
        Port "veth3c7d2dbf"
            tag: VLAN1
            Interface "veth3c7d2dbf"
                error: "could not open network device veth3c7d2dbf (No such device)"
        Port "veth25d2cc33"
            tag: VLAN1
            Interface "veth25d2cc33"
        Port "veth34ad159d"
            tag: VLAN1
            Interface "veth34ad159d"
                error: "could not open network device veth34ad159d (No such device)"
        Port "veth07294c63"
            tag: VLAN1
            Interface "veth07294c63"
                error: "could not open network device veth07294c63 (No such device)"
        Port "veth05fe3ad8"
            tag: VLAN1
            Interface "veth05fe3ad8"
                error: "could not open network device veth05fe3ad8 (No such device)"
        Port "vethac4e6758"
            tag: VLAN1
            Interface "vethac4e6758"
                error: "could not open network device vethac4e6758 (No such device)"
        Port "veth642d011a"
            tag: VLAN3
            Interface "veth642d011a"
                error: "could not open network device veth642d011a (No such device)"
        Port "vethf080f29f"
            tag: VLAN1
            Interface "vethf080f29f"
                error: "could not open network device vethf080f29f (No such device)"
        Port "vethf5e8cb11"
            tag: VLAN1
            Interface "vethf5e8cb11"
                error: "could not open network device vethf5e8cb11 (No such device)"
        Port "veth8ad6132f"
            tag: VLAN1
            Interface "veth8ad6132f"
                error: "could not open network device veth8ad6132f (No such device)"
        Port "veth8d896a60"
            tag: VLAN1
            Interface "veth8d896a60"
                error: "could not open network device veth8d896a60 (No such device)"
        Port "br0"
            Interface "br0"
                type: internal
        Port "veth477c0d97"
            tag: VLAN1
            Interface "veth477c0d97"
                error: "could not open network device veth477c0d97 (No such device)"
        Port "vethd9943456"
            tag: VLAN1
            Interface "vethd9943456"
                error: "could not open network device vethd9943456 (No such device)"
        Port vlan1name
            tag: VLAN1
            Interface vlan1name
                type: internal
        Port "vethfa397504"
            tag: VLAN1
            Interface "vethfa397504"
                error: "could not open network device vethfa397504 (No such device)"
        Port "veth07033237"
            tag: VLAN1
            Interface "veth07033237"
                error: "could not open network device veth07033237 (No such device)"
        Port "veth21e5aea9"
            tag: VLAN1
            Interface "veth21e5aea9"
                error: "could not open network device veth21e5aea9 (No such device)"
        Port "veth28526fc9"
            tag: VLAN1
            Interface "veth28526fc9"
                error: "could not open network device veth28526fc9 (No such device)"
        Port "veth5af87bcf"
            tag: VLAN1
            Interface "veth5af87bcf"
                error: "could not open network device veth5af87bcf (No such device)"
        Port "veth9bad62da"
            tag: VLAN1
            Interface "veth9bad62da"
                error: "could not open network device veth9bad62da (No such device)"
        Port "vethcdc6f273"
            tag: VLAN1
            Interface "vethcdc6f273"
        Port "veth27abb7f9"
            tag: VLAN1
            Interface "veth27abb7f9"
                error: "could not open network device veth27abb7f9 (No such device)"
        Port "veth57829ebf"
            tag: VLAN1
            Interface "veth57829ebf"
                error: "could not open network device veth57829ebf (No such device)"
        Port "veth15124003"
            tag: VLAN1
            Interface "veth15124003"
                error: "could not open network device veth15124003 (No such device)"
        Port vlan3name
            tag: VLAN3
            Interface vlan3name
                type: internal
        Port "bond1"
            Interface "eno4"
            Interface "eno3"
        Port "veth55b78dda"
            tag: VLAN1
            Interface "veth55b78dda"
                error: "could not open network device veth55b78dda (No such device)"
        Port "vethe1207e35"
            tag: VLAN1
            Interface "vethe1207e35"
                error: "could not open network device vethe1207e35 (No such device)"
        Port "veth9254af89"
            tag: VLAN1
            Interface "veth9254af89"
                error: "could not open network device veth9254af89 (No such device)"
    ovs_version: "2.11.0"
config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- candid_authentication
- backup_compression
- candid_config
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- candid_config_key
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- rbac
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses: []
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
  certificate_fingerprint: eb
  driver: lxc
  driver_version: 3.2.1
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
    netnsid_getifaddrs: "true"
    seccomp_listener: "true"
    shiftfs: "false"
    uevent_injection: "true"
    unpriv_fscaps: "true"
  kernel_version: 5.0.0-13-generic
  lxc_features:
    mount_injection_file: "true"
    network_gateway_device_route: "true"
    network_ipvlan: "true"
    network_l2proxy: "true"
    network_phys_macvlan_mtu: "true"
    seccomp_notify: "true"
  project: default
  server: lxd
  server_clustered: false
  server_name: xxxx-xxx
  server_pid: 2812
  server_version: "3.16"
  storage: zfs
  storage_version: 0.7.12-1ubuntu5

Can you show output of ip neigh show on host?

Also, you can use three backticks to wrap any code/config/command output to make it easier to read.

How many hosts are on your network?