OVN High CPU usage with Duplicate IP set warning

Freedom_Fury · July 31, 2024, 9:27am

Good day,

Im running Ubuntu 22.04 LTS incus 6.3.

After setting a static IP on a container, I’m getting a high CPU warning and Duplicate IP set warning in the logs. If I remove the container with the static, the warnings will disappear.

==> /var/log/ovn/ovn-northd.log <==
2024-07-31T09:16:17.595Z|02131|poll_loop|INFO|Dropped 5542 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2024-07-31T09:16:17.595Z|02132|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (192.168.10.50:35554<->192.168.10.50:6641) at lib/stream-fd.c:157 (71% CPU usage)
2024-07-31T09:16:23.594Z|02133|poll_loop|INFO|Dropped 5486 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2024-07-31T09:16:23.594Z|02134|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (192.168.10.50:35554<->192.168.10.50:6641) at lib/stream-fd.c:157 (72% CPU usage)
2024-07-31T09:16:29.595Z|02135|poll_loop|INFO|Dropped 5897 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2024-07-31T09:16:29.595Z|02136|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (192.168.10.50:35554<->192.168.10.50:6641) at lib/stream-fd.c:157 (74% CPU usage)
2024-07-31T09:16:31.093Z|02137|ipam|WARN|Dropped 58590 log messages in last 60 seconds (most recently, 0 seconds ago) due to excessive rate
2024-07-31T09:16:31.093Z|02138|ipam|WARN|6bad30e2-d319-4373-ad6f-eb35c476713d: Duplicate IP set: 10.12.220.250

I tried to launch the instance with a static IP.

 inc launch images:alpine/edge/cloud --network ovn5 dnss01 --device  eth0,ipv4.address=10.12.220.250

I also set the IP after the container was created.

inc launch images:alpine/edge/cloud --network ovn5 dnss01 --config ipv4.address=eth0,10.12.220.250

the OVN nbctl command shows no duplicates.

sudo  ovn-nbctl --columns=name,dynamic_addresses,addresses list logical_switch_port

name                : incus-net15-instance-6efa14a9-a470-4d8d-9fe8-2a92b8cf3a6a-eth0
dynamic_addresses   : "00:16:3e:2f:39:5f 10.12.220.2"
addresses           : ["00:16:3e:2f:39:5f dynamic"]

name                : incus-net15-ls-int-lsp-router
dynamic_addresses   : []
addresses           : [router]

name                : incus-net15-instance-f3372a80-62ea-47be-8bd9-b0e79a7d8b37-eth0
dynamic_addresses   : []
addresses           : ["00:16:3e:85:07:c5 10.12.220.250"]

name                : incus-net15-ls-ext-lsp-provider
dynamic_addresses   : []
addresses           : [unknown]

name                : incus-net15-ls-ext-lsp-router
dynamic_addresses   : []
addresses           : [router]

Freedom_Fury · August 1, 2024, 9:13am

Searching the C code for OVN, I can see where this message is being emitted. It’s the only place in the code base with the exact warning for the duplicate IP. I’m no C expert here, but it indicates that the static IP overlaps the router’s IP range.

github.com

ovn-org/ovn/blob/6234e092d2b7dea845d71d8948b45050d9bbb5c9/northd/ipam.c#L62


      
          {
              if (!info->allocated_ipv4s) {
                  return true;
              }
          
              if (ip >= info->start_ipv4 &&
                  ip < (info->start_ipv4 + info->total_ipv4s)) {
                  if (bitmap_is_set(info->allocated_ipv4s,
                                    ip - info->start_ipv4)) {
                      static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
                      VLOG_WARN_RL(&rl, "%s: Duplicate IP set: " IP_FMT,
                                   info->id, IP_ARGS(htonl(ip)));
                      return false;
                  }
                  bitmap_set1(info->allocated_ipv4s,
                              ip - info->start_ipv4);
              }
              return true;
          }
          
          uint32_t

Freedom_Fury · August 3, 2024, 9:30am

It turns out that I have a different device on my network using the first IP address for the routed subnet I set up on the physical uplink device. This seemed to cause an infinite loop, which consumed a good deal of CPU time.