LXC Default Network

Ive been trying to follow Canonicals implantation of Micro Cloud - which is know is just packaging up LXC/OVN/Ceph, and Im running into issues with the default network link. Im not following their instructions to the T since Im creating 3 Ubuntu 22.04 VMs, and then building them out from there. Each of the VMs have dual nics assigned to them.

Main nic: eth0 - dhcp from pfsense router
Second nic: ens19 - no ip - using for the distributed networking

Output from the servers is identical for Netplan:

network:
    version: 2
    ethernets:
        eth0:
            dhcp4: true
            dhcp6: no
            match:
                macaddress: 3a:29:b2:d8:8d:25
            set-name: eth0
        ens19:
          dhcp4: no
          dhcp6: no
          link-local: []

I had to set the link-local since it was showing an IPv6 address, and LXC will ignore it if there is an address.

Output for the interfaces:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 3a:29:b2:d8:8d:25 brd ff:ff:ff:ff:ff:ff
    altname enp0s18
    inet 192.168.10.6/24 metric 100 brd 192.168.10.255 scope global dynamic eth0
       valid_lft 5178sec preferred_lft 5178sec
    inet6 fe80::3829:b2ff:fed8:8d25/64 scope link 
       valid_lft forever preferred_lft forever
3: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000
    link/ether e6:c0:3b:78:db:c2 brd ff:ff:ff:ff:ff:ff
    altname enp0s19
4: ovs-system: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 7a:53:3f:34:ab:23 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::7853:3fff:fe34:ab23/64 scope link 
       valid_lft forever preferred_lft forever
5: br-int: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fe:d1:3a:47:9f:90 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fcd1:3aff:fe47:9f90/64 scope link 
       valid_lft forever preferred_lft forever
6: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
    link/ether 86:2f:fc:9b:70:3f brd ff:ff:ff:ff:ff:ff
    inet6 fe80::4865:a4ff:fef8:b5bb/64 scope link 
       valid_lft forever preferred_lft forever
7: lxdovn1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 36:10:a9:c7:89:42 brd ff:ff:ff:ff:ff:ff
    inet6 fd44:707e:d19a:dd4a:3410:a9ff:fec7:8942/64 scope global dynamic mngtmpaddr 
       valid_lft 1786sec preferred_lft 1786sec
    inet6 fe80::3410:a9ff:fec7:8942/64 scope link 
       valid_lft forever preferred_lft forever

lxc network output:

+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
|  NAME   |   TYPE   | MANAGED |      IPV4      |           IPV6            | DESCRIPTION | USED BY |  STATE  |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
| UPLINK  | physical | YES     |                |                           |             | 1       | CREATED |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
| br-int  | bridge   | NO      |                |                           |             | 0       |         |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
| default | ovn      | YES     | 10.254.30.1/24 | fd42:1a12:c9fd:7108::1/64 |             | 2       | CREATED |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
| ens19   | physical | NO      |                |                           |             | 1       |         |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
| eth0    | physical | NO      |                |                           |             | 0       |         |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+
| lxdovn1 | bridge   | NO      |                |                           |             | 0       |         |
+---------+----------+---------+----------------+---------------------------+-------------+---------+---------+

Routing information:

default via 192.168.10.1 dev eth0 proto dhcp src 192.168.10.6 metric 100 
8.8.8.8 via 192.168.10.1 dev eth0 proto dhcp src 192.168.10.6 metric 100 
192.168.10.0/24 dev eth0 proto kernel scope link src 192.168.10.6 metric 100 
192.168.10.1 dev eth0 proto dhcp scope link src 192.168.10.6 metric 100

Lxc default network info:

config:
  bridge.mtu: "1442"
  ipv4.address: 10.254.30.1/24
  ipv4.nat: "true"
  ipv6.address: fd42:1a12:c9fd:7108::1/64
  ipv6.nat: "true"
  network: UPLINK
  volatile.network.ipv4.address: 192.168.10.100
description: ""
name: default
type: ovn
used_by:
- /1.0/instances/u1
- /1.0/profiles/default
managed: true
status: Created
locations:
- cloud1
- cloud3
- cloud2

What in the world am I missing? I can’t ping that volatile IPv4 address.

I got it figured out. I forgot that when I created these in Proxmox that I needed to make sure I was tagging the correct VLAN ID on the interface. After ~10 rebuilds using Terraform/Ansible, I was able to get everything working right. Now to try in the actual environment.