Copying an arch vm results in different mac address, but same ip address

Incus
,
1

On a clean arch linux system, initializing incus, creating two arch linux VMs, if I use incus copy, both VMs end up with the same ip address.

Tried:

  • restarting systemd-networkd results in same ip
  • trying another solution: Error: Setting "volatile.eth0.dhcp.identifier" on virtual-machine "arch-vm-2" in project "user-1000" is forbidden

My code below:

sudo usermod -v 1000000-1000999999 -w 1000000-1000999999 root
sudo systemctl start incus
sudo systemctl start incus.socket
sudo systemctl start incus-user.socket

incus init images:archlinux arch-vm-1 --vm \
    -c security.secureboot=false \
    -c limits.memory=8GiB \
    -c limits.cpu=$(nproc) \
    -c boot.autostart=false

incus start arch-vm-1
incus exec arch-vm-1 -- pacman -Syu
incus copy arch-vm-1 arch-vm-2
incus start arch-vm-2
incus list

+-----------+---------+------------------------+-------------------------------------------------+-----------------+-----------+
|   NAME    |  STATE  |          IPV4          |                      IPV6                       |      TYPE       | SNAPSHOTS |
+-----------+---------+------------------------+-------------------------------------------------+-----------------+-----------+
| arch-vm-1 | RUNNING | 10.48.171.195 (enp5s0) | fd42:6c26:1675:d160:216:3eff:feac:3f8a (enp5s0) | VIRTUAL-MACHINE | 0         |
+-----------+---------+------------------------+-------------------------------------------------+-----------------+-----------+
| arch-vm-2 | RUNNING | 10.48.171.195 (enp5s0) | fd42:6c26:1675:d160:216:3eff:fea0:2e0 (enp5s0)  | VIRTUAL-MACHINE | 0         |
+-----------+---------+------------------------+-------------------------------------------------+-----------------+-----------+

incus exec arch-vm-1 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:ac:3f:8a brd ff:ff:ff:ff:ff:ff
    inet 10.48.171.195/24 metric 1024 brd 10.48.171.255 scope global dynamic enp5s0
       valid_lft 3083sec preferred_lft 3083sec
    inet6 fd42:6c26:1675:d160:216:3eff:feac:3f8a/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:feac:3f8a/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever

incus exec arch-vm-2 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: enp5s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:16:3e:a0:02:e0 brd ff:ff:ff:ff:ff:ff
    inet 10.48.171.195/24 metric 1024 brd 10.48.171.255 scope global dynamic enp5s0
       valid_lft 3166sec preferred_lft 3166sec
    inet6 fd42:6c26:1675:d160:216:3eff:fea0:2e0/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::216:3eff:fea0:2e0/64 scope link proto kernel_ll 
       valid_lft forever preferred_lft forever


incus network list
+------+------+---------+------+------+-------------+---------+-------+
| NAME | TYPE | MANAGED | IPV4 | IPV6 | DESCRIPTION | USED BY | STATE |
+------+------+---------+------+------+-------------+---------+-------+



sudo incus network list
To start your first container, try: incus launch images:ubuntu/22.04
Or for a virtual machine: incus launch images:ubuntu/22.04 --vm

+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+
|     NAME     |   TYPE   | MANAGED |      IPV4      |           IPV6            |                    DESCRIPTION                     | USED BY |  STATE  |
+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+
| eno1         | physical | NO      |                |                           |                                                    | 0       |         |
+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+
| enp5s0       | physical | NO      |                |                           |                                                    | 0       |         |
+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+
| incusbr0     | bridge   | YES     | 10.97.209.1/24 | fd42:b190:4a32:73d2::1/64 |                                                    | 1       | CREATED |
+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+
| incusbr-1000 | bridge   | YES     | 10.48.171.1/24 | fd42:6c26:1675:d160::1/64 | Network for user restricted project user-user-1000 | 3       | CREATED |
+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+
| wlp8s0       | physical | NO      |                |                           |                                                    | 0       |         |
+--------------+----------+---------+----------------+---------------------------+----------------------------------------------------+---------+---------+

incus exec arch-vm-2 -- systemctl restart systemd-networkd

incus exec arch-vm-2 -- journalctl | tail -n1
Feb 09 22:29:03 archlinux systemd-networkd[636]: enp5s0: DHCPv4 address 10.48.171.195/24, gateway 10.48.171.1 acquired from 10.48.171.1

# Try setting dhcp.identifier=mac? https://discuss.linuxcontainers.org/t/ipv4-conflict-between-all-copied-vms-using-macvlan/10586/17

incus config set arch-vm-2 enp5s0 dhcp.identifier=mac
Error: Invalid config: Unknown configuration key: enp5s0

incus config device list arch-vm-2
# no output

incus config show arch-vm-2 --expanded
architecture: x86_64
config:
  boot.autostart: "false"
  image.architecture: amd64
  image.description: Archlinux current amd64 (20240209_04:18)
  image.os: Archlinux
  image.release: current
  image.requirements.secureboot: "false"
  image.serial: "20240209_04:18"
  image.type: disk-kvm.img
  image.variant: default
  limits.cpu: "16"
  limits.memory: 8GiB
  security.secureboot: "false"
  volatile.base_image: c908864677e28430ca8011af9634385eb8fa02708e970b2fedc6b191d6821577
  volatile.cloud-init.instance-id: e29b6eee-1291-48f5-a8a8-666ffd539a6c
  volatile.eth0.host_name: tapee423610
  volatile.eth0.hwaddr: 00:16:3e:a0:02:e0
  volatile.last_state.power: RUNNING
  volatile.uuid: 6bbaa758-4239-42ab-b31c-d9c821b3fdf9
  volatile.uuid.generation: 6bbaa758-4239-42ab-b31c-d9c821b3fdf9
  volatile.vsock_id: "2615442052"
devices:
  eth0:
    name: eth0
    network: incusbr-1000
    type: nic
  root:
    path: /
    pool: default
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

incus config set arch-vm-2 eth0 dhcp.identifier=mac
Error: Invalid config: Unknown configuration key: eth0

incus config set arch-vm-2 volatile.eth0.dhcp.identifier=mac
Error: Setting "volatile.eth0.dhcp.identifier" on virtual-machine "arch-vm-2" in project "user-1000" is forbidden
2

It says project user-1000. Is that a non-root account that only belongs to the restrictive incus group (and not incus-admin)?

An Incus instance gets the same IP address if it requests DHCP configuration with the same MAC address. Changing the MAC address should require either membership to incus-admin or a sudo.

3

That should take care of that but will take 24h to hit all images and will only then affect new instances. You can manually apply the change to your instances though.

1 Like
4

Is that a non-root account that only belongs to the restrictive incus group (and not incus-admin )?

Yes I cautiously avoided use of the incus-admin group. Is this warning on the arch linux wiki valid?

Warning: Anyone added to the incus-admin group is root equivalent. For more information, see [1] and [2].

Out of curiosity, in what scenario would it make sense to allow privileged access to incus-admin?
I also note this is the same warning with docker:

Warning: Anyone added to the docker group is root equivalent because they can use the docker run --privileged command to start containers with root privileges. For more information see [3] and [4].

An Incus instance gets the same IP address if it requests DHCP configuration with the same MAC address. Changing the MAC address should require either membership to incus-admin or a sudo.

From the output of ip addr on each machine they had unique mac addresses already, but still kept getting the same IP.
link/ether 00:16:3e:ac:3f:8a
link/ether 00:16:3e:a0:02:e0

That should take care of that but will take 24h to hit all images and will only then affect new instances. You can manually apply the change to your instances though.

@stgraber thank you!