Two containers with same ip address if restored from same backup

I just did following and now two containers are running with same IP.

$ lxc backup test1 test1.tar.gz --instance-only
$ lxc import test1.tar.gz test2
$ lxc start test2
$ lxc list

+----------------+---------+-------------------+------+-----------+-----------+----------------+
|      NAME      |  STATE  |       IPV4        | IPV6 |   TYPE    | SNAPSHOTS |    LOCATION    |
+----------------+---------+-------------------+------+-----------+-----------+----------------+
| test1          | RUNNING | 240.0.2.61 (eth0) |      | CONTAINER | 0         | do-01          |
+----------------+---------+-------------------+------+-----------+-----------+----------------+
| test2          | RUNNING | 240.0.2.61 (eth0) |      | CONTAINER | 0         | do-01          |
+----------------+---------+-------------------+------+-----------+-----------+----------------+

$ lxc info test1

Ips:
  lo:   inet    127.0.0.1
  lo:   inet6   ::1
  eth0: inet    240.0.2.61     veth564bad03
  eth0: inet6   fe80::216:3eff:fed2:de0f        veth564bad03

$ lxc info test2
Ips:
  eth0: inet    240.0.2.61     veth34c79e21
  eth0: inet6   fe80::216:3eff:fed2:de0f        veth34c79e21
  lo:   inet    127.0.0.1
  lo:   inet6   ::1

 

Is this an accepted behaviour?

I thought dnsmasq will not assign same ip to two containers.

Recently discovered this issue myself… Its because the export backs up the hardware address (mac address) of the container.

Since DHCP keys IP address from the hardware address they will both be assigned the same IP.

The fix is to unset the hardware address on clones of your import and then lxd will automaticly assign it a new hardware address.

If your like myself and rely on the default profile for the lxdbr0 entry… You will have to unset ‘volatile.eth0.hwaddr’ which works but is bad practice as all the volatile config is internal and subject to change without notice.

So on my system I import the container… if its running I stop it…
then I run

lxc config unset <containername> volatile.eth0.hwaddr

Also, if you’re looking to create multiple instances from a backup, then you could also consider using lxc publish to publish the instance as an image instead of lxc export. This would mean that each instance created from that image (using lxc init <image name>) will get its own fresh config (including MAC address).

I feel the correct behaviour should be to assign new IP . It leads to all kinds of errors if two instances are running with same IP.

When using lxc export you are creating a backup of a single instance, and the intended use-case for that is to restore a single instance (because it is a backup).

When restoring the backup all of the original config is restored, including the MAC address.
A DHCP server generates IP assignments based on the MAC address, so if the MAC address is the same then the same IP will be allocated. However even aside from that issue, if you have two active instances with the same MAC address you are going to see other network issues.

Also if we started altering MAC address config on backup restoration this would cause issues for people who have used static IP allocation or network security rules based on MAC address.

If you’re looking to create multiple instances from a source instance, then you need to change the MAC address as @Ozymandias described, or use the lxc publish mechanism to create an image that can be used as the basis of multiple instances (which LXD then treats differently from restoring a backup by removing any volatile config, such as MAC address, so that each instance generates its own MAC address, and thus own IP allocation in DHCP).

1 Like

Here’s an example of using lxc publish:

# Create your source container and add your own modifications as needed.
lxc launch images:ubuntu/focal c1
lxc exec c1 -- touch /root/helloworld.txt
lxc stop c1

# Publish under a friendly alias.
lxc publish c1 --alias myimage
lxc image ls myimage
+---------+--------------+--------+-------------------------------------+--------------+-----------+----------+------------------------------+
|  ALIAS  | FINGERPRINT  | PUBLIC |             DESCRIPTION             | ARCHITECTURE |   TYPE    |   SIZE   |         UPLOAD DATE          |
+---------+--------------+--------+-------------------------------------+--------------+-----------+----------+------------------------------+
| myimage | facac9b50efd | no     | Ubuntu focal amd64 (20210223_07:42) | x86_64       | CONTAINER | 133.30MB | Feb 25, 2021 at 9:51am (UTC) |
+---------+--------------+--------+-------------------------------------+--------------+-----------+----------+------------------------------+

# Create container from image:
lxc launch myimage c2

# Check the new container contains our modifications:
lxc exec c2 -- ls /root/helloworld.txt
/root/helloworld.txt

# Start c1 and check both get own IPs:
lxc start c1
lxc ls
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| NAME |  STATE  |         IPV4         |                     IPV6                      |   TYPE    | SNAPSHOTS |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| c1   | RUNNING | 10.107.121.10 (eth0) | fd42:bd51:7486:d7b5:216:3eff:fe35:e509 (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+
| c2   | RUNNING | 10.107.121.7 (eth0)  | fd42:bd51:7486:d7b5:216:3eff:fe48:63fa (eth0) | CONTAINER | 0         |
+------+---------+----------------------+-----------------------------------------------+-----------+-----------+

If you want to restore from backup, and the original instance exists on the machine and is already running, you should be stopping it anyway.

I am ok with the current behaviour and made necessary changes in my system for backup and restore.

Still the command syntax of lxc import implies that a new container is created from the backup. If it’s lxc restore it will than be correct to assume that container will be the same including name. If the old container exist lxc restore will fail (this will indicate to sys admin to either rename the old container or delete it, before restoring the backup). This will mean the container name should automatically come from backup.

In my view this will help avoid having same Mac or IP address which will result in difficult to diagnose errors.

lxc restore already exists, it’s the command used to restore a snapshot.

How about extending it to define the backup file as another option?

I haven’t used lxc restore to restore from snapshots. I always launched a new container from snapshot and then just point the load-balancer towards new container. This makes it easier to have immutable container based infrastructure, in case the new container has an issue can point load-balancer to old container.