Reinitializing network-config in existing container

I can do the following to configure networking in a new container:

lxc init ubuntu:18.04 test-cloud-init
lxc config set test-cloud-init user.network-config "$(cat test-ci.yml)"
lxc start

This works fine. Now I want to re-run cloud-init in the same container.

I modify test-ci.yml, use it to reset user.network-config, and use cloud-init clean to force cloud-init to run again on next boot:

lxc config set test-cloud-init user.network-config "$(cat test-ci.yml)"
lxc exec test-cloud-init cloud-init clean
lxc restart test-cloud-init

Unfortunately, the old configuration is re-applied. This is because the initial value of user.network-config was written to /var/lib/cloud/seed/nocloud-net/network-config inside the container, and it has not been updated with the new value of the user.network-config property.

If I do cloud-init clean --seed then the whole /var/lib/cloud/seed/ subtree is deleted, but it’s not recreated when the container restarts.

Is there a way to get lxd to refresh the contents of /var/lib/cloud/seed/nocloud-net/ on an existing container from the user.* properties?

Doing cloud-init clean --seed in the container followed by:

  • lxc stop NAME
  • lxc config set NAME volatile.apply_template create
  • lxc start NAME

Should do the trick. This will cause both cloud-init to wipe its state and instruct LXD to re-run the templates that would normally only be run on the container’s first boot after creation.

That works - even with lxc restart rather than lxc stop and lxc start. Thank you!

Hello,

Is there any newer method to do this on Incus?

Nope

Is there any way to do this on incus?

editing the containers config so that

config:
  cloud-init.network-config: |-
    version: 2
    ethernets:
      eth0:
        addresses:
          - 69.41.138.102/27
        gateway4: 69.41.138.97
        nameservers:
          addresses:
            - 69.41.138.98
            - 8.8.4.4
      eth1:
        addresses:
          - 192.168.31.141/24
... other stuff ...
devices:
  eth1:
    name: eth1
    nictype: bridged
    parent: br1
    type: nic
... other stuff ...

cloud-init clean --seed

followed by

incus config set NAME volatile.apply_template=create

doesn’t seem to work.

root@tk2022:~# incus list
±----------±--------±-----------------------------±-----±----------±----------+
|   NAME    |  STATE  |             IPV4             | IPV6 |   TYPE    | SNAPSHOTS |
±----------±--------±-----------------------------±-----±----------±----------+
…
±----------±--------±-----------------------------±-----±----------±----------+
| ezra      | RUNNING | 69.41.138.110 (eth0)         |      | CONTAINER | 2   |
|           |         | 69.41.138.102 (eth0)         |      |           |     |


The original container was created with

incus init trixie -p default -p susdev25 ezra -c cloud-init.network-config="$(cat <<EOF
version: 2
ethernets:
  eth0:
    addresses:
      - 69.41.138.110/27
    gateway4: 69.41.138.97
    nameservers:
      addresses:
        - 69.41.138.98
        - 8.8.4.4
EOF
)"



Where trixie is an alias to debian/13/cloud.
I can’t get it to let go of that address. It picks it up regardless of what is in /etc/network/interfaces or the containers configuration.

Sorry to highjack an old thread.

Where did that local “trixie” image come from? Try making a standalone reproducer, e.g. with images:debian/13/cloud

Trixie is an alias to debian/13/cloud

I can make an entirely new image but I would rather update the existing configured system to on the new ip address. It seems to want to hold onto the ip it was initiated with.

I would also be fine with being able to remove the cloud init all together and use a static interfaces style configuration.

Have you looked at /etc/netplan/ folder inside container?

Maybe try creating an entirely new instance and see what yaml file that creates there? The compare the two.

Let’s start with an actual reproducer please. If I try this:

incus init images:debian/13/cloud ezra
incus config set ezra cloud-init.network-config=- <<EOF
version: 2
ethernets:
  eth0:
    addresses:
      - 69.41.138.110/27
    gateway4: 69.41.138.97
    nameservers:
      addresses:
        - 69.41.138.98
        - 8.8.4.4
EOF
incus start ezra
incus exec ezra -- cloud-init status --wait
# status: done
incus list ezra
# +------+---------+----------------------+--------------------------------------------+-----------+-----------+
# | NAME |  STATE  |         IPV4         |                    IPV6                    |   TYPE    | SNAPSHOTS |
# +------+---------+----------------------+--------------------------------------------+-----------+-----------+
# | ezra | RUNNING | 69.41.138.110 (eth0) | XXXXXXXXXXXXXXXXXXXX:6aff:fed6:c2ec (eth0) | CONTAINER | 0         |
# +------+---------+----------------------+--------------------------------------------+-----------+-----------+
incus exec ezra -- cloud-init clean --seed --logs --configs all
incus stop ezra
incus config set ezra cloud-init.network-config=- <<EOF
version: 2
ethernets:
  eth0:
    addresses:
      - 69.41.138.102/27
    gateway4: 69.41.138.97
    nameservers:
      addresses:
        - 69.41.138.98
        - 8.8.4.4
EOF
incus config set ezra volatile.apply_template=create
incus start ezra
incus exec ezra -- cloud-init status --wait
# status: done
incus list ezra
# +------+---------+----------------------+---------------------------------------------+-----------+-----------+
# | NAME |  STATE  |         IPV4         |                    IPV6                     |   TYPE    | SNAPSHOTS |
# +------+---------+----------------------+---------------------------------------------+-----------+-----------+
# | ezra | RUNNING | 69.41.138.102 (eth0) | XXXXXXXXXXXXXXXXXXXXX:6aff:fed6:c2ec (eth0) | CONTAINER | 0         |
# +------+---------+----------------------+---------------------------------------------+-----------+-----------+

This all looks fine to me. (incus 7.0.1 on Ubuntu 22.04.5)

Maybe you needed cloud-init clean ... --configs all which deletes network config, as well as ssh_config and some other things.

EDIT: use --configs network if you only want to reconfigure the network, and don’t want to regenerate your ssh host keys.

Since most of my systems are statically defined on a /27 cloud init does just fine (until it doesn’t and we wind up here). I had been using netplan on ubuntu for years before canonical managed to make that less reliable and I just went old school /etc/network/interfaces. Debian out of the box these days favors nmcli (network manager :face_vomiting: ) which is a convoluted pile if I ever saw one. So I would prefer to just use the cloud image out of the box. But thank you.

The –configs flag was what I was missing Thank you!

root@tk2022:~# incus config edit ezra
root@tk2022:~# incus exec ezra -- cloud-init clean --seed --logs --configs network
root@tk2022:~# incus stop ezra
root@tk2022:~# incus config set ezra volatile.apply_template=create
root@tk2022:~# incus start ezra
root@tk2022:~# incus exec ezra -- cloud-init status --wait
.........................................................................................................................................................................................................................status: error

(....not really an error profile has a reboot in it....)

root@tk2022:~# incus exec ezra -- cloud-init status --wait
Error: Instance is not running
root@tk2022:~# incus exec ezra -- cloud-init status --wait
...........................................................................................................................................................................................................................................................................................................................................................................................................................................status: done
root@tk2022:~# incus list
+-----------+---------+------------------------------+------+-----------+-----------+
|   NAME    |  STATE  |             IPV4             | IPV6 |   TYPE    | SNAPSHOTS |
+-----------+---------+------------------------------+------+-----------+-----------+
.... other stuff ....
-----+
| ezra      | RUNNING | 69.41.138.102 (eth0)         |      | CONTAINER | 2         |
+-----------+---------+------------------------------+------+-----------+-----------+

It doesn’t configure the internal interface. Which I am curious about but the public facing is fixed and I can receive email again. As always this group of people are spot on and helpful. Thank you again.

So…. Since the above only seems to have configured the installed interface and ifupdown was required to configure both interfaces I believe that the right thing to do would be to delete cloud-init.network-config from the container and do the steps above. As described here. Again thank you for the help.

What do you mean by “the installed interface”? If I create a container with two interfaces (eth0/eth1), and I provide network-config with both eth0 and eth1, it works for me. (*)

I don’t know what you mean by “delete cloud-init.network-config from the container”. Do you mean to unset this incus setting? In that case cloud-init gets no network config information when it next runs. Or do you mean removing something from inside the container, in which case, what?

(Aside: cloud-init runs certain parts on first boot only, and certain parts on every boot. I think the network configuration is on every boot. If you don’t provide any network config, I think it will do some default like configure eth0 with DHCP, but I’ve not tested this)

I also see no need to install ifupdown2. Without it, Debian uses systemd-networkd, and cloud-init writes configs under /etc/systemd/network, and this all just works.

(*) Simple POC:

incus init images:debian/13/cloud ezra
incus config device add ezra eth1 nic network=bridge0
incus config set ezra cloud-init.network-config=- <<EOF
version: 2
ethernets:
  eth0:
    addresses:
      - 69.41.138.110/27
    gateway4: 69.41.138.97
    nameservers:
      addresses:
        - 69.41.138.98
        - 8.8.4.4
  eth1:
    addresses:
      - 192.168.31.141/24
EOF
incus start ezra
incus exec ezra -- cloud-init status --wait
# status: done
incus list ezra
# +------+---------+-----------------------+--------------------------------------------+-----------+-----------+
# | NAME |  STATE  |         IPV4          |                    IPV6                    |   TYPE    | SNAPSHOTS |
# |------+---------+-----------------------+--------------------------------------------+-----------+-----------+
# | ezra | RUNNING | 69.41.138.110 (eth0)  | fdfd::14ab (eth1)                          | CONTAINER | 0         |
# |      |         | 192.168.31.141 (eth1) | XXXXXXXXXXXXXXXXXXXX:6aff:fe05:adec (eth0) |           |           |
# +------+---------+-----------------------+--------------------------------------------+-----------+-----------+
incus exec ezra -- cloud-init clean --configs network
incus stop ezra
incus config set ezra cloud-init.network-config=- <<EOF
version: 2
ethernets:
  eth0:
    addresses:
      - 69.41.138.102/27
    gateway4: 69.41.138.97
    nameservers:
      addresses:
        - 69.41.138.98
        - 8.8.4.4
  eth1:
    addresses:
      - 192.168.31.123/24
EOF
incus config set ezra volatile.apply_template=create
incus start ezra
incus exec ezra -- cloud-init status --wait
# status: done
incus list ezra
# +------+---------+-----------------------+--------------------------------------------+-----------+-----------+
# | NAME |  STATE  |         IPV4          |                    IPV6                    |   TYPE    | SNAPSHOTS |
# +------+---------+-----------------------+--------------------------------------------+-----------+-----------+
# | ezra | RUNNING | 69.41.138.102 (eth0)  | fdfd::14ab (eth1)                          | CONTAINER | 0         |
# |      |         | 192.168.31.123 (eth1) | XXXXXXXXXXXXXXXXXXXX:6aff:fe05:adec (eth0) |           |           |
# +------+---------+-----------------------+--------------------------------------------+-----------+-----------+
ls /etc/systemd/network
# 10-cloud-init-eth0.network  10-cloud-init-eth1.network

This is what I meant by “please provide an actual reproducer”: a series of steps that anyone else can follow, which demonstrates the issue you are talking about. Using the above steps, I cannot find any problem with debian 13 + cloud-init + multiple network interfaces.

If you do have a problem, it probably means you’re doing something slightly different.

When I tried both interfaces as above it didn’t work. So I must have missed something.

I was working on a functioning server as part of a larger process so I didn’t do a lot of POCing.
You pointed me in the right direction and I was able to get it working. I will lab this more later.
Again, thank you.