Setting a static IP address using network bridge

1

I am trying to set the IP address of my container to 192.168.1.70 (on production server, i want to do the same with a public IP address) so that it is visible on my home network. It is not working for me, not reachable and no internet.

Any help would be appreciated. thanks.

Here is what I did:

Create network interface

$ sudo nmcli con add ifname custombr1 type bridge con-name custombr1
$ sudo nmcli con add type bridge-slave ifname wlx28ee52172bc master custombr1
$ sudo nmcli con up custombr1

Check ifconfig

$ ifconfig
custombr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        ether 0a:23:ad:aa:22:2d  txqueuelen 1000  (Ethernet)
        RX packets 73  bytes 23128 (23.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 260 (260.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I created a profile and then added this to the container

$ lxc info apache
Name: apache
Location: none
Remote: unix://
Architecture: x86_64
Created: 2021/05/08 07:15 UTC
Status: Running
Type: container
Profiles: custom-default, custom-bridged
Pid: 61775
Ips:
  eth0:	inet6	fe80::216:3eff:fe74:bffe	veth8440bec7
  lo:	inet	127.0.0.1
  lo:	inet6	::1
Resources:
  Processes: 12
  Disk usage:
    root: 16.09MB
  CPU usage:
    CPU usage (in seconds): 0
  Memory usage:
    Memory (current): 9.43MB
    Memory (peak): 10.92MB
  Network usage:
    eth0:
      Bytes received: 10.61kB
      Bytes sent: 21.11kB
      Packets received: 201
      Packets sent: 71
    lo:
      Bytes received: 0B
      Bytes sent: 0B
      Packets received: 0
      Packets sent: 0

This is what the profile shows

$ lxc profile show custom-bridged
config: {}
description: Custom Bridged Network Profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: custombr1
    type: nic
name: custom-bridged
used_by:
- /1.0/instances/apache

This is the other profile that is also attached to the instance

$ lxc profile show custom-default
config: {}
description: ""
devices:
  root:
    path: /
    pool: default
    type: disk
name: custom-default
used_by:
- /1.0/instances/apache

$ lxc list
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
|   NAME    |  STATE  |         IPV4          |                     IPV6                     |   TYPE    | SNAPSHOTS |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| apache    | RUNNING |                       |                                              | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| web-app   | RUNNING | 10.173.158.156 (eth0) | fd42:6ed:2d75:d56b:216:3eff:fe58:cbd1 (eth0) | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
2

Does it work when the external interface is not a WiFi device? I think you will gave trouble bridging onto a WiFi device because wifi’s auth uses the client Mac and so doesn’t play nicely when multiple macs are in use.

3

Using the routed nic type could work better in this situation as it would share the WiFi devices Mac and wouldn’t need a bridge.

4

Thanks for your fast response, i will test with the lan cable.

5

I have connected up using the LAN cable, deleted the connection, recreated and restarted the container, still no joy.

jsharief@desktop:~$ sudo nmcli con show
NAME                UUID                                  TYPE      DEVICE       
Wired connection 2  e7a6fcb1-e873-30c7-ab88-9bf0c245527e  ethernet  vethf68b1efc 
Wired connection 1  3024e066-fce0-3917-a075-588311a66365  ethernet  eno1         
lxdbr0              9c54c69a-fe64-4de3-8ea4-67697afafbcc  bridge    lxdbr0       
custombr0            5642f604-8099-48dc-91fb-c473426a1641  bridge    custombr0     
INTERNET_PROVIDER_PLUS_4698  f637a454-cd2f-4282-aac5-f6532b5821a7  wifi      --           
Wired connection 3  c631ef8b-9842-376d-983f-f92e44a907af  ethernet  --           
jsharief@desktop:~$ sudo nmcli con add ifname custombr1 type bridge con-name custombr1
Connection 'custombr1' (bcb166e2-69ac-4724-ac7f-65cdfaca02d1) successfully added.
jsharief@desktop:~$ sudo nmcli con add type bridge-slave ifname eno1 master custombr1
Connection 'bridge-slave-eno1' (148dbaa0-cbdb-4ec5-9e6c-3089051b158c) successfully added.
jsharief@desktop:~$ sudo nmcli con up custombr1
Connection successfully activated (master waiting for slaves) (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/26)
jsharief@desktop:~$ sudo nmcli con show
NAME                UUID                                  TYPE      DEVICE       
Wired connection 2  e7a6fcb1-e873-30c7-ab88-9bf0c245527e  ethernet  vethf68b1efc 
custombr1            bcb166e2-69ac-4724-ac7f-65cdfaca02d1  bridge    custombr1     
Wired connection 1  3024e066-fce0-3917-a075-588311a66365  ethernet  eno1         
lxdbr0              9c54c69a-fe64-4de3-8ea4-67697afafbcc  bridge    lxdbr0       
custombr0            5642f604-8099-48dc-91fb-c473426a1641  bridge    custombr0     
INTERNET_PROVIDER_PLUS_4698  f637a454-cd2f-4282-aac5-f6532b5821a7  wifi      --           
Wired connection 3  c631ef8b-9842-376d-983f-f92e44a907af  ethernet  --           
bridge-slave-eno1   148dbaa0-cbdb-4ec5-9e6c-3089051b158c  ethernet  --           
jsharief@desktop:~$ lxc list
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
|   NAME    |  STATE  |         IPV4          |                     IPV6                     |   TYPE    | SNAPSHOTS |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| apache    | RUNNING |                       |                                              | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| custom-app | RUNNING | 10.173.158.156 (eth0) | fd42:6ed:2d75:d56b:216:3eff:fe58:cbd1 (eth0) | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
jsharief@desktop:~$ lxc list
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
|   NAME    |  STATE  |         IPV4          |                     IPV6                     |   TYPE    | SNAPSHOTS |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| apache    | RUNNING |                       |                                              | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| custom-app | RUNNING | 10.173.158.156 (eth0) | fd42:6ed:2d75:d56b:216:3eff:fe58:cbd1 (eth0) | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
jsharief@desktop:~$ lxc restart apache
jsharief@desktop:~$ lxc list
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
|   NAME    |  STATE  |         IPV4          |                     IPV6                     |   TYPE    | SNAPSHOTS |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| apache    | RUNNING |                       |                                              | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
| custom-app | RUNNING | 10.173.158.156 (eth0) | fd42:6ed:2d75:d56b:216:3eff:fe58:cbd1 (eth0) | CONTAINER | 0         |
+-----------+---------+-----------------------+----------------------------------------------+-----------+-----------+
6

How have you configured the IP statically inside the container? Or are you planning to rely on an external DHCP server?

7

I am setting the IP address of the container using a patch request.

This morning i have powered up and eventually (there is a delay maybe 1 minute) the containers are assigned a static ip address, but not the one i set with LXC.

e.g. I set 192.168.1.100 , I get 192.168.1.43 , so it seems that I am setting the IP but i am being assigned by DHCP.

8

I can now access the container on my local network, and I can ping from the container out , so you were correct about the WIFI connection.

It seems to work this morning after powering up the machine, so it just needed some restart care.

The problem now is just the fact that there is large delay after starting the container for it to get the IP address, probably more than 30 seconds.

9

The IP set in LXD device config only applies if the device is connected to a lxd managed bridge not an external network.

10

I totally misunderstood, i thought i needed a bridged network to set a static ip (public or private) IP address.

11

When you do lxc config device set <instance> <nic> ipv{n}.address=<address> LXD creates a static DHCP allocation in its own DHCP server (dnsmasq) for the instance’s MAC address.

So in cases where your instance’s NIC is connected to an unmanaged bridge (that may or may not be connected to the external network), then LXD’s DHCP server isn’t being used and as such the static IP settings in LXD’s config are not consulted.

So with that in mind, and leaving aside any specific implementation details, please can you describe what it is you’re trying to achieve?

Do you just want to get an instance on to the external network with a static IP?

12

Here at home, i want to run various web applications in instances, and i want these to be available over the local network, e.g. 192.168.1.X.

Then i want to install LXD on a baremetal server, and have a separate public IP address for mail server, and possibly add other applications with their own public IP addresses. Not all containers, just when needed.

13

If you don’t need the containers that are connected to the external network to be able to communicate with the LXD host (and vice versa) then you can just use a macvlan NIC type, specifying the external interface as the parent. This way you won’t need any unmanaged bridges at all.

lxc config device add <instance> eth0 nic nictype=macvlan parent=<external interface>

Then inside the container, configure its static IP as you would with a normal system.

See https://linuxcontainers.org/lxd/docs/master/instances#nic-macvlan

If you need to the containers to be able to communicate with the host then you will be limited to using either bridged or routed NIC types. The former requires setting up an unmanaged bridge and connecting it to the external interface (see https://netplan.io/examples/#configuring-network-bridges) and then using a bridged NIC type with the parent set to the bridge interface, e.g.:

lxc config device add <instance> eth0 nic nictype=bridged parent=<external interface>

Then inside the container, configure its static IP as you would with a normal system.

See https://linuxcontainers.org/lxd/docs/master/instances#nic-bridged

Keep in mind that for both macvlan and bridged NIC types, the instances will get their own MAC address on the external network (as if they are physically connected to it), and in some production environments (and sometimes when running inside another hypervisor) the number of different MAC addresses per physical port is limited to one.

To get around this we have the routed NIC type which shares the MAC address of the host’s external interface and doesn’t require a separate unmanaged bridge be setup.

The routed NIC type can also be useful in environments where the upstream network doesn’t provide a dedicated subnet of IPs to use, and instead routes non-sequential IPs to your LXD host.

To add a routed NIC type use:

lxc config device add <instance> eth0 nic nictype=routed parent=<external interface> ipv{n}.address=<static ip>

Then inside the container, you need to ensure that either A) the container’s network setup doesn’t remove the static IP and routing added to the NIC by LXD or B) set up the network config to it mirrors the setup added by LXD.

See https://linuxcontainers.org/lxd/docs/master/instances#nic-routed

See How to get LXD containers get IP from the LAN with routed network

14

Thank you for this detailed answer explaining the types. I had tried to routed nic command but it came back as an error, i found in one of your older posts you added nic after the eth0, this allowed the command to run but still can’t access, going to try again now.

15

Ah yes I’ve updated that with “nic” now.

16

When you say communicate with the host, are you referring to communicating with other containers?

17

Nope, the host (i.e where the LXD process is running). Other containers are fine with macvlan.

18

Interesting, thanks.

19

I need my apache container to access the mysql container, i created this profile, dropped the eth0 device on the container, and applied it. The problem is macvlan no longer works, if add lxdbr0 as the second one i was able to get IPV6 address but not version 4. What am i doing wrong?

config: {}
description: ""
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  eth1:
    name: eth1
    nictype: macvlan
    parent: wlx28ee52172bcc
    type: nic
name: multi

$ lxc list
+-----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
|   NAME    |  STATE  |         IPV4          |                     IPV6                      |   TYPE    | SNAPSHOTS |
+-----------+---------+-----------------------+-----------------------------------------------+-----------+-----------+
| c2        | RUNNING | 10.177.108.109 (eth0) | fd42:6507:3321:8a12:216:3eff:fede:5a06 (eth0) | CONTAINER | 0         |
20

Can you show me the output of ip a and ip r inside the container, also can you show me the static network config (if any) you’re using inside the container.