Setting ipv4.routes with next hop?

LXD
1

When lxd creates a network bridge I want it to add static routes - but those static routes need a next hop (pointing to a container IP).

I can’t see the syntax to do this. The following fails:

# lxc network set br-dns ipv4.routes "10.10.4.0/22 via 10.10.0.224"
Error: Failed to run: ip -4 route add dev br-dns 10.10.4.0/22 via 10.10.0.224 proto static: \
Error: inet prefix is expected rather than "10.10.4.0/22 via 10.10.0.224".

Is it possible in lxd?

lxd lets me add routes without a next hop, but for me that’s not very useful. (The host will send ARP broadcasts for addresses within that range, but the host doesn’t have any IP address on that subnet for the target to reply to)

I am running Ubuntu 18.04 with lxd 3.0.3-0ubuntu1~18.04.1

FYI, here’s the rest of the bridge config:

# lxc network show br-dns
config:
  bridge.external_interfaces: eno1
  ipv4.address: 10.10.0.254/24
  ipv4.dhcp.ranges: 10.10.0.1-10.10.0.199
  ipv4.nat: "true"
  ipv6.address: fd42:b533:58da:47c5::1/64
  ipv6.nat: "true"
description: ""
name: br-dns
type: bridge
used_by: []
managed: true
status: Created
locations:
- none
2

I don’t understand what you’re trying to accomplish. Specifically, the expression next hop doesn’t mean anything to me. Can you elaborate on this? If you’re asking “is it possible to set up a bridge and assign static IP addresses to the containers using this bridge?” the answer is yes. I can post a simple recipe, but I’m guessing that’s not what you’re asking.

3

I suppose you are trying to keep the LXD managed network (and not do it manually with cloud-init) but make a tiny configuration change to use a different gateway.

You need to find how to instruct dnsmasq to set a different gateway, the try to pass that info to LXD’s dnsmasq. You should create a separate managed network interface for this, like lxdbr1.

4

" Routers are capable of managing hop counts, but other types of network devices (e.g. Ethernet hubs and bridges) are not."
Because Bridges are working on Layer 2. But the IP-Layer is Layer 3.

As Simos mentioned before, you have to create at least a second bridge manually, means you create an unmanged bridge. Then you are able to set all your needs in /etc/network/interfaces. But I am not sure this will fit all your metric needs.

1 Like
5

Have a look here:

If you go the default LXD-way creating containers by LXD-INIT you will set up a «managed container» with a managed bridge. DNSMASQ in combination with ip-tables-rules will take control of your containers automatically how to reach them.

In special use-cases you want to avoid those scenario, because you need more control what happens beyond the curtain. Therefore you have to set up an “unmanged container” from an LXD point of view. This means you have to take care about by yourself how your container is reachable (ip-addressing, gateway{=hop}, forwarding and so on), right?

1 Like
6

“Next hop” says what device to forward traffic to. The simplest example is a default route.

If your IP address is 1.1.1.1 and the router that you reach the rest of the Internet through is 1.1.1.254, then you would do something like

ip -4 route add 0.0.0.0/0 via 1.1.1.254

So if you’ve ever added a static route, you will have used this. These are bog-standard static routes.

No. I have multiple containers, each of which has an IP subnet behind it. (Imagine: each container is a router, and behind that router are other things. Actually more containers, but that doesn’t really matter)

So I need to add static routes to reach those behind-the-container networks.

1 Like
7

In that respect, you would try to avoid setting static routes.
Have a look at GitHub - FRRouting/frr: The FRRouting Protocol Suite
There has been a discussion on frr recently and @bodleytunes may be able to point you to the correct direction.

8

Ha ha :slight_smile:

In fact these containers are running FRR. Here’s a trimmed topology: it’s for a teaching classroom.

net-bb  10.10.0      .254
-----+-------+-------* host
     |.221   |.222
     |eth0   |eth0
    frr1    frr2
     |eth1   |eth1
 ----+---  --+-----
 10.1.0    10.2.0

So on the host I need route to 10.1.0.0/16 via 10.10.0.221, and 10.2.0.0/16 via 10.10.0.222.

net-bb is an lxd managed network, with local address 10.10.0.254.

But the answer to my original question seems to be “no, lxd doesn’t have a way to add a static route with next hop”

9

:slight_smile: and we learn that on comment #8.

LXD is a container hypervisor and offers a networking managed mode and a networking none mode. If you cannot figure out a way to do your work with managed mode, then you go for none and configure yourself.

In managed mode, LXD uses dnsmasq. Therefore, the question becomes, can I do what I need using dnsmasq? And, if I can do what I need with dnsmasq, does LXD offer a way to pass raw dnsmasq configuration to the managed interface? The answer to the second question is the configuration option raw.dnsmasq.

1 Like