How to set up an lxd container internet/network gateway, in order to allow other containers to access the Internet through this gateway

LXD
1

There are 1 debian-gateway-cli and 2 Ubuntu 22.04 lxd containers:

Internet (through NAT) ← debian-gateway-cli ← internet-gateway ← end-user

If I remove the debian-gateway container from the network chain and connect the internet-gateway container via the default lxdbr0 bridge, then using the ping utility, one of the problems on the internet-gateway is visible:
There is a problem with traffic forwarding between 2, eth1 and eth0 interfaces in the internet-gateway container
I’m also sure that I need to add route, but I don’t know exactly how to do it correctly

root@nternet-gateway:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=21.2 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=27.5 ms

root@nternet-gateway:~# ping -I eth1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) from 172.16.0.1 eth1: 56(84) bytes of data.
From 172.16.0.1 icmp_seq=1 Destination Host Unreachable

root@nternet-gateway:~# ping -I eth1 10.63.169.1
PING 10.63.169.1 (10.63.169.1) from 172.16.0.1 eth1: 56(84) bytes of data.
From 172.16.0.1 icmp_seq=1 Destination Host Unreachable
From 172.16.0.1 icmp_seq=2 Destination Host Unreachable

root@nternet-gateway:~# ping -I eth1 10.63.169.200 - IP of eth0 (dhcp 4)
PING 10.63.169.200 (10.63.169.200) from 172.16.0.1 eth1: 56(84) bytes of data.
From 172.16.0.1 icmp_seq=1 Destination Host Unreachable
From 172.16.0.1 icmp_seq=2 Destination Host Unreachable

debian-gateway-cli:
The debian-gateway is a fully working container, there are no problems with it.
It redirected all our traffic through TOR, which makes it difficult to use the troubleshooting ping utility
root@debian-gateway:~# ifconfig -a
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.2.15 netmask 255.255.255.0 broadcast 10.0.2.255
ether 00:16:3e:ba:92:9e txqueuelen 1000 (Ethernet)
RX packets 18225 bytes 12375456 (11.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 15522 bytes 5415235 (5.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.152.152.10 netmask 255.255.192.0 broadcast 10.152.191.255
ether 00:16:3e:82:79:28 txqueuelen 1000 (Ethernet)
RX packets 2510 bytes 149307 (145.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2468 bytes 6426402 (6.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions

internet-gateway:
~$ lxc config show internet-gateway
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 22.04 LTS amd64 (release) (20240125)
image.label: release
image.os: ubuntu
image.release: jammy
image.serial: “20240125”
image.type: squashfs
image.version: “22.04”
volatile.base_image: 18124c86c222348c858ff33dab3e8afcf218353747172d93f4ad8c5a827c014e
volatile.cloud-init.instance-id: 241c2986-6ecc-47e2-87fd-67922c90582e
volatile.eth0.host_name: veth3a0e49a7
volatile.eth0.hwaddr: 00:16:3e:a6:77:56
volatile.eth1.host_name: vetha30094ef
volatile.eth1.hwaddr: 00:16:3e:c4:9e:ed
volatile.eth1.name: eth1
volatile.idmap.base: “0”
volatile.idmap.current: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.idmap.next: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.last_state.idmap: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.last_state.power: RUNNING
volatile.last_state.ready: “false”
volatile.uuid: 3d1bc1d9-fbea-434e-9167-a3217162cbcb
volatile.uuid.generation: 3d1bc1d9-fbea-434e-9167-a3217162cbcb
devices: {}
ephemeral: false
profiles:

  • who-profile-client
    stateful: false
    description: “”

~$ lxc profile show who-profile-client
config: {}
description: Default Whonix Client profile
devices:
eth0:
name: eth0
nictype: bridged
parent: who-clientbr0
type: nic
eth1:
nictype: bridged
parent: vpn-clientbr0
type: nic
root:
path: /
pool: default
type: disk
name: who-profile-client
used_by:

  • /1.0/instances/internet-gateway

~$ lxc network show who-clientbr0
config:
ipv4.address: 10.152.152.0/18
ipv4.dhcp.gateway: 10.152.152.10
ipv4.nat: “true”
ipv6.address: none
description: “”
name: who-clientbr0
type: bridge
used_by:

  • /1.0/instances/internet-gateway
  • /1.0/instances/debian-gateway
  • /1.0/profiles/who-profile-client
  • /1.0/profiles/who-profile-gat
    managed: true
    status: Created
    locations:
  • none

~$ lxc network show vpn-clientbr0
config:
ipv4.address: none
ipv6.address: none
description: “”
name: vpn-clientbr0
type: bridge
used_by:

  • /1.0/instances/end-user
  • /1.0/instances/internet-gateway
  • /1.0/profiles/enduser-profile-byo
  • /1.0/profiles/who-profile-client
    managed: true
    status: Created
    locations:
  • none

root@internet-gateway:~# netplan get
network:
version: 2
ethernets:
eth0:
dhcp4: true
eth1:
addresses:
- “172.16.0.1/24”
dhcp4: false
dhcp6: false
accept-ra: false

root@internet-gateway:~# ip route show dev eth0
default via 10.152.152.10 proto dhcp src 10.152.129.235 metric 100
10.152.128.0/18 proto kernel scope link src 10.152.129.235 metric 100
10.152.152.0 proto dhcp scope link src 10.152.129.235 metric 100
10.152.152.10 proto dhcp scope link src 10.152.129.235 metric 100

root@internet-gateway:~# ip route show dev eth1
172.16.0.0/24 proto kernel scope link src 172.16.0.1

Enabling IP forwarding in /etc/sysctl.conf:

root@internet-gateway:~# sysctl -p
net.ipv4.conf.all.forwarding = 1

Forwarding trafic between 2 interfaces using iptables:

root@internet-gateway:~# iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
root@vinternet-gateway:~# iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
root@internet-gateway:~# iptables-save > /etc/iptables.rules

To restore the “iptables” rules on reboot, create a file ‘/etc/network/if-pre-up.d/iptables’ with following content:

root@internet-gateway:~# nano /etc/network/if-pre-up.d/iptables

root@internet-gateway:~# cat /etc/network/if-pre-up.d/iptables
#!/bin/sh
/sbin/iptables-restore < /etc/iptables.rules

root@internet-gateway:~# chmod +x /etc/network/if-pre-up.d/iptables

root@internet-gateway:~# cat /etc/iptables.rules
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth1 -o eth0 -j ACCEPT
COMMIT

root@internet-gateway:~# ifconfig -a
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.152.129.235 netmask 255.255.192.0 broadcast 10.152.191.255
inet6 fe80::216:3eff:fea6:7756 prefixlen 64 scopeid 0x20
ether 00:16:3e:a6:77:56 txqueuelen 1000 (Ethernet)
RX packets 51 bytes 9923 (9.9 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 72 bytes 7330 (7.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.0.1 netmask 255.255.255.0 broadcast 172.16.0.255
inet6 fe80::216:3eff:fec4:9eed prefixlen 64 scopeid 0x20
ether 00:16:3e:c4:9e:ed txqueuelen 1000 (Ethernet)
RX packets 684 bytes 49888 (49.8 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 610 bytes 46976 (46.9 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions

end-user:
~$ lxc config show end-user
architecture: x86_64
config:
image.architecture: amd64
image.description: ubuntu 22.04 LTS amd64 (release) (20240126)
image.label: release
image.os: ubuntu
image.release: jammy
image.serial: “20240126”
image.type: squashfs
image.version: “22.04”
volatile.base_image: 9ab2ae1bf60aa4376701422f44fa37b6dcdad709385b87416ad0cf5d49398369
volatile.cloud-init.instance-id: b6f960be-2e31-4f5e-886f-68eb3f9c035a
volatile.eth0.host_name: veth55cc3141
volatile.eth0.hwaddr: 00:16:3e:08:9e:58
volatile.eth0.name: eth0
volatile.idmap.base: “0”
volatile.idmap.current: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.idmap.next: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.last_state.idmap: ‘[{“Isuid”:true,“Isgid”:false,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000},{“Isuid”:false,“Isgid”:true,“Hostid”:1000000,“Nsid”:0,“Maprange”:1000000000}]’
volatile.last_state.power: RUNNING
volatile.last_state.ready: “false”
volatile.uuid: 55d7f9ec-de86-469a-b683-1a4d9f6baec0
volatile.uuid.generation: 55d7f9ec-de86-469a-b683-1a4d9f6baec0
devices:
root:
path: /
pool: default
type: disk
ephemeral: false
profiles:

  • enduser-profile-byo
    stateful: false
    description: “”

root@end-user:~# ifconfig -a
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.0.10 netmask 255.255.255.0 broadcast 172.16.0.255
inet6 fe80::216:3eff:fe08:9e58 prefixlen 64 scopeid 0x20
ether 00:16:3e:08:9e:58 txqueuelen 1000 (Ethernet)
RX packets 1776 bytes 134413 (134.4 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2575 bytes 188915 (188.9 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

~$ lxc profile show enduser-profile-byo
config: {}
description: “”
devices:
eth0:
nictype: bridged
parent: vpn-clientbr0
type: nic
root:
path: /
pool: default
type: disk
name: enduser-profile-byo
used_by:

  • /1.0/instances/end-user

~$ lxc network show vpn-clientbr0
config:
ipv4.address: none
ipv6.address: none
description: “”
name: vpn-clientbr0
type: bridge
used_by:

  • /1.0/instances/end-user
  • /1.0/instances/internet-gateway
  • /1.0/profiles/enduser-profile-byo
  • /1.0/profiles/who-profile-client
    managed: true
    status: Created
    locations:
  • none

root@end-user:~# ip route show dev eth0
172.16.0.0/24 proto kernel scope link src 172.16.0.10

root@end-user:~# netplan get
network:
version: 2
ethernets:
eth0:
addresses:
- “172.16.0.10/24”
nameservers:
addresses:
- 172.16.0.1
routes:
- to: “default”
via: “172.16.0.10/24”

2

Welcome to Incus!

Can you please format the post with the appropriate markdown instructions?

Example:

incus launch images:debian/12/cloud mydebian

More formatting instructions: How to best ask questions on this discussion forum

2 Likes
3

Thank you for your attention. You, as the main one on this forum, were the first to respond to my post. To be honest, I have not installed incus yet, because it is a very new fork and it is often very difficult for a beginner to find specific information. I wanted to cheat a little by putting a post in this section. I really need the help of a specialist. Sorry.

4

You know, simos, I really respect the rules of this forum, because I teach virtualization from its posts and of course I agree with your comments on the structuring of my questions.
But, if a specialist wants to help, he immediately asks for the output of a specific command, as you often do in your answers. I wanted to convey a brief and clear overview of the problem.

5

@alex69 , you are asking for free support on a free software forum.

Since no one is obligated to provide you with an answer, whether they are an expert or not, I think it would be interesting for you to help those who are trying to help you by meeting simple requests to better format and structure your questions.

To lighten the mood a little, I remembered this gif:

help-me-help-you

2 Likes
6

I hope I’ve done well by making my post more informative and structured. I want to thank you for your friendly advice and your helpful comments

7

See the part about fenced code blocks.

This
image
looks like this:

$ incus launch images:debian/12/cloud mycontainer

or

$ incus launch images:debian/12/cloud mycontainer