Ubuntu, LXD and masquerade

LXD
1

i have problem with LXD masquerade and SNAT.

full description is there: SuperUser - LXD masquerade

this is little :slight_smile:
Host (Ubuntu + LXD) with multiple LXC containers.
Each container have own IP Lan address (eg. 10.10.1.141, 10.10.1.137).
Incoming packages - no problem, iptables rules and ok.

`iptables -t nat -A PREROUTING -d 10.10.1.141 -i ens160 -p udp --dport 53 -j DNAT --to-destination 10.71.85.107:53`

but what with outgoing packages? im using SNAT and POSTROUTING and ok!
iptables -t nat -I POSTROUTING -s 10.71.85.107 ! -d 10.71.85.0/24 -j SNAT --to-source 10.10.1.141

But after restart host, the first line in POSTROUTING is always:
330 27582 MASQUERADE all -- * * 10.71.85.0/24 !10.71.85.0/24 /* generated for LXD network lxdbr0 */

where can i change this, i need to use SNAT and specific ip address for each container

2

Hi!

The default configuration of LXD, with the lxdbr0 managed network device, gives you incoming/outgoing connectivity to the Internet. LXD adds for you the appropriate iptables rules.

It is not clear whether you really want to deviate from this configuration, and what you are doing really differently.

If you really want to do something different, then you do not use the LXD managed network interface, and you perform the configuration on your own. You setup in LXD not to autoconfigure for you the network interface. This is performed in the default profile of LXD (lxc profile edit default). Also, you can remove any managed network devices through the lxc network list commands.

3

you can find LXD documentation at

https://lxd.readthedocs.io

there is a search engine, entering iptables could lead you to the following article:

https://lxd.readthedocs.io/en/latest/api-extensions/#network95firewall95filtering

4

It is really simple what I want to do…
example from live:
one LXD host and multiple container:
DNS (10.10.1.141)
MAIL (10.10.1.138)
Web1 (10.10.1.145)
etc… (network /24)

some container have to go with own IP - specially DNS and Mail.

what you suggested - use different networks for each container?
it is posible to configure networks without iptables, just in lxd?

5

The default LXD setup (single private network lxdbr0) gives you what you describe here.

I do not understand what you mean with this,

Regarding this question,

It is possible to configure some containers without networking at all.
But to have them do something useful, you would need to create proxy devices.
For example, to expose the web server to the LAN, you would

lxc config device add mywebcontainer myproxy80 proxy listen=tcp:0.0.0.0:80 connect=tcp:127.0.0.1:80

With this command, any connections to the LXD host at port 80, will get proxied to port 80 on mywebcontainer.

6

What you need here is the destination NATs for inbound connectivity to LXD containers.

I use iptables-persistent for this.

One DNAT / port forward line per service then at the bottom the default masquerade for lxd outbound traffic.

nano /etc/iptables/rules.v4

### Plex server
-A PREROUTING -i enp2s0 -p tcp -m tcp --dport 32400 -j DNAT --to-destination 10.66.99.199:32400

-A PREROUTING -i enp2s0 -p tcp -m tcp --dport 19569 -j DNAT --to-destination 10.66.99.133:19569
### NAT all outbound to Internet
-A POSTROUTING -o enp2s0 -j MASQUERADE

iptables-restore < /etc/iptables/rules.v4

Thats if you actually want to nat.

If you have control of your upstream router then you can route your traffic directly to LXD thuse negate the need for any natting.,

e.g.
assuming your lxd host server is 10.10.1.100
on upstream router: ( could also be a l3 switch or a firewall if its at home)

static route:
ip route add 10.71.85.0/24 next-hop 10.10.1.100

Also remember to enable l3 routing/forwarding on the lxd host in sysctl.conf

you will need to nat the lxd network on the firewall as well, so remember the rule for that if there isn’t a catch all nat.

7

Also as stated by @simos you can also proxy the connections using the proxy command, or manually build a HAproxy container and forward the ports in to that which then in turn forwards to the “backend” containers.

8

sorry simos for my english - maybe that is problem.

Try to explain:
When create Virtual Machine on VMware, XEN, FreeBSD Jail, etc. you can attribute IP (public or LAN) for this VM.
All Incoming/Outgoing package have this IP.

It is possible to do this with LXD?
iptables can do this - Prerouting and Postrouting.
But MASQUARADE on first place destroy everything.

Example: if we have Host with 24 Container and all of this Container go to Router with the same IP address from Host. Masquerade do this.

It is possible to create LXD container with dedicated in/out IP address?

9

bodleytunes

i created iptables rules - incoming and outgoing and work fine.
But when restart host (or LXD) the line with MASQUERADE “jump” to the first line and destroy all POSTROUTING.

this is problem.

10

I assume that the LXD host is baremetal and not in a VM. Because if it is in a VM it needs some extra work.

You can do what you want using the macvlan network device, or the bridged network device.
See How to make your LXD containers get IP addresses from your LAN using macvlan – Mi blog lah! and How to make your LXD containers get IP addresses from your LAN using a bridge – Mi blog lah!

By doing so, each container gets its own IP address from the LAN, and LXD is not involved at all to give IP. If you want, you can set the IP statically inside the container.

11

simos - you are genius, really

for testing is VM - but in production will be bare metal.
im using your instruction with macvlan and work great (set Promiscuous mode in VMware network config)
Disabled DHCP and after created container just edit (in container) netplan file (ubuntu 18), set IP, Gateway and DNS from my LAN and working, working, working…

Restart LXD host and container again work fine - just like i need.

another question but maybe in different task - in this configuration, container have own iptables? can we run for example fail2ban? i will check it

simos - you make my year!

12

I use fail2ban in a container and it works all right.

13

Yeah fail2ban works fine.