Images:debian/10 container image cannot setup networking for docker

LXD
1

I start a fresh images:debian/10 based LXD container, install docker inside, set required configuration (see below) and try to ping 8.8.8.8 from a Docker container. When the Docker container is run with [code]–privileged[code] the ping works. If not, the ping fails with ping: can't create raw socket: Permission denied.
All networking otherwise works OK.

  • debian/10 - doesn’t work for containers (but does work in a VM)
  • debian/9 - works
  • ubuntu/20.10 - works
  • ubuntu/18.04 - works

Am I missing some configuration options for the LXD container?

The LXD container has the unmodified default profile, and runs on the lxdbr0 network.

output of the pings from the inner Docker container:

# This works
$ lxc exec "$CONTAINER_ID" -- bash -c "docker run --rm --privileged busybox ping -c 4 8.8.8.8"
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
4c892f00285e: Pull complete 
Digest: sha256:e1488cb900233d035575f0a7787448cb1fa93bed0ccc0d4efc1963d7d72a8f17
Status: Downloaded newer image for busybox:latest
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=116 time=8.929 ms
64 bytes from 8.8.8.8: seq=1 ttl=116 time=7.968 ms
64 bytes from 8.8.8.8: seq=2 ttl=116 time=8.482 ms
64 bytes from 8.8.8.8: seq=3 ttl=116 time=8.301 ms

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 7.968/8.420/8.929 ms

# This fails
$ lxc exec "$CONTAINER_ID" -- bash -c "docker run --rm busybox ping -c 4 8.8.8.8"
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: can't create raw socket: Permission denied

full script to reproduce:

#! /bin/bash
CONTAINER_ID="test-docker"
lxc launch images:debian/10 "$CONTAINER_ID"

echo -e "\nInstall prerequisities into $CONTAINER_ID"
lxc exec "$CONTAINER_ID" -- sh -c 'apt update; apt install -y --no-install-recommends apt-transport-https ca-certificates curl gnupg-agent software-properties-common gnupg'
lxc exec "$CONTAINER_ID" -- sh -c 'curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -'
lxc exec "$CONTAINER_ID" -- sh -c 'add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"'

echo -e "\nInstall docker into $CONTAINER_ID"
lxc exec "$CONTAINER_ID" -- sh -c "apt update; apt install -y docker-ce docker-ce-cli containerd.io"
lxc exec "$CONTAINER_ID" -- sh -c "rm -rf /var/lib/apt/* /var/cache/apt/*"

# echo -e "\nStart $CONTAINER_ID"
# lxc start "$CONTAINER_ID"
lxc config set "$CONTAINER_ID" security.nesting="true" security.privileged="true"
echo -e "lxc.cgroup.devices.allow = a\nlxc.cap.drop =" | lxc config set "$CONTAINER_ID" raw.lxc -
lxc restart "$CONTAINER_ID"

echo -e "\nThis works"
lxc exec "$CONTAINER_ID" -- bash -c "docker run --rm --privileged busybox ping -c 4 8.8.8.8"
echo -e "\nThis fails"
lxc exec "$CONTAINER_ID" -- bash -c "docker run --rm busybox ping -c 4 8.8.8.8"
lxc stop "$CONTAINER_ID"
1 Like
2

When launching the container, can you try to append -c security.nesting=true?

3

Hi, thank you for such a quick reply:

I already have lxc config set "$CONTAINER_ID" security.nesting="true" security.privileged="true" in the script.
To be sure I added -c security.nesting="true" to the first launch invocation as well - but still with the same result.

This is the config of the container:

architecture: x86_64
config:
  image.architecture: amd64
  image.description: Debian buster amd64 (20210211_05:24)
  image.os: Debian
  image.release: buster
  image.serial: "20210211_05:24"
  image.type: squashfs
  image.variant: default
  raw.lxc: |
    lxc.cgroup.devices.allow = a
    lxc.cap.drop =
  security.nesting: "true"
  security.privileged: "true"
  volatile.base_image: 106ea503d427cfad57da01fedb09278428af3af64a6b750f0ddf17937f70c67c
  volatile.eth0.host_name: veth9e3ae778
  volatile.eth0.hwaddr: 00:16:3e:80:15:d5
  volatile.idmap.base: "0"
  volatile.idmap.current: '[]'
  volatile.idmap.next: '[]'
  volatile.last_state.idmap: '[]'
  volatile.last_state.power: RUNNING
  volatile.uuid: 98a120b6-77c9-4e94-b402-0f28b1509e49
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""
4

So I tried it on a fresh instance of ubuntu 20.04 in DigitalOcean just to be sure it’s not some faulty setting on my machine.

The problem is the same with debian/10, but the ping outside is working fine for debian/9, ubuntu/20.10 and ubuntu/18.04.

The same is true on my machine. It seems to be a problem with debian/10 then. Any thoughts?

P.S.: you have to change the download docker line in the reproduction script to work in both debian and ubuntu:

lxc exec "$CONTAINER_ID" -- sh -c 'add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/$(lsb_release -is | tr '[:upper:]' '[:lower:]') $(lsb_release -cs) stable"'
5

Here is my try, and it worked for me with the ubuntu:20.04 container image. I notice that it worked for you as well, but does not work on images:debian/10.

$ lxc launch ubuntu: mydocker -c security.nesting=true
Creating mydocker
Starting mydocker
$ lxc shell mydocker
root@mydocker:~# apt update 
...
root@mydocker:~# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
OK
root@mydocker:~# apt-key fingerprint 0EBFCD88
...
root@mydocker:~# add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
root@mydocker:~# apt install docker-ce
...
root@mydocker:~# docker run --rm busybox ping -c 3 8.8.8.8
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
4c892f00285e: Pull complete 
Digest: sha256:e1488cb900233d035575f0a7787448cb1fa93bed0ccc0d4efc1963d7d72a8f17
Status: Downloaded newer image for busybox:latest
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=115 time=82.381 ms
64 bytes from 8.8.8.8: seq=1 ttl=115 time=39.129 ms
64 bytes from 8.8.8.8: seq=2 ttl=115 time=80.403 ms

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 39.129/67.304/82.381 ms
root@mydocker:~#
6

Here is the failed attempt with images:debian/10.
The issue is more generic, it’s not a special issue with just raw sockets.

@Jan_Sourek, can you edit the title of the thread to something like images:debian/10 container image cannot setup networking for docker.

$ lxc launch images:debian/10 mydocker -c security.nesting=true
Creating mydocker
Starting mydocker
$ lxc shell mydocker
root@mydocker:~# apt update
...
root@mydocker:~# apt install -y curl gnupg2 lsb-release software-properties-common
...
root@mydocker:~# curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
OK
root@mydocker:~# apt-key fingerprint 0EBFCD88
...
root@mydocker:~# add-apt-repository    "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
root@mydocker:~# apt update
...
root@mydocker:~# apt install docker-ce docker-ce-cli containerd.io
...
root@mydocker:~# docker run --rm busybox ping -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
ping: can't create raw socket: Permission denied
root@mydocker:~# docker run --rm busybox wget www.google.com
wget: bad address 'www.google.com'
root@mydocker:~# docker run --rm busybox wget 216.58.206.196
Connecting to 216.58.206.196 (216.58.206.196:80)
wget: socket: Permission denied
7

@simos, I tried to find how to change the topic title, but I can’t find how - any help? Should I ask an admin?

8

There should be a pencil at the end of the title. By clicking that pencil, you can edit the title.
I did this change myself. If you want to further change the title, go for it.

My understanding with the images:debian/10 container image is that the whole networking is messed up. ping is a bit special because it uses raw icmp. An alternative test is to wget a file. You can see that name resolutions do not work either.

I think it would help if you can edit the initial post and summarize which container images do not let Docker work.

9

I probably don’t have the required rights to modify either the title (no pencil icon there) or older posts.

Good point with the ping and wget difference - but as you said, name resolution (and wget from IP as well) doesn’t work either in debian/10.

So from my research:
debian/10 - doesn’t work
debian/9 - works
ubuntu/20.10 - works
ubuntu/18.04 - works

Thank you for your time, Simos.

10

I’ll change it for you.

I’m not sure if this will help, but the debian 10 VM image works with docker as you need.

lxc launch images:debian/10 v1 --vm
11

Thank you - the VM approach is also an option, but I’ll probably stick with debian/9 for now.