How do I build a profile that sets-up containers correctly to run docker?

I have read How to run Docker inside LXD containers, and would like to have a profile that I can use during lxc launch that applies the configuration changes, and uses the correct storage.

I created a storage volume like this:

lxc storage create docker btrfs
❯ lxc storage list
+---------------------+--------+-------------------------------------------+-------------+---------+---------+
|        NAME         | DRIVER |                  SOURCE                   | DESCRIPTION | USED BY |  STATE  |
+---------------------+--------+-------------------------------------------+-------------+---------+---------+
| default             | zfs    | rpool/lxd                                 |             | 5       | CREATED |
+---------------------+--------+-------------------------------------------+-------------+---------+---------+
| docker              | btrfs  | /var/snap/lxd/common/lxd/disks/docker.img |             | 2       | CREATED |
+---------------------+--------+-------------------------------------------+-------------+---------+---------+
| workstation-default | zfs    | barra500/Ubuntu-Workstation/lxd-storage   |             | 99      | CREATED |
+---------------------+--------+-------------------------------------------+-------------+---------+---------+

My workstation file-systems are zfs.

I created a profile docker after inspecting the config of the container created in the exercise. It looks like this:

---
config:
  security.nesting: "true"
  security.syscalls.intercept.mknod: "true"
  security.syscalls.intercept.setxattr: "true"
description: ""
devices: {}
name: docker
used_by: {}

I ran it on a command-line like this:

lxc launch images:ubuntu/jammy --profile default --profile docker --storage docker $c1

The problem may be in translating:

lxc config device add $c1 docker disk pool=docker source=$c1 path=/var/lib/docker

Into the form of a profile. I can’t personally parse that line. Here it is directly from the instructions:

lxc config device add demo docker disk pool=docker source=demo path=/var/lib/docker

The cli process hangs during the lxc launch.
I am stuck as this point…

Unless someone has a simple solution readily to hand - I will avoid docker in lxc containers.

1 Like

Hello,

I had some success with the profiles present in this tutorial:

Take a look over at the differences between zfs/ext4 in the profiles there. There might be some hints in there for you.

1 Like

This video might help:

https://www.youtube.com/watch?v=_fCSSEyiGro

##lxc docker 
config:
  raw.lxc: |
    lxc.apparmor.profile=unconfined
    lxc.mount.auto=proc:rw sys:rw cgroup:rw
    lxc.cgroup.devices.allow=a
    lxc.cap.drop=
  security.nesting: "true"
  security.privileged: "true"
description: Default docker LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
  root:
    path: /
    pool: default
    type: disk
name: docker
used_by:
- /1.0/instances/happy-goose
lxc profile copy default docker - (do not edit the default copy it back it up and use a new one)
lxc profile edit docker
insert / save
lxc launch ubuntu
lxc assign docker ubuntu

Lxc profiles can be somewhat confusing to a new comer and the logic of how they work is also different so def check out the tuts on profiling in the interim here is a "complete example of a complete basic nested/privileged lxc profile… " modify to fit.
This has br0 instead lxdbr0 - basic dir/ext4 storage.

(just searched the net for a good few… I found allot of partially acceptable or very customized results via github && Ubuntu search articles on the main site include a portion of code (snippet only to set the config) https://ubuntu.com/tutorials/how-to-run-docker-inside-lxd-containers#2-create-lxd-container

Newcomers to lxd need to see exact complete code examples in order to gain perspective and learn/ snippets comments and one liner inserts, only add to confusion, fatigue, and more error.

(use with caution always test before prod.)
ref:
(Network doesn't work w/o `lxc.apparmor.profile = unconfined` in unprivileged containers - #2 by tomp

BigUps to the LXD/ubuntu team it has progressed so much. Thank you for all your efforts/
LXD rules Ubuntu <3! ~md

Its worth pointing out that the use of these two settings together effectively removes all isolation from the host, as it is trivial for the container to break out onto the host.

This is fine if you trust the workloads inside the container, but something to be aware of.

1 Like

@Tomp - Thanks for the input - Will docker run just as well unprivileged ? with those set to false?
thnx -)

I’m using the following profile for running Docker, it’s working well unprivileged for most cases (I never tried running K8S/Rancher inside, I think it will be a bit wonky in unprivileged mode).

name: docker
description: "Profile to configure instances for running Docker"
config:
  security.nesting: true
  security.syscalls.intercept.setxattr: true
  security.syscalls.intercept.mknod: true
  cloud-init.user-data: |
    #cloud-config
    write_files:
    - path: /etc/systemd/system/docker.service.d/10-listen-socket.conf
      owner: 'root:root'
      permissions: '0644'
      defer: true
      content: |
        # Generated by cloud-init
        #
        [Service]
        ExecStart=
        ExecStart=/usr/bin/dockerd -H unix:///run/docker.sock --containerd=/run/containerd/containerd.sock
    - path: /etc/docker/daemon.json
      owner: 'root:root'
      permissions: '0644'
      defer: true
      content: |
        {
          "iptables": true,
          "live-restore": true,
          "userland-proxy": false,
          "features": {
            "buildkit": true
          },
          "log-driver": "journald" 
        }
    apt:
      sources:
        docker-ce:
          source: 'deb [arch=amd64 signed-by=/etc/apt/trusted.gpg.d/docker-ce.gpg] https://download.docker.com/linux/ubuntu jammy stable' 
          key: <GPG KEY>
    package_update: true
    packages:
    - docker-ce
    - docker-ce-cli
    - docker-compose-plugin
    - docker-buildx-plugin
    - containerd.io
    runcmd:
    - systemctl daemon-reload
    - systemctl disable --now docker.socket
    - systemctl enable --now docker.service

NOTE: I also use ZFS in block mode with overlay2 driver, it works fine :slight_smile:

2 Likes