Hi,lxc exec container -- ...lxc exec container -- command to be roughly equivalent to what I would get with:
lxc exec container -- bash
command
Actually to be fully compatible I should use lxc exec container -- bash -c command.
mcon@cinderella:~$ lxc exec yocto-builder -- ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/
aarch64
all
sa2150p_nand
mcon@cinderella:~$ lxc exec yocto-builder -- ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/a*
ls: cannot access '/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/a*': No such file or directory
mcon@cinderella:~$ lxc exec yocto-builder -- bash -c ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/a*
mcon@cinderella:~$ lxc exec yocto-builder -- bash -c 'ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/a*'
mcon@cinderella:~$ lxc exec yocto-builder -- bash
root@yocto-builder:~# ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/a*
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64:
acl-dbg_2.2.52-r0_aarch64.ipk
acl-dev_2.2.52-r0_aarch64.ipk
acl-doc_2.2.52-r0_aarch64.ipk
acl-staticdev_2.2.52-r0_aarch64.ipk
...
xz-doc_5.2.4-r0_aarch64.ipk
xz-staticdev_5.2.4-r0_aarch64.ipk
xz_5.2.4-r0_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/all:
autoconf-archive-doc_2018.03.13-r0_all.ipk
autoconf-archive_2018.03.13-r0_all.ipk
ca-certificates-dbg_20190110-r0_all.ipk
...
volatile-binds-dbg_1.0-r0_all.ipk
volatile-binds-dev_1.0-r0_all.ipk
volatile-binds_1.0-r0_all.ipk
root@yocto-builder:~# exit
mcon@cinderella:~$ lxc exec yocto-builder -- bash
root@yocto-builder:~# ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/sp*
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spitools-dbg_0.8.3-r0_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spitools-dev_0.8.3-r0_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spitools-doc_0.8.3-r0_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spitools_0.8.3-r0_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-embedded-mw-dbg_1.7.0-r1_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-embedded-mw-dev_1.7.0-r1_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-embedded-mw_1.7.0-r1_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-its-stack-dbg_1.2.3-r1_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-its-stack-dev_1.2.3-r1_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-its-stack-test_1.2.3-r1_aarch64.ipk
/home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/spoke-its-stack_1.2.3-r1_aarch64.ipk
root@yocto-builder:~# exit
mcon@cinderella:~$ lxc exec yocto-builder -- bash -c 'ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/sp*'
mcon@cinderella:~$
Notice that, even disregarding not working bash command apparent failure, actual listing in the container list the whole path from /, while from lxc command line only the basename is printed.
stgraber
January 26, 2022, 3:34am
2
Trying to reproduce here with a stock Ubuntu 20.04 container.
stgraber@dakara:~$ lxc exec u1 bash
root@u1:~# ls -1 /var/lib/apt/lists/sec*
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_InRelease
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_main_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_main_i18n_Translation-en
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_multiverse_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_multiverse_i18n_Translation-en
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_restricted_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_restricted_i18n_Translation-en
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_universe_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_universe_i18n_Translation-en
Now trying some ways to list this through lxc exec.
stgraber@dakara:~$ lxc exec u1 -- ls -1 /var/lib/apt/lists/sec*
ls: cannot access '/var/lib/apt/lists/sec*': No such file or directory
^ That’s expected as what makes this work in a shell is the shell expansion of the sec*, as lxc exec doesn’t invoke a shell, it’s trying to find a litteral file called /var/lib/apt/lists/sec* which doesn’t exist.
stgraber@dakara:~$ lxc exec u1 -- bash -c "ls -1 /var/lib/apt/lists/sec*"
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_InRelease
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_main_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_main_i18n_Translation-en
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_multiverse_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_multiverse_i18n_Translation-en
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_restricted_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_restricted_i18n_Translation-en
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_universe_binary-amd64_Packages
/var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_focal-security_universe_i18n_Translation-en
This works just fine, however:
stgraber@dakara:~$ lxc exec u1 -- bash -c ls -1 /var/lib/apt/lists/sec*
stgraber@dakara:~$
This fails because LXD will pass a list of argument like:
bash
-c
ls
-1
/var/lib/apt/lists/sec*
To the kernel, rather than what you need which is:
bash
-c
ls -1 /var/lib/apt/lists/sec*
stgraber
January 26, 2022, 3:35am
3
This doesn’t really line up with your own output though as you did attempt what should have been the correct syntax.
It may be worth running lxc monitor in a separate shell to see exactly what gets passed as the request to the server.
Thanks for the hint.
lxc exec yocto-builder -- bash -c "ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/sp*"
which seems to be parsed “strangely”:
location: none
metadata:
class: websocket
created_at: "2022-01-26T08:53:56.955915402+01:00"
description: Executing command
err: ""
id: 52b963ae-687f-4174-a403-0a8ebee09500
location: none
may_cancel: false
metadata:
command:
- bash
- -c
- ls
- "-1"
- /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/sp*
environment:
HOME: /root
LANG: C.UTF-8
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TERM: xterm-256color
USER: root
fds:
"0": 42d07e515e3fd39aab219a0e838584e6806694edb405278e549485f7fccd82c1
control: 335c8c2f767180e04f8b349591bcd47d51daa1093140ba54e09a6719e1e0949e
interactive: true
resources:
containers:
- /1.0/containers/yocto-builder
instances:
- /1.0/instances/yocto-builder
status: Pending
status_code: 105
updated_at: "2022-01-26T08:53:56.955915402+01:00"
timestamp: "2022-01-26T08:53:56.958783812+01:00"
type: operation
So I tried escaping the quotes with even stranger results:
mcon@cinderella:~$ lxc exec yocto-builder -- bash -c \"ls -1 /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/sp*\"
-1: -c: line 0: unexpected EOF while looking for matching `"'
-1: -c: line 1: syntax error: unexpected end of file
this is relevant lxc monitor output (I can attach the whole file, if deemed useful):
location: none
metadata:
class: websocket
created_at: "2022-01-26T08:55:30.69608113+01:00"
description: Executing command
err: ""
id: 8825807c-1ae0-4c1d-86a4-996ad97222fb
location: none
may_cancel: false
metadata:
command:
- bash
- -c
- '"ls'
- "-1"
- /home/ubuntu/builds/workdir/build/tmp-glibc/deploy/ipk/aarch64/sp*"
environment:
HOME: /root
LANG: C.UTF-8
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
TERM: xterm-256color
USER: root
fds:
"0": efaff45cc9c0365d8bc6ccd12d2e1002c0a1e2b6ab4fe98b937ab26951b4e69a
control: 21c2130e4c46e72b242e52317559f30e568b099bc3a6d2a091dba82bef8b5d4f
interactive: true
resources:
containers:
- /1.0/containers/yocto-builder
instances:
- /1.0/instances/yocto-builder
status: Pending
status_code: 105
updated_at: "2022-01-26T08:55:30.69608113+01:00"
timestamp: "2022-01-26T08:55:30.700359257+01:00"
type: operation
It seems argument parsing completely ignore quoting, for some reason.
Many Thanks in Advance
I should add this LXD installation is self-built on a fully-up-to-date Linux Mint host.might introduce some degree of variation.apparently without any trouble.
Normal way to install LXD is through snapd which might not be a wonderful idea for several reasons, including disk access efficiency.
This drives through installing it “from source” on a reasonably recent (post-2018) Linux machine.
preliminaries:
check you have prerequisite packages (this assumes Debian derivative distro):sudo apt update
sudo apt install acl autoconf dnsmasq-base git golang libacl1-dev libcap-dev \
liblxc1 liblxc-dev libtool libuv1-dev make pkg-config rsync squashfs-tools \
tar tcl xz-utils liblz4-dev libsqlite3-dev
sudo apt install lvm2 thin-provisioning-tools btrfs-progs zfsutils-linux
NOTE: in plain Debian (not Ubuntu) the package liblxc-dev is called lxc-dev.
install “recent” golang:
remove old golang (if installed):sudo apt remove --autoremove golang
sudo rm -rf /usr/local/go
install golang (>= 1.17):cd /tmp
wget https://go.dev/dl/go1.17.5.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go*.linux-amd64.tar.gz
echo "export PATH=$PATH:/usr/local/go/bin" >> ~/.profile
echo "export GOPATH=~/.go" >> ~/.profile
source ~/.profile
make prerequisites:cd /tmp
git clone https://github.com/lxc/lxd
sudo mv lxd /usr/local/src
cd /usr/local/src/lxd
make deps
compile lxd:make deps. Putting them at the end of your ~/.bashrc is completely optional (the following lines were on my specific PC):export CGO_CFLAGS="-I/home/mcon/.go/deps/raft/include/ -I/home/mcon/.go/deps/dqlite/include/"
export CGO_LDFLAGS="-L/home/mcon/.go/deps/raft/.libs -L/home/mcon/.go/deps/dqlite/.libs/"
export LD_LIBRARY_PATH="/home/mcon/.go/deps/raft/.libs/:/home/mcon/.go/deps/dqlite/.libs/"
export CGO_LDFLAGS_ALLOW="(-Wl,-wrap,pthread_create)|(-Wl,-z,now)"
make
install in a global directory:cd /usr/local/bin
cat <<EOF >/tmp/lx-helper
#!/bin/sh
export LD_LIBRARY_PATH=$(go env GOPATH)/deps/raft/.libs/:$(go env GOPATH)/deps/dqlite/.libs/
$(go env GOPATH)/bin/\$(basename \$0) \$*
EOF
chmod +x /tmp/lx-helper
sudo mv /tmp/lx-helper .
for f in lxc lxd fuidshift generate lxc-to-lxd lxd-agent lxd-p2c
do
sudo ln -s lx-helper $f
done
setup systemd support:cat <<EOF | sudo tee /etc/systemd/system/lxd.service
[Unit]
Description=LXD
[Service]
Environment="LD_LIBRARY_PATH=$(go env GOPATH)/deps/dqlite/.libs/:$(go env GOPATH)/deps/raft/.libs/"
ExecStart=$(go env GOPATH)/bin/lxd --group sudo
Restart=on-failure
Type=simple
[Install]
WantedBy=multi-user.target
EOF
NOTE: depending on your distribution you might need to add root:100000:65536 to both /etc/subuid and /etc/subgid
initialize LXD:lxd init --minimal
NOTE: full configuration of lxd is outside the scope of this recipe and depends heavily on resources available in host (in my case I created the default storage pool in a zfs zpool).
start daemon:sudo systemctl enable lxd && sudo systemctl start lxd
Perusing my own recipe I see problem probably stems from lxd being a symlink to a helper script.$* is probably too naive.
stgraber
January 26, 2022, 6:17pm
6
Yeah, $* would cause this kind of issue, $@ is the one to use to properly keep the arguments separate.