When "incus list" overlaps with "incus exec"

This is a weird gotcha which had me scratching my head for a while. I’ve solved it now, but I thought it was worth noting in case it saves anyone else some time.

The following command shows all containers:

incus list -f csv -c n status=RUNNING | while read a; do
  echo "$a"; done

But this one executes a command on the first container only, then terminates:

incus list -f csv -c n status=RUNNING | while read a; do
  incus exec "$a" -- hostname; echo OK; done

The “echo OK” is to prove that the shell doesn’t immediately terminate at that point. It also doesn’t make a difference if I put “false” or “true” there.

If I add --debug to “incus list” I can see that the full set of containers is returned by the API, but still only the first iteration occurs.

SOLUTION: it’s because “incus exec” is consuming from stdin. It works properly if I do:

incus list -f csv -c n status=RUNNING | while read a; do
  incus exec "$a" -- hostname </dev/null; echo OK; done

or:

incus list -f csv -c n status=RUNNING | while read a; do 
  incus exec -n "$a" -- hostname; echo OK; done

or:

while read -ru9 a; do
  incus exec "$a" -- hostname; echo OK
done 9< <(incus list -f csv -c n status=RUNNING)

incus list -fcsv -cn status=RUNNING | xargs -I% incus exec % hostname

1 Like

Thanks. Although if you actually want stdin to be connected to the terminal, for example because apt-get -y dist-upgrade still may prompt interactively, the xargs way doesn’t work. The best solution I have is still:

while read -ru9 a; do
  echo "=== $a ==="
  incus exec "$a" -- apt-get -y dist-upgrade
done 9< <(incus list -f csv -c n status=RUNNING)

although something like this should work:

xargs -a /dev/fd/9 -I% incus exec % -- apt-get -y dist-upgrade 9< <(incus list -fcsv -cn status=RUNNING)

Sure! xargs is kinda my go-to tool for simple loops because I never know what to write after read :slight_smile:
For more complex uses, a good ol’ while is hard to beat.