Issue with lxc exec + dump to restic from stdin

Hello,

Before opening an issue on restic, I wanted to verify that I’m using lxc correctly.

I want to backup a dump to restic from a container via the LXD host.

The dump itself is working perfectly:

root@asuna ~# time lxc exec mysql -- mysqldump archive > dump.sql
0.13user 0.24system 0:00.93elapsed 40%CPU (0avgtext+0avgdata 19912maxresident)k
0inputs+40056outputs (0major+5428minor)pagefaults 0swaps
root@asuna ~# du -hs dump.sql
20M     dump.sql
root@asuna ~# lxc exec mysql -- mysqldump archive | head
-- MySQL dump 10.17  Distrib 10.3.13-MariaDB, for debian-linux-gnu (x86_64)
--
-- Host: localhost    Database: archive
-- ------------------------------------------------------
-- Server version       10.3.13-MariaDB-1:10.3.13+maria~stretch-log

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8mb4 */;

But piping the dump to restic does not work:

root@asuna ~# lxc exec mysql -- mysqldump archive | restic backup --verbose --stdin --stdin-filename archive.sql
open repository
repository 9e543735 opened successfully, password is correct
lock repository
load index files
using parent snapshot bad59229
read data from stdin
start scan on [archive.sql]
start backup on [archive.sql]
scan finished in 1.985s: 1 files, 0 B
[1:10] 0 files 0 B, total 1 files 0 B, 0 errors
/archive.sql

It just runs indefinitely. FYI it works fine when using mysqldump | restic directly.

I figured out the answer while creating the thread, but I will post it since it might be useful for others:

What I was missing was the -n option:

-n, --disable-stdin          Disable stdin (reads from /dev/null)

It works fine now:

root@asuna ~# lxc exec -n mysql -- mysqldump archive | restic backup --verbose --stdin --stdin-filename archive.sql
open repository
repository 9e543735 opened successfully, password is correct
lock repository
load index files
using parent snapshot 98577368
read data from stdin
start scan on [archive.sql]
start backup on [archive.sql]
scan finished in 2.090s: 1 files, 0 B

Files:           0 new,     1 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Data Blobs:      1 new
Tree Blobs:      1 new
Added to the repo: 609.105 KiB

processed 1 files, 0 B in 0:03
snapshot a869ff74 saved

I am having a similiar problem:

incus exec -n <instancename> -- "/usr/bin/docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres | gzip > /opt/stacks/immich/dbbackup/test"

Do you have an idea what I am doing wrong?

Thanks in advance!

incus exec with parameter --disable-stdin will disable the STDIN so that you can pipe the output of the command that runs in the instance, to some target on the host. Depending on the command for the piping, you may or may not need --disable-stdin.

In your case, you are just running this big process in the instance /usr/bin/docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres | gzip > /opt/stacks/immich/dbbackup/test. If that is the intention, then create some script in the instance, like

ubuntu@node1:~$ cat /usr/local/bin/mycommand.sh
/usr/bin/docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres | gzip > /opt/stacks/immich/dbbackup/test
ubuntu@node1:~$

and then you can just run it from the host as

$ incus exec node1 -- bash /usr/local/bin/mycommand.sh

If your intention was vastly different, like extracting the DB and saving on the host, then you do this like this.

$ incus exec node1 -- '/usr/bin/docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres' | gzip > /opt/stacks/immich/dbbackup/test

I have not tested these. But it should take you a bit further towards the solution.

1 Like

Thanks for the answer!

In the meantime I had the same idea! :slight_smile:
I do it this way.