Lxc-to-incus Failed to rsync: exit status 23

Incus
1

I have a lxc that I export to cluster. When moving the container I get an error
Creating container
Transferring container: bacula: 6.82GB (19.55MB/s)Skipping container ‘bacula’: Failed to rsync: exit status 23
rsync: read errors mapping “/var/lib/lxc/bacula/rootfs/var/lib/postgresql/10/main/base/510961/510985”: Input/output error (5)
rsync: read errors mapping “/var/lib/lxc/bacula/rootfs/var/lib/postgresql/10/main/base/510961/510988.6”: Input/output error (5)
rsync: read errors mapping “/var/lib/lxc/bacula/rootfs/var/lib/postgresql/10/main/base/510961/510985”: Input/output error (5)
rsync: read errors mapping “/var/lib/lxc/bacula/rootfs/var/lib/postgresql/10/main/base/510961/510988.6”: Input/output error (5): Input/output error (5)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1205) [sender=3.1.3]

Searching the forum for similar problems did not yield anything. How can I fix this?

2

Those look like actual disk read errors to me.
Are you seeing anything scary in sudo dmesg on that system?

3

No nothing. Looked over the last 24 hours, no errors. The disk was recently changed, the raid is fine with the UU.
Could this be due to some kind of space reservation on the area allocated to the virtualization? That’s the only hypothesis I didn’t bother to test. I have set the lv lxc 25GB space occupancy as we discussed recently

 incus profile device set default root size=30GiB

cat /var/log/incus/incusd.log

time="2024-02-21T14:11:43+03:00" level=warning msg="" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
time="2024-02-21T14:11:43+03:00" level=warning msg="" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored"
time="2024-02-21T14:11:44+03:00" level=warning msg="Failed to initialize fanotify, falling back on inotify" err="Failed to initialize fanotify: invalid argument"
4

Doesn’t seem likely as the errors are specifically on the sending side, not on the receiving side.

Can you try running:
cat /var/lib/lxc/bacula/rootfs/var/lib/postgresql/10/main/base/510961/510988.6 > /dev/null

To make sure that the file in question is in fact fully readable without errors?

5

I checked like you said. Input input error. Checked the raid again, it’s fine. I went into the container, turned off postgres, turned off the container, but got exactly the same result - output input error. Looks like a mystery. Thanks for the tip, now I realize that there is a hardware problem.

Workaround to make a dump of the base, exclude it from the transfer and after the transfer pour the dump.

6 
incus profile device set default root size=30GiB

lxc-to-incus --containers bacula --storage pool

lvdisplay
--- Logical volume ---
  LV Path                /dev/vg0/containers_bacula
  LV Name                containers_bacula
  VG Name                vgc
  LV UUID                nW4XvN-rPph-8uIH-kh0i-EyD9-qw8U-a3d0dg
  LV Write Access        read/write
  LV Creation host, time srv1, 2024-03-28 14:44:34 +0300
  LV Pool name           pool
  LV Status              available
  # open                 1
  LV Size                10,00 GiB
  Mapped size            2,24%
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:2

It’s on a server that’s in a cluster. In the default profile I see that 30gb value is set.