Unable to start container - read-only file system

d24lau · October 4, 2020, 6:33pm

Hello,

When I attempt to run lxc start marmoset03-submit-01 I am getting the following error:

[d24lau-adm@marmoset03 home]$ lxc start marmoset03-submit-01

Error: Common start logic: chown /var/snap/lxd/common/lxd/containers/marmoset03-submit-01: read-only file system

Try `lxc info --show-log marmoset03-submit-01` for more info

Also, if I try to create a new snapshot for backup purposes, lxc snapshot marmoset03-submit-01 backup_08 I get the following error:

[d24lau-adm@marmoset03 home]$ lxc snapshot marmoset03-submit-01 backup_08

Error: Failed to create file '/var/snap/lxd/common/lxd/containers/marmoset03-submit-01/backup.yaml': open /var/snap/lxd/common/lxd/containers/marmoset03-submit-01/backup.yaml: read-only file system

Running the suggested lxc info --show-log marmoset03-submit-01:

[d24lau-adm@marmoset03 home]$ lxc info --show-log marmoset03-submit-01
Name: marmoset03-submit-01
Location: none
Remote: unix://
Architecture: x86_64
Created: 2019/12/17 22:01 UTC
Status: Stopped
Type: container
Profiles: default
Snapshots:
  apache2_setup (taken at 2019/12/17 22:18 UTC) (stateless)
  mysql_setup (taken at 2019/12/17 22:33 UTC) (stateless)
  marmoset_cloned (taken at 2019/12/18 15:29 UTC) (stateless)
  database_setup (taken at 2019/12/18 15:54 UTC) (stateless)
  submitserver_configured (taken at 2019/12/18 16:56 UTC) (stateless)
  marmoset_builded (taken at 2019/12/18 17:25 UTC) (stateless)
  mod_jk_installed (taken at 2019/12/18 18:04 UTC) (stateless)
  certificates_installed (taken at 2019/12/18 18:36 UTC) (stateless)
  mod_auth_cas_installed (taken at 2019/12/18 20:57 UTC) (stateless)
  test_dpkg (taken at 2019/12/19 21:06 UTC) (stateless)
  before_ME_101_projects_uploaded (taken at 2020/01/03 14:58 UTC) (stateless)
  backup_03 (taken at 2020/02/25 15:43 UTC) (stateless)
  backup_06 (taken at 2020/08/18 14:51 UTC) (stateless)
  backup_07 (taken at 2020/09/29 19:25 UTC) (stateless)

Log:

Which as you can see has nothing within the “Log:” portion.

Running lxc storage info default:

[d24lau-adm@marmoset03 home]$ lxc storage info default
info:
  description: ""
  driver: lvm
  name: default
  space used: 31.32GB
  total space: 34.85GB
used by:
  images:
  - 39a93d0b355279d430e8ce21c689aa88515212ee99874276e77f7f31ad7bf810
  instances:
  - marmoset-build-ece-06
  - marmoset-build-mme-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  - marmoset03-submit-01
  profiles:
  - default

The disk space seems to indicate a couple GB available now. Although this is after I deleted some snapshots. When I first started looking into this, there was less than 500MB of free space indicated by this.

It may have something to do with running out of storage space somewhere, but I’m not sure where to look. The container was working fine yesterday.

Any suggestions or help would be appreciated.

Thanks,
David Lau

stgraber · October 4, 2020, 7:16pm

Look at dmesg on your system. A filesystem flipping to read-only is usually a sign of something being quite wrong with your storage.

d24lau · October 4, 2020, 7:35pm

OK, here are the last few lines from dmesg:

[3382549.854486] lxdbr0: port 1(veth05448831) entered disabled state

[3382549.854577] device veth05448831 entered promiscuous mode

[3382549.875751] device veth05448831 left promiscuous mode

[3382549.875766] lxdbr0: port 1(veth05448831) entered disabled state

[3382580.536642] IPv6: ADDRCONF(NETDEV_UP): veth8087ec82: link is not ready

[3382580.545488] lxdbr0: port 1(veth8087ec82) entered blocking state

[3382580.545492] lxdbr0: port 1(veth8087ec82) entered disabled state

[3382580.545589] device veth8087ec82 entered promiscuous mode

[3382580.762179] EXT4-fs (dm-7): mounted filesystem with ordered data mode. Opts: discard

[3382580.920898] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready

[3382580.921869] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

[3382580.921923] IPv6: ADDRCONF(NETDEV_CHANGE): veth8087ec82: link becomes ready

[3382580.921946] lxdbr0: port 1(veth8087ec82) entered blocking state

[3382580.921950] lxdbr0: port 1(veth8087ec82) entered forwarding state

[3382886.344380] IPv6: ADDRCONF(NETDEV_UP): veth9a2bfedc: link is not ready

[3382886.354300] lxdbr0: port 2(veth9a2bfedc) entered blocking state

[3382886.354305] lxdbr0: port 2(veth9a2bfedc) entered disabled state

[3382886.354394] device veth9a2bfedc entered promiscuous mode

[3382886.373224] device veth9a2bfedc left promiscuous mode

[3382886.373262] lxdbr0: port 2(veth9a2bfedc) entered disabled state

[3382908.227700] IPv6: ADDRCONF(NETDEV_UP): veth2e3293b0: link is not ready

[3382908.238020] lxdbr0: port 2(veth2e3293b0) entered blocking state

[3382908.238026] lxdbr0: port 2(veth2e3293b0) entered disabled state

[3382908.238117] device veth2e3293b0 entered promiscuous mode

[3382908.261176] device veth2e3293b0 left promiscuous mode

[3382908.261236] lxdbr0: port 2(veth2e3293b0) entered disabled state

[3383189.180378] IPv6: ADDRCONF(NETDEV_UP): vethdebc50d4: link is not ready

[3383189.189242] lxdbr0: port 2(vethdebc50d4) entered blocking state

[3383189.189246] lxdbr0: port 2(vethdebc50d4) entered disabled state

[3383189.189319] device vethdebc50d4 entered promiscuous mode

[3383189.206480] device vethdebc50d4 left promiscuous mode

[3383189.206518] lxdbr0: port 2(vethdebc50d4) entered disabled state

[3383572.140022] IPv6: ADDRCONF(NETDEV_UP): veth9a05dcce: link is not ready

[3383572.149043] lxdbr0: port 2(veth9a05dcce) entered blocking state

[3383572.149048] lxdbr0: port 2(veth9a05dcce) entered disabled state

[3383572.149155] device veth9a05dcce entered promiscuous mode

[3383572.168430] device veth9a05dcce left promiscuous mode

[3383572.168474] lxdbr0: port 2(veth9a05dcce) entered disabled state

[3383785.767032] EXT4-fs (dm-8): write access unavailable, skipping orphan cleanup

[3383785.767039] EXT4-fs (dm-8): recovery complete

[3383785.767046] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: discard

I’m not quite sure what I’m looking at. So I also ran dmesg | grep EXT4-fs to get the following:

[3378525.808229] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416259 (offset 2256896 size 4096 starting block 751654)

[3378525.808257] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416257 (offset 2248704 size 4096 starting block 752677)

[3378525.808277] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416257 (offset 0 size 0 starting block 752676)

[3378531.248368] EXT4-fs warning: 2 callbacks suppressed

[3378531.248377] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523559 (offset 2260992 size 4096 starting block 1322023)

[3378535.824448] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416258 (offset 2260992 size 4096 starting block 751144)

[3378535.824471] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416258 (offset 0 size 0 starting block 751143)

[3378535.824494] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416262 (offset 2277376 size 4096 starting block 749611)

[3378536.256474] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523557 (offset 2273280 size 4096 starting block 1296938)

[3378540.832628] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416252 (offset 0 size 0 starting block 748076)

[3378540.832663] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416253 (offset 2314240 size 4096 starting block 736821)

[3378540.832683] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416253 (offset 0 size 0 starting block 736820)

[3378545.840770] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416255 (offset 0 size 0 starting block 748588)

[3378555.857142] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416251 (offset 2265088 size 4096 starting block 750633)

[3378555.857173] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416251 (offset 0 size 0 starting block 750632)

[3378555.857194] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416254 (offset 0 size 0 starting block 750121)

[3378555.857325] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416256 (offset 2252800 size 4096 starting block 752165)

[3378555.857375] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416250 (offset 2301952 size 4096 starting block 728113)

[3378556.289155] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523560 (offset 2256896 size 4096 starting block 1322534)

[3378556.289184] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523558 (offset 2265088 size 4096 starting block 1326632)

[3378556.289211] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 394181 (offset 2260992 size 4096 starting block 1332263)

[3378556.289233] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 263095 (offset 2256896 size 4096 starting block 1334310)

[3378556.289251] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523513 (offset 2256896 size 4096 starting block 1332774)

[3378556.289280] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523514 (offset 2256896 size 4096 starting block 1334822)

[3378560.865270] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416246 (offset 2338816 size 4096 starting block 723002)

[3378560.865318] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416249 (offset 2297856 size 4096 starting block 728625)

[3378560.865338] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416249 (offset 0 size 0 starting block 728624)

[3378560.865364] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416250 (offset 0 size 0 starting block 728114)

[3378560.865387] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416260 (offset 0 size 0 starting block 749100)

[3378560.865413] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416256 (offset 0 size 0 starting block 752166)

[3378561.297317] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523514 (offset 0 size 0 starting block 1334823)

[3378561.297345] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 394181 (offset 0 size 0 starting block 1332264)

[3378561.297364] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523513 (offset 0 size 0 starting block 1332775)

[3378561.297380] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 263095 (offset 0 size 0 starting block 1334311)

[3378570.881612] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416258 (offset 2265088 size 4096 starting block 751144)

[3378570.881649] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416261 (offset 2281472 size 4096 starting block 747564)

[3378570.881669] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416262 (offset 2281472 size 4096 starting block 749612)

[3378570.881693] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416248 (offset 0 size 0 starting block 747059)

[3378570.881708] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416259 (offset 0 size 0 starting block 751655)

[3378570.881722] EXT4-fs warning (device dm-7): ext4_end_bio:302: I/O error -28 writing to inode 416257 (offset 2252800 size 4096 starting block 752677)

[3378576.321822] EXT4-fs warning (device dm-8): ext4_end_bio:302: I/O error -28 writing to inode 523559 (offset 0 size 0 starting block 1322024)

[3378991.758474] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: discard

[3380555.414978] EXT4-fs (dm-7): mounted filesystem with ordered data mode. Opts: discard

[3382580.762179] EXT4-fs (dm-7): mounted filesystem with ordered data mode. Opts: discard

[3383785.767032] EXT4-fs (dm-8): write access unavailable, skipping orphan cleanup

[3383785.767039] EXT4-fs (dm-8): recovery complete

[3383785.767046] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: discard

Does this tell you anything more about what is going on?

stgraber · October 4, 2020, 7:42pm

Could be the thin pool running out of disk space and causing some issues.
I’m not very familiar with LVM, but I’d recommend making sure you have a bunch of free space on the VG and thinpool. Then ideally stop the containers and run fsck on each of the LVs before bringing things back up.

The failure mode of ext4 is to go read-only, so that part makes sense. The only way to flip it back to writable is to unmount and remount, ideally after running a test.

d24lau · October 4, 2020, 8:11pm

OK, I can see how this might have been a case of “thin pool” running out of disk space. When I ran sudo lvs at the start, it showed Data% as being at 100% for LXDThinPool. This is what it shows now:

[d24lau-adm@marmoset03 ~]$ sudo lvs
  LV                                                                      VG                Attr       LSize   Pool        Origin                                                            Data%  Meta%  Move Log Cpy%Sync Convert
  root                                                                    centos            -wi-ao---- 185.82g                                                                                                                      
  swap                                                                    centos_marmoset03 -wi-ao---- 119.24g                                                                                                                      
  LXDThinPool                                                             default           twi-aotz-- <32.46g                                                                               86.26  3.63                            
  containers_marmoset--build--ece--05                                     default           Vwi---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset--build--ece--06                                     default           Vwi-aotz-k  10.00g LXDThinPool                                                                   44.59                                  
  containers_marmoset--build--mme--01                                     default           Vwi---tz-k  <9.32g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01                                       default           Vwi-aotz-k  10.00g LXDThinPool containers_marmoset03--submit--01-certificates_installed          72.90                                  
  containers_marmoset03--submit--01-apache2_setup                         default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-backup_03                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--01-backup_04                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--01-backup_05                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--01-backup_06                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--01-before_ME_101_projects_uploaded       default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--01-certificates_installed                default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-database_setup                        default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-marmoset_builded                      default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-marmoset_cloned                       default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-mod_auth_cas_installed                default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--01-mod_jk_installed                      default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-mysql_setup                           default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-submitserver_configured               default           Vri---tz-k  10.00g LXDThinPool                                                                                                          
  containers_marmoset03--submit--01-test_dpkg                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--05                                       default           Vwi---tz-k  <9.32g LXDThinPool containers_marmoset03--submit--01                                                                        
  containers_marmoset03--submit--05-apache2_setup                         default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-apache2_setup                                                          
  containers_marmoset03--submit--05-backup_03                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-backup_03                                                              
  containers_marmoset03--submit--05-backup_04                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-backup_04                                                              
  containers_marmoset03--submit--05-backup_05                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-backup_05                                                              
  containers_marmoset03--submit--05-before_ME_101_projects_uploaded       default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-before_ME_101_projects_uploaded                                        
  containers_marmoset03--submit--05-certificates_installed                default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-certificates_installed                                                 
  containers_marmoset03--submit--05-database_setup                        default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-database_setup                                                         
  containers_marmoset03--submit--05-marmoset_builded                      default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-marmoset_builded                                                       
  containers_marmoset03--submit--05-marmoset_cloned                       default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-marmoset_cloned                                                        
  containers_marmoset03--submit--05-mod_auth_cas_installed                default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-mod_auth_cas_installed                                                 
  containers_marmoset03--submit--05-mod_jk_installed                      default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-mod_jk_installed                                                       
  containers_marmoset03--submit--05-mysql_setup                           default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-mysql_setup                                                            
  containers_marmoset03--submit--05-submitserver_configured               default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-submitserver_configured                                                
  containers_marmoset03--submit--05-test_dpkg                             default           Vri---tz-k  10.00g LXDThinPool containers_marmoset03--submit--01-test_dpkg                                                              
  images_a92eaa65a5c5e53c6bf788b4443f4e5d2afac1665486247c336aa90959522bb6 default           Vwi---tz-k  <9.32g LXDThinPool

I’m not quite sure what you mean when you say “unmount and remount”. Do you mean that I need to unmount and remount LXDThinPool, and that I should run fsck on LXDThinPool in between the unmount and remount?

-----edit-----

From my previous experience unmounting and remounting on a Ubuntu host (this is a Centos host), here are the commands that are my best guess at doing this:

For the unmount:
sudo umount /var/snap/lxd/common/lxd/storage-pools/default

For the remount:
sudo mount /var/snap/lxd/common/lxd/disks/default.img /var/snap/lxd/common/lxd/storage-pools/default

Does this look like I’m on the right track?

-----edit-----

No, the umount didn’t work:

[d24lau-adm@marmoset03 ~]$ sudo umount /var/snap/lxd/common/lxd/storage-pools/default

umount: /var/snap/lxd/common/lxd/storage-pools/default: not mounted

-----edit-----

Here is part of what lsblk shows:

sdi                                                       8:128  0    1.8T  0 disk  
├─sdi1                                                    8:129  0    1.6T  0 part  
│ └─md127                                                 9:127  0    8.1T  0 raid5 /home
└─sdi2                                                    8:130  0  202.9G  0 part  
  └─md126                                                 9:126  0 1013.8G  0 raid5 /scratch
loop0                                                     7:0    0     55M  1 loop  /var/lib/snapd/snap/core18/1880
loop1                                                     7:1    0   96.6M  1 loop  /var/lib/snapd/snap/core/9804
loop2                                                     7:2    0   55.3M  1 loop  /var/lib/snapd/snap/core18/1885
loop3                                                     7:3    0   68.6M  1 loop  /var/lib/snapd/snap/lxd/17320
loop4                                                     7:4    0   70.6M  1 loop  
loop5                                                     7:5    0   68.6M  1 loop  /var/lib/snapd/snap/lxd/17497
loop6                                                     7:6    0   34.5G  0 loop  
├─default-LXDThinPool_tmeta                             253:2    0      1G  0 lvm   
│ └─default-LXDThinPool-tpool                           253:4    0   32.5G  0 lvm   
│   ├─default-LXDThinPool                               253:5    0   32.5G  0 lvm   
│   ├─default-containers_marmoset03----submit----01     253:6    0     10G  0 lvm   
│   └─default-containers_marmoset----build----ece----06 253:7    0     10G  0 lvm   
└─default-LXDThinPool_tdata                             253:3    0   32.5G  0 lvm   
  └─default-LXDThinPool-tpool                           253:4    0   32.5G  0 lvm   
    ├─default-LXDThinPool                               253:5    0   32.5G  0 lvm   
    ├─default-containers_marmoset03----submit----01     253:6    0     10G  0 lvm   
    └─default-containers_marmoset----build----ece----06 253:7    0     10G  0 lvm   
loop7                                                     7:7    0   97.1M  1 loop  /var/lib/snapd/snap/core/9993
loop8                                                     7:8    0   59.6G  0 loop  
├─pool02-LXDThinPool_tmeta                              253:8    0      1G  0 lvm   
│ └─pool02-LXDThinPool                                  253:10   0   57.6G  0 lvm   
└─pool02-LXDThinPool_tdata                              253:9    0   57.6G  0 lvm   
  └─pool02-LXDThinPool                                  253:10   0   57.6G  0 lvm

So I ran fsck on default-containers_marmoset03----submit----01 and it just shows

[root@marmoset03 disks]# fsck default-containers_marmoset03----submit----01

fsck from util-linux 2.23.2

Since fsck didn’t report any errors, am I to assume that things look OK with that volume?

d24lau · October 5, 2020, 1:03am

In the end, after freeing up space I just rebooted the entire server, which cleared the read-only state.

Working on creating a new storage pool to offload a container to the new storage pool.

Thanks for your responses Stephane!