File push/pull from/to storage volume?

Incus
1

Is it possible to push/pull files from a storage custom volume? If not, how can I access its content programatically?

2

You can mount the volume to the host filesystem using the normal tooling for the storage backend.

There might be another way using Incus directly. Maybe attach it to a temporary instance.

3

I am not sure on your use case, but I just thought that maybe syncing files to a bucket might be a better option. You mentioned doing things programmatically. If you use a bucket then anything that can read and write to an S3 bucket can be used.

4

You can install, for example, GitHub - scality/cloudserver: Zenko CloudServer, an open-source Node.js implementation of the Amazon S3 protocol on the front-end and backend storage capabilities to multiple clouds, including Azure and Google. into that container with the storage volume so that the storage can be accessed through S3.

1 Like
5

Incus also provides an S3 API compatible bucket implementation. I think it uses Minio for the backend.

If you want to write a bash script or similar, you could use rclone as the client. That is what I use for most of my work.

6

Oh, it’s this one, About storage pools, volumes and buckets - Incus documentation

Storage buckets provide object storage functionality via the S3 protocol.

They can be used in a way that is similar to custom storage volumes. However, unlike storage volumes, storage buckets are not attached to an instance. Instead, applications can access a storage bucket directly using its URL.

7

Bumping this topic up:

I use several application clusters as incus nodes. The simplest way to share configuration and shared data using shred volumes in ceph and cephfs storages. This give me several advantages, major ones are below:

  • Single point of configuration updates
  • Read-only configuration folders/files in application cluster nodes
  • Simple disaster recovery procedure as application cluster configuration and data stored separately of application code.

The only one disadvantage:

  • There is no way to access to shared volumes without adding additional instance with read-write access to these shared volumes.
    API for manipulation files/folders on storage volumes (type=filesystem) would help a lot.
8