Bucket Object Expiration

LXD
#1

Is there a way to set up an Object Expiration policy for a storage bucket? I’d like to have a bucket that expires older objects after a certain amount of time.

#2

So it’s worth keeping in mind that LXD itself has no visibility on the objects stored within a bucket. That’s handled by whatever backend is in use, minio for local storage or ceph-rgw for remote storage.

It looks like Ceph at least supports setting the expiration rule through the S3 API. I don’t know if minio does too and if that uses the exact same API.

Maybe do a few tests poking the S3 API to add expiration rules and see if that works with whatever backend you have.

#3

Ok, I was able to do this through the mc command (I’m using minio). Per https://min.io/docs/minio/linux/administration/object-management/create-lifecycle-management-expiration-rule.html

lxc storage bucket create default first
mc alias set first https://lxd1:8555 <access_key> <secret_key>
mc ilm rule add first/first --expire-days "1"
mc ilm rule ls first/first
┌───────────────────────────────────────────────────────────────────────────────────────┐
│ Expiration for latest version (Expiration)                                            │
├──────────────────────┬─────────┬────────┬──────┬────────────────┬─────────────────────┤
│ ID                   │ STATUS  │ PREFIX │ TAGS │ DAYS TO EXPIRE │ EXPIRE DELETEMARKER │
├──────────────────────┼─────────┼────────┼──────┼────────────────┼─────────────────────┤
│ ch7rlo3djg0irph21520 │ Enabled │ -      │ -    │              1 │ false               │
└──────────────────────┴─────────┴────────┴──────┴────────────────┴─────────────────────┘

The mc tool seems to give me most of the control I need, including providing anonymous access to the buckets. Unfortunately some of the admin commands fail. Is this by design, or can they be made to work through the LXD proxy?

─❯ mc admin trace first
mc: <ERROR> Unable to listen to http trace. Failed to parse server response (unexpected end of JSON input):.

─❯ mc admin info first
mc: <ERROR> Unable to get service status
#4

I think this is expected because LXD provides the common S3 API of MinIO or Ceph radosgw and not the implementation specific admin APIs of each.