Delay in mounting files in OCI

Incus
1

I’m trying to configure an OCI container (atmoz/sftp) to bind directories at startup. The container has an option to place scripts in /etc/sftp.d/ which are executed at boot time by the entrypoint using the following snippet from /entrypoint :

# Source custom scripts, if any
if [ -d /etc/sftp.d ]; then
    for f in /etc/sftp.d/*; do
        if [ -x "$f" ]; then
            log "Running $f ..."
            $f
        else
            log "Could not run $f, because it's missing execute permission (+x)."
        fi
    done
    unset f
fi

When I initially build and start an instance, the above doesn’t spot my script in /etc/sftp.d/, and my bind mounts aren’t created. But if I subsequently restart the container, my script is executed and the bind mounts work. This suggests that at the time of initially running /entrypoint, /etc/sftp.d/ is either empty or doesn’t exist.

I’m creating the content of this directory with OpenTofu as follows:

resource "incus_instance" "sshd" {
  project = data.terraform_remote_state.project.outputs.name
  profiles = [incus_profile.default.name]
  name  = "sftp"
  image = incus_image.sftp.fingerprint

  config = {
    "boot.autostart" = true
    "environment.SFTP_USERS" = "data::1000"
  }

  file {
    content = file("./bindmount.sh")
    target_path = "/etc/sftp.d/bindmount.sh"
    create_directories = true
  }
  ...

Note that I also tried to create an /etc/sftp/users.conf file using similar OpenTofu code, and while the file existed in the container, the entrypoint failed to pick it up. I had to resort to adding the user with an environment variable instead, which is fine for a single user, but less so if I had many. This suggests the same problem.

My thought are that there’s a delay in mounting the file within the container, so the entrypoint script misses it on the first run. By the time I restart the container, the file has been mounted, so the entrypoint spots it. Is this a bug, or have I missed something obvious?

As a workaround, I can add the following to the incus_instance block:

provisioner "local-exec" {
    command = "incus --project ${data.terraform_remote_state.project.outputs.name} restart ${self.name}"
  }

This restarts the instance as soon as it’s built, at which point my script runs.

2

Can you maybe file an issue at GitHub - lxc/terraform-provider-incus: Incus provider for Terraform/OpenTofu to extend the file section to include an option of when to perform the file creation?

In the vast majority of cases, we actually should be handling file push/creation PRIOR to the instance being started up. That’s with the exception of VMs where we unfortunately don’t have a way to mess with their root disk until they’re started and the agent is running.

I think @breml actually ran into something similar before and also wanted a bit more flexibility.

Basically I see two ways to handle the issue here:

  1. As mentioned, we add an option to control when to perform the file creation with the default for containers being to do it prior to startup which should handle this for you.
  2. Add a good way to tell Incus that a change to that file needs to restart the instance to have it be applied. This would then write the file on the running instance and immediately restart it. This approach would also work with VMs which is convenient but it would cause a pointless restart for containers and may cause some issues by having the container boot at least once without the expected file.
3

Thank you. I’ve raised Extend the file section to include an option of when to perform the file creation · Issue #335 · lxc/terraform-provider-incus · GitHub

4

Yes, I have had a similar situation with an OCI container, where the service expected the presence of a configuration file, but the container it self was shipped without this file. In this case, the user has been expected to provide the file via bind mount (docker). If I remember correctly, the container would stop so quickly, that there was not even time for incus / terraform to inject a file, so I think in this case, option 2 would not work.