Issue Creating Cloud-init Profiles using Golang client

hachi · April 3, 2022, 1:09pm

I am using Golang client to create LXD profiles which some of them contain cloud-init settings. The once without cloud-init work flawlessly but with cloud-init not.

Here is a simple cloud-init conf:

config: 
  user.user-data: 
  #cloud-config
  ssh_pwauth: yes
  users:
    - name: deployer
      plain_text_passwd: "**********"
      lock_passwd: false
      groups: sudo
      shell: /bin/bash
      sudo: ALL=(ALL) NOPASSWD:ALL

I also tried with cloud-init.user-data but to no avail.

Here is the error message LXD version 4.24 is complaining about:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x868f54]

goroutine 1 [running]:
github.com/hamzaachi/lxd-compose/config.(*LXD).NewConn(0x978821)
/home/hachi/MyDATA/eadn/Projects/Programming/Go/lxd-compose/config/config.go:46 +0x14
main.main()
/home/hachi/MyDATA/eadn/Projects/Programming/Go/lxd-compose/main.go:89 +0x94
exit status 2

Any clue?

tomp · April 4, 2022, 10:53am

Looks like you’ve not initialized the config variable perhaps.

hachi · April 4, 2022, 9:24pm

Thank you for your reply. Here is my config.yaml and config.go which consumes and decodes config.yaml:

snap_local:
  socket: "/var/snap/lxd/common/lxd/unix.socket"

profiles:
  payaraGo:
    config: 
      user.user-data: 
      #cloud-config
      ssh_pwauth: yes
      users:
        - name: deployer
          plain_text_passwd: "********"
          lock_passwd: false
          groups: sudo
          shell: /bin/bash
          sudo: ALL=(ALL) NOPASSWD:ALL
    devices:
      config:
         source: cloud-init:config
         type: disk
      eth0:
        name: eth0
        nictype: bridged
        parent: lxdbr0
        type: nic
      root:
         path: /
         pool: default
         size: 60GB
         type: disk
    description: Payara specific configuration
    name: PayaraGO
    used_by:

instances:
  - name: test-ct
    profiles: [PayaraGO]
    source:
      type: image
      alias: ubuntu-ct-20
    type: container
  - name: test-vm
    source:
      type: image
      alias: ubuntu-vm-20
    type: virtual-machine

operations:
  test-vm:
    devices:
      eth0:
        nictype: bridged
        parent: lxdbr0
        ipv4.address: 10.232.131.222
        type: "nic"

config.go:

package config

import (
	"fmt"
	"os"

	lxd "github.com/lxc/lxd/client"
	"github.com/lxc/lxd/shared/api"
	yaml "gopkg.in/yaml.v3"
)

type LXD struct {
	Snap_local Snap                        `yaml:"snap_local"`
	Profiles   map[string]api.ProfilesPost `yaml:"profiles"`
	Instances  []api.InstancesPost         `yaml:"instances"`
	Operations map[string]Devices          `yaml:"operations"`
}

type Devices struct {
	Devs map[string]map[string]string `yaml:"devices"`
}
type Snap struct {
	Socket string `yaml:"socket"`
}

func NewConfig(path string) (conf *LXD, err error) {
	conf = &LXD{}

	yamlReader, err := os.Open(path)
	if err != nil {
		return nil, fmt.Errorf("error reading config file: %s", err)

	}
	defer yamlReader.Close()

	decoder := yaml.NewDecoder(yamlReader)

	if err = decoder.Decode(conf); err != nil {
		return nil, fmt.Errorf("error reading config file: %s", err)
	}
	return conf, nil
}

func (c *LXD) NewConn() (connection lxd.InstanceServer, err error) {

	connection, err = lxd.ConnectLXDUnix(c.Snap_local.Socket, nil)
	if err != nil {
		return nil, err
	}
	return connection, nil
}

I don’t really see where to initialize these variables if any

tomp · April 5, 2022, 8:22am

Check that this var isn’t nil:

hachi · April 5, 2022, 9:16am

I think i figured out the root cause of the issue, If i set cloud-init conf in the Profiles definition i cannot get any variables wihether c.Snap_local or others.

It seems Golang client cannot decode and render any YAML data with cloud-init, isn’t it because cloud-init is more complicated and cannot fit into Config map[string]string json:“config” yaml:“config” ` defined in ProfilePut here

github.com

lxc/lxd/blob/9178e1ed6648/shared/api/profile.go#L26

      
        
            // swagger:model
            type ProfilePost struct {
            	// The new name for the profile
            	// Example: bar
            	Name string `json:"name" yaml:"name"`
            }
            
            
// ProfilePut represents the modifiable fields of a LXD profile
            //
            // swagger:model
            type ProfilePut struct {
            	// Instance configuration map (refer to doc/instances.md)
            	// Example: {"limits.cpu": "4", "limits.memory": "4GiB"}
            	Config map[string]string `json:"config" yaml:"config"`
            
            
	// Description of the profile
            	// Example: Medium size instances
            	Description string `json:"description" yaml:"description"`
            
            
	// List of devices
            	// Example: {"root": {"type": "disk", "pool": "default", "path": "/"}, "eth0": {"type": "nic", "network": "lxdbr0", "name": "eth0"}}

tomp · April 5, 2022, 9:20am

The cloud-init config is expected to be a string of yaml in the config value field, not embedded yaml.

hachi · April 5, 2022, 9:31am

I see, this is my problem then. Thanks @tomp i will see how to inject it as a string

olmax99 · June 15, 2022, 9:44am

I am also using the Golang client and trying to start a container instance with a custom cloud-init or any possible InstanceUpdate() for that matter. I believe this Topic kind of relates closest by avoiding to create an entirely new topic for my question.

I’ve created a test case:

userdata := api.InstancePut{
		Config: map[string]string{"limits.cpu": "4",},
}
if _, etag, err := c.GetInstance(expected); err != nil {
		t.Fatal(err)
} else {
		op, err := c.UpdateInstance(expected, userdata, etag)
		if err != nil {
				t.Fatal(err)
			}
			err = op.Wait()
			if err != nil {
				t.Fatal(err)
			}
		}

Myunderstanding of how idmaps work in LXC is most certainly lacking, so please forgive if I am missing the obvious here. The following error is thrown (./lxd/instance/drivers/driver_lxc.go:4190):

lxc_test.go:148: Volatile idmap keys can't be deleted by the user

This is happening for cloud-init (“user.vendor-data”) or any Config, Profile, etc change.
There is no problem whatsoever using the standard cli method:

lxc config set test-container limits.cpu=2

Help appreciated.

tomp · June 15, 2022, 9:47am

This is a recent change to detect and prevent making the error that your program is making, which is to attempt to replace (PUT) the instance’s config without reading the instance’s current config and merging it with your desired changes first.

The volatile idmap keys are created by LXD on instance creation, and you are effectively requesting they be removed via your UpdateInstance request, which would render the container unusable.

olmax99 · June 15, 2022, 9:49am

Actually, that makes a lot of sense. Thx a lot.

For the sake of completeness, I want to leave the implementation:

ic, etag, err := client.GetInstance(expected)
if err != nil {
		t.Error(err)
}
o := NewConfigRead(ic)
o.Read()  // initializes all functions from ConfigReader interface (see below)
o.ILocConf["limits.cpu"] = "4"  // add new config entry

req := api.InstancePut{
		Architecture: o.ILocConf["image.architecture"],
		Config:       o.ILocConf,
		Devices:      o.ICurr.ExpandedDevices,
		Ephemeral:    false,
		Profiles:     []string{"default"},
		Stateful:     false,
		Description:  "updated instance",
}
op, err := c.UpdateInstance(o.ICurr.Name, req, etag)
if err != nil {
		t.Fatal(err)
}
err = op.Wait()
	if err != nil {
		t.Fatal(err)
}

ConfigRead fulfills the ConfigReader interface:

type ConfigReader interface {
	Project() string
	Type() instancetype.Type
	Architecture() int
	ExpandedConfig() map[string]string
	ExpandedDevices() deviceConfig.Devices
	LocalConfig() map[string]string
	LocalDevices() deviceConfig.Devices
}

corresponding an example struct:

type ConfigRead struct {
	ICurr *api.Instance

	IProject string
	IType    api.InstanceType
	IArch    int
	IExpConf map[string]string
	IExpDevs deviceConfig.Devices
	ILocConf map[string]string
	ILocDevs deviceConfig.Devices
}

func NewConfigRead(obj *api.Instance) *ConfigRead {
	ins := &ConfigRead{}
	ins.ICurr = obj
	return ins
}
...
func (ic *ConfigRead) Read() ConfigRead {..}