How to sepcify Ceph/rbd user

LXD
1

Hello,
I’m trying to use the new ceph/rbd feature on LXD 2.16 and can’t figure out how to specify a user/id to access to my Ceph cluster.
I have one user in my keyring. I can use it with ceph/rbd command ( ie: rbd --id=qlf ls lxd-qlf is ok )
If i try to create a ceph stroage on lxd, i can’t tell it to use that ‘qlf’ id. It seems to use a default one, admin:

lxc storage create ceph-qlf ceph ceph.osd.pool_name=lxd-qlf

error: Error getting CEPH version: Failed to run: ceph version: 2017-07-26 16:45:31.472127 7efd4ca45700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication
2017-07-26 16:45:31.472131 7efd4ca45700 0 librados: client.admin initialization error (2) No such file or directory
Error connecting to cluster: ObjectNotFound

Do i miss something?

Thanks

2

Pinging @brauner for this one :slight_smile:

3

Oh, and there’s our first API extension for the ceph storage driver. Can someone make an issue for this one the GitHub repo? @stgraber should we consider this a bugfix and sru it or just release 2.17 sooner?

4

Hi,

Just created: https://github.com/lxc/lxd/issues/3597

Thanks again!

5

@yneveu I sent a branch https://github.com/lxc/lxd/pull/3598 that allows to specify a ceph.user.name property when creating osd storage pools.

6

Hi,

I quickly tested it and can’t work now.
You add a user to all exec commands in storage_ceph_utils.go but not in storage_ceph.go :slight_smile:
I’ll try to hardcode it in storage_ceph.go for my own tests now and if you don’t i’ll patch it as you did for storage_ceph_utils.go.

7

Ah, I forgot to add it to the calls in StoragePoolCreate(). Should be fixed now. Would you please test again?

8

Not only in StoragePoolCreate(). First error i have is on StorageCoreInit when it asks the Ceph version. Everywhere the ceph command is used, it should be with --id= as parameter.

9

Oh right, thanks! I’ll add it there right now. Anywhere else I missed?

10

I don’t know yet, i’ll check it soon!

1 Like
11

3 things:

I found a small typo, see https://github.com/brauner/lxd/pull/1

I had to change one call to rbd, changing argument “–image-features” to “–image-feature” and the layering parameter to integer 2. But it’s because we use a pretty old version of Ceph and options naming changed since (We use hammer version).

I successfully passed first step of pool creation.

Now i’m facing another problem. It seems the key s.pool.Config[“ceph.user.name”] is not defined so, i got a Go panic:

Jul 27 16:21:49 paprika lxd[2795]: 2017/07/27 16:21:49 http: panic serving @: runtime error: invalid memory address or nil pointer dereference
Jul 27 16:21:49 paprika lxd[2795]: goroutine 163 [running]:
Jul 27 16:21:49 paprika lxd[2795]: net/http.(*conn).serve.func1(0xc4203d1480)
Jul 27 16:21:49 paprika lxd[2795]: /usr/lib/go-1.7/src/net/http/server.go:1491 +0x12a
Jul 27 16:21:49 paprika lxd[2795]: panic(0xb32ca0, 0xc42000e070)
Jul 27 16:21:49 paprika lxd[2795]: /usr/lib/go-1.7/src/runtime/panic.go:458 +0x243
Jul 27 16:21:49 paprika lxd[2795]: main.(*storageCeph).StorageCoreInit(0xc42042c630, 0xc42042c630, 0x1)
Jul 27 16:21:49 paprika lxd[2795]: /opt/go/src/github.com/lxc/lxd/lxd/storage_ceph.go:31 +0x9a
Jul 27 16:21:49 paprika lxd[2795]: main.storageCoreInit(0xc420174998, 0x4, 0x1, 0xc420496050, 0x0, 0x0)

12

Sorry, stacktrace was not the full one:

Jul 27 16:58:24 paprika lxd[2795]: 2017/07/27 16:58:24 http: panic serving @: runtime error: invalid memory address or nil pointer dereference
Jul 27 16:58:24 paprika lxd[2795]: goroutine 164 [running]:
Jul 27 16:58:24 paprika lxd[2795]: net/http.(*conn).serve.func1(0xc4204a8100)
Jul 27 16:58:24 paprika lxd[2795]: /usr/lib/go-1.7/src/net/http/server.go:1491 +0x12a
Jul 27 16:58:24 paprika lxd[2795]: panic(0xb32ca0, 0xc42000e070)
Jul 27 16:58:24 paprika lxd[2795]: /usr/lib/go-1.7/src/runtime/panic.go:458 +0x243
Jul 27 16:58:24 paprika lxd[2795]: main.(*storageCeph).StorageCoreInit(0xc42042c750, 0xc42042c750, 0x1)
Jul 27 16:58:24 paprika lxd[2795]: /opt/go/src/github.com/lxc/lxd/lxd/storage_ceph.go:31 +0x9a
Jul 27 16:58:24 paprika lxd[2795]: main.storageCoreInit(0xc420174998, 0x4, 0x1, 0xc420496f00, 0x0, 0x0)
Jul 27 16:58:24 paprika lxd[2795]: /opt/go/src/github.com/lxc/lxd/lxd/storage.go:287 +0x1ee
Jul 27 16:58:24 paprika lxd[2795]: main.api10Get(0xc4201167e0, 0xc42041eb40, 0xbbf96d, 0x3)
Jul 27 16:58:24 paprika lxd[2795]: /opt/go/src/github.com/lxc/lxd/lxd/api_1.0.go:215 +0x9b8
Jul 27 16:58:24 paprika lxd[2795]: main.(*Daemon).createCmd.func1(0x10d10c0, 0xc42040e270, 0xc42041eb40)
Jul 27 16:58:24 paprika lxd[2795]: /opt/go/src/github.com/lxc/lxd/lxd/daemon.go:252 +0x5e4
Jul 27 16:58:24 paprika lxd[2795]: net/http.HandlerFunc.ServeHTTP(0xc4202a7e50, 0x10d10c0, 0xc42040e270, 0xc42041eb40)
Jul 27 16:58:24 paprika lxd[2795]: /usr/lib/go-1.7/src/net/http/server.go:1726 +0x44
Jul 27 16:58:24 paprika lxd[2795]: github.com/gorilla/mux.(*Router).ServeHTTP(0xc4202a6280, 0x10d10c0, 0xc42040e270, 0xc42041eb40)
Jul 27 16:58:24 paprika lxd[2795]: /opt/go/src/github.com/gorilla/mux/mux.go:114 +0x10d
Jul 27 16:58:24 paprika lxd[2795]: main.(*lxdHttpServer).ServeHTTP(0xc4203ce000, 0x10d10c0, 0xc42040e270, 0xc42041e960)
Jul 27 16:58:24 paprika lxd[2795]: /opt/go/src/github.com/lxc/lxd/lxd/daemon.go:1394 +0x237
Jul 27 16:58:24 paprika lxd[2795]: net/http.serverHandler.ServeHTTP(0xc4203d0000, 0x10d10c0, 0xc42040e270, 0xc42041e960)
Jul 27 16:58:24 paprika lxd[2795]: /usr/lib/go-1.7/src/net/http/server.go:2202 +0x7d
Jul 27 16:58:24 paprika lxd[2795]: net/http.(*conn).serve(0xc4204a8100, 0x10d1a00, 0xc4203d52c0)
Jul 27 16:58:24 paprika lxd[2795]: /usr/lib/go-1.7/src/net/http/server.go:1579 +0x4b7
Jul 27 16:58:24 paprika lxd[2795]: created by net/http.(*Server).Serve
Jul 27 16:58:24 paprika lxd[2795]: /usr/lib/go-1.7/src/net/http/server.go:2293 +0x44d
Jul 27 16:58:30 paprika systemd[1]: Stopping Container hypervisor based on LXC…
Jul 27 16:58:30 paprika systemd[1]: Stopped Container hypervisor based on LXC.

13

@yneveu, updated again. I’m using rbd --version now which shouldn’t require the user name be specified.