Prometheus when using cluster

Hi
I have a container named prometheus and install it there.

On container are metrics.crt and metrics.key when I openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:secp384r1 -sha384 -keyout metrics.key -nodes -out metrics.crt -days 3650 -subj “/CN=metrics.local”

On the host server, when I lxc file pull prometheus/root/metrics.crt - | lxc config trust add --type metrics --name prometheus

I find this :

Error: Cannot use metrics type certificate when using a token

I followed the steps here LXD metrics with Prometheus and Grafana - YouTube and I just realized it doesn’t use clusters.

Thank You

That shouldn’t be the issue, I think it may simply be because you didn’t tell lxc config trust add to read from stdin.

See the exact same thing happening to me here:

Basically you want to pipe into lxc config trust add --type metrics --name prometheus -

Hi, stgraber thanks for respone

Sorry for being stupid and i didnt see that. But there’s still a problem when lxc file pull prometheus/root/metrics.crt - | lxc config trust add --type metrics --name prometheus -

Like this :

Error: x509: invalid ECDSA parameters

By the way, I’m using debian 11 to host server and centos 7 to prometheus container

@sdeziel any idea what that may be about?

Could you please share the metrics.crt file in question?

Here the metric.crt :

-----BEGIN CERTIFICATE-----
MIIDDDCCApKgAwIBAgIJAKT27xn2BxVHMAoGCCqGSM49BAMDMBgxFjAUBgNVBAMM
DW1ldHJpY3MubG9jYWwwHhcNMjIwNDIxMDYzMTI0WhcNMzIwNDE4MDYzMTI0WjAY
MRYwFAYDVQQDDA1tZXRyaWNzLmxvY2FsMIIBzDCCAWQGByqGSM49AgEwggFXAgEB
MDwGByqGSM49AQECMQD//////////////////////////////////////////v//
//8AAAAAAAAAAP////8wewQw////////////////////////////////////////
//7/////AAAAAAAAAAD////8BDCzMS+n4j7n5JiOBWvj+C0ZGB2cbv6BQRIDFAiP
UBOHWsZWOY2KLtGdKoXI7dPsKu8DFQCjNZJqoxmieh0AiWpnc6SCes2scwRhBKqH
yiK+iwU3jrHHHvMgrXRuHTtii6ebmFn3QeCCVCo4VQLyXb9VKWw6VF44cnYKtzYX
3kqWJixvXZ6Yv5KS3Cn49B29KJoUfOnaMRO18LjACmCxzh1+gZ16Qx18kOoOXwIx
AP///////////////////////////////8djTYH0Ny3fWBoNskiwp3rs7BlqzMUp
cwIBAQNiAAR9xjb+wDlQHnbieogstVQ5WzpvLBUoNcJrFoEbncWtUnn4QHwSR8DN
4GdRM1JBUzJn9qGMJg3cP1Q7KtkbeVLrKr+7bvj6Ev6xlIyzFPwku5Ve3nHlwD/n
rHR1jKsOUwqjUDBOMB0GA1UdDgQWBBTAEPOq0GPmVspYJlU4cGdKaroD4TAfBgNV
HSMEGDAWgBTAEPOq0GPmVspYJlU4cGdKaroD4TAMBgNVHRMEBTADAQH/MAoGCCqG
SM49BAMDA2gAMGUCMQCOdzq76t3Ni49fiA1/e7JsswfvxLvJC+BZeNJHPiegrguf
Q6xRcfKl114T8YcnMwwCMFXDrsjc2kvuTLY082G3fNEjA6NGiayOZ4aMLkLp9dGU
4h+XmzeGNfZm1WxnkXEccA==
-----END CERTIFICATE-----

Your key doesn’t use a standard curve it seems:

$ openssl x509 -noout -text -in /tmp/metrics.crt
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            a4:f6:ef:19:f6:07:15:47
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: CN = metrics.local
        Validity
            Not Before: Apr 21 06:31:24 2022 GMT
            Not After : Apr 18 06:31:24 2032 GMT
        Subject: CN = metrics.local
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (384 bit)
                pub:
                    04:7d:c6:36:fe:c0:39:50:1e:76:e2:7a:88:2c:b5:
                    54:39:5b:3a:6f:2c:15:28:35:c2:6b:16:81:1b:9d:
                    c5:ad:52:79:f8:40:7c:12:47:c0:cd:e0:67:51:33:
                    52:41:53:32:67:f6:a1:8c:26:0d:dc:3f:54:3b:2a:
                    d9:1b:79:52:eb:2a:bf:bb:6e:f8:fa:12:fe:b1:94:
                    8c:b3:14:fc:24:bb:95:5e:de:71:e5:c0:3f:e7:ac:
                    74:75:8c:ab:0e:53:0a
                Field Type: prime-field
                Prime:
                    00:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:
                    ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:
                    ff:ff:fe:ff:ff:ff:ff:00:00:00:00:00:00:00:00:
                    ff:ff:ff:ff
                A:   
                    00:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:
                    ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:
                    ff:ff:fe:ff:ff:ff:ff:00:00:00:00:00:00:00:00:
                    ff:ff:ff:fc
                B:   
                    00:b3:31:2f:a7:e2:3e:e7:e4:98:8e:05:6b:e3:f8:
                    2d:19:18:1d:9c:6e:fe:81:41:12:03:14:08:8f:50:
                    13:87:5a:c6:56:39:8d:8a:2e:d1:9d:2a:85:c8:ed:
                    d3:ec:2a:ef
                Generator (uncompressed):
                    04:aa:87:ca:22:be:8b:05:37:8e:b1:c7:1e:f3:20:
                    ad:74:6e:1d:3b:62:8b:a7:9b:98:59:f7:41:e0:82:
                    54:2a:38:55:02:f2:5d:bf:55:29:6c:3a:54:5e:38:
                    72:76:0a:b7:36:17:de:4a:96:26:2c:6f:5d:9e:98:
                    bf:92:92:dc:29:f8:f4:1d:bd:28:9a:14:7c:e9:da:
                    31:13:b5:f0:b8:c0:0a:60:b1:ce:1d:7e:81:9d:7a:
                    43:1d:7c:90:ea:0e:5f
                Order: 
                    00:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:
                    ff:ff:ff:ff:ff:ff:ff:ff:ff:ff:c7:63:4d:81:f4:
                    37:2d:df:58:1a:0d:b2:48:b0:a7:7a:ec:ec:19:6a:
                    cc:c5:29:73
                Cofactor:  1 (0x1)
                Seed:
                    a3:35:92:6a:a3:19:a2:7a:1d:00:89:6a:67:73:a4:
                    82:7a:cd:ac:73
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                C0:10:F3:AA:D0:63:E6:56:CA:58:26:55:38:70:67:4A:6A:BA:03:E1
            X509v3 Authority Key Identifier: 
                keyid:C0:10:F3:AA:D0:63:E6:56:CA:58:26:55:38:70:67:4A:6A:BA:03:E1

            X509v3 Basic Constraints: 
                CA:TRUE
    Signature Algorithm: ecdsa-with-SHA384
         30:65:02:31:00:8e:77:3a:bb:ea:dd:cd:8b:8f:5f:88:0d:7f:
         7b:b2:6c:b3:07:ef:c4:bb:c9:0b:e0:59:78:d2:47:3e:27:a0:
         ae:0b:9f:43:ac:51:71:f2:a5:d7:5e:13:f1:87:27:33:0c:02:
         30:55:c3:ae:c8:dc:da:4b:ee:4c:b6:34:f3:61:b7:7c:d1:23:
         03:a3:46:89:ac:8e:67:86:8c:2e:42:e9:f5:d1:94:e2:1f:97:
         9b:37:86:35:f6:66:d5:6c:67:91:71:1c:70

I would suggest you generate a fresh pair using:

openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:secp384r1 -sha384 -keyout metrics.key -nodes -out metrics.crt -days 3650 -subj "/CN=metrics.local"

This will give you something like:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            14:f1:8f:92:d2:35:53:b8:32:68:45:3d:4d:10:39:ce:fc:90:1d:a9
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: CN = metrics.local
        Validity
            Not Before: Apr 27 12:57:23 2022 GMT
            Not After : Apr 24 12:57:23 2032 GMT
        Subject: CN = metrics.local
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (384 bit)
                pub:
                    04:eb:2b:c0:d3:3a:ac:d8:88:fd:4e:42:c7:62:b7:
                    6b:fc:6e:54:d7:50:57:13:40:d4:7f:88:d0:10:16:
                    b9:78:0e:4c:64:6f:31:e8:f2:b5:48:cb:b9:d8:e3:
                    c5:39:0f:cc:33:a1:0d:b8:8b:db:8c:f6:cc:e5:0a:
                    e4:46:a2:43:58:b9:00:79:a7:06:52:aa:d4:89:70:
                    22:53:bd:d0:75:57:13:ba:7e:2a:aa:27:48:9d:24:
                    42:58:7d:de:22:07:b7
                ASN1 OID: secp384r1
                NIST CURVE: P-384
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                3E:12:CD:AD:B8:67:C9:36:B9:76:53:EA:95:F8:2D:F1:FE:A7:28:20
            X509v3 Authority Key Identifier: 
                keyid:3E:12:CD:AD:B8:67:C9:36:B9:76:53:EA:95:F8:2D:F1:FE:A7:28:20

            X509v3 Basic Constraints: critical
                CA:TRUE
    Signature Algorithm: ecdsa-with-SHA384
         30:65:02:31:00:80:0f:f7:5b:04:34:9a:63:44:d0:68:3f:11:
         2c:a7:7b:86:2c:29:54:a1:78:f7:d9:3a:b0:c8:2a:dc:6a:42:
         42:05:62:48:44:27:34:e9:88:31:1d:f3:f2:5e:68:3d:08:02:
         30:78:85:a0:13:eb:62:23:fc:37:b5:33:2f:ea:ae:40:0b:bd:
         3e:f2:4b:08:5b:d0:01:d6:a4:3e:47:f4:18:db:b4:c7:e5:9e:
         74:70:89:00:a7:a5:c4:b6:3a:a1:d1:bc:db

Which LXD can grok.

I would have expected the above command to generate the right kind of key as it does here but obviously not for your. Could you share your output of openssl version please?

I’ve tried to generate, but still got the same problem like this :

Error: x509: invalid ECDSA parameters

Here the output :

OpenSSL 1.0.2k-fips 26 Jan 2017

It seems that ECDSA certificate generation is broken in OpenSSL 1.0.2, at least I was able to get a similarly broken one using Ubuntu 16.04:

root@xenial-openssl:~# openssl version
OpenSSL 1.0.2g  1 Mar 2016
root@xenial-openssl:~# openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:secp384r1 -sha384 -keyout metrics.key -nodes -out metrics.crt -days 3650 -subj "/CN=metrics.local"
Generating a 2048 bit EC private key
writing new private key to 'metrics.key'
-----

Having OpenSSL report generating a 2048 bit EC is well… surprising :wink:

I was able to test with an old OpenSSL 1.1.0 version and it worked so we’ll update our doc to mention that OpenSSL 1.1.0+ is needed:

To workaround this problem, you could either generate the cert/key on a newer system or go with older RSA like this:

openssl req -x509 -newkey rsa:2048 -sha384 -keyout metrics.key -nodes -out metrics.crt -days 3650 -subj "/CN=metrics.local"
1 Like