Linuxcontainers.org image server appears to be broken?

Also, where abouts are you located, are you nearer US or UK?

I’m also getting this same error. Same version. Multiple host machines. Started some time within the last 15 hours. Located in Australia.

Tested working OK on a fresh LXD 4.13 install here in the UK using:

lxc launch images:ubuntu/focal c1
lxc launch images:archlinux blah

Perhaps the US is mirror playing up.

One of my machines is now working. Came good about 08:38 UTC. The other one located in a different location (but still in Aus) still has the error. Perhaps it will come good soon also.

Both machines seem to use images.linuxcontainers.org.

Yeah, so images.linuxcontainers.org redirects clients to either us.images.linuxcontainers.org or uk.images.linuxcontainers.org, and us.images.linuxcontainers.org seems to be down ATM.

It connects, but just hangs:

telnet us.images.linuxcontainers.org 80
Trying 2001:67c:1562::41...
Connected to gible.canonical.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: us.images.linuxcontainers.org

...

But its ok over IPv4:

telnet 91.189.91.21 80
Trying 91.189.91.21...
Connected to 91.189.91.21.
Escape character is '^]'.
GET / HTTP/1.1
Host: us.images.linuxcontainers.org

HTTP/1.1 200 OK
Date: Wed, 14 Apr 2021 08:50:15 GMT
Server: Apache/2.4.7 (Ubuntu)
Last-Modified: Wed, 14 Apr 2021 07:27:09 GMT
ETag: "e36c-5bfe9acbb05ba"
Accept-Ranges: bytes
Content-Length: 58220
Vary: Accept-Encoding
Content-Type: text/html

<!DOCTYPE html>
<html lang="en">
...
1 Like

If you add an /etc/hosts entry for us.images.linuxcontainers.org pointing to 91.189.91.21 it should fix it temporarily.

Just checked the internal Canonical IS team and there are known IPv6 issues at the US data center currently.

I added the /etc/hosts entry and it has not resolved the issue.

Check that has taken effect using tcpdump, also can you replicate the telnet tests I did above and confirm the issue is only affecting IPv6. Thanks

sudo tcpdump -nn -i any host 91.189.91.21

It was using IPv4 confirmed using tcpdump. However on my second machine the issue has been resolved at about 9:09 UTC.

1 Like

No, I am not accessing over IPv6

I am in London, UK

Can you provide output of:

curl -v -I images.linuxcontainers.org
$ curl -v -I images.linuxcontainers.org
*   Trying 170.39.196.167:80...
* Connected to images.linuxcontainers.org (170.39.196.167) port 80 (#0)
> HEAD / HTTP/1.1
> Host: images.linuxcontainers.org
> User-Agent: curl/7.76.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Moved Temporarily
HTTP/1.1 302 Moved Temporarily
< server: nginx/1.18.0 (Ubuntu)
server: nginx/1.18.0 (Ubuntu)
< date: Wed, 14 Apr 2021 09:35:34 GMT
date: Wed, 14 Apr 2021 09:35:34 GMT
< content-type: text/html
content-type: text/html
< content-length: 154
content-length: 154
< location: http://uk.images.linuxcontainers.org/
location: http://uk.images.linuxcontainers.org/
< access-control-allow-origin: *
access-control-allow-origin: *

< 
* Connection #0 to host images.linuxcontainers.org left intact

It is now working for me. Location: London (UK), using IPv4:

I didn’t have to apply the /etc/hosts workaround.

Many thanks for solving this.

L.

@tomp in case it is still ongoing and it’s of any help, I am definitely not accessing it over IPv6 from one of the machines, which is in the USA.

I could have been accessing it over IPv6 from home (Australia), not sure on that.

Have you tried any diagnostics steps, such as the one I suggested here?

I have not as I’m afraid it’s now working for me. :frowning: But I just wanted to chime in that on the first machine I noticed it on, IPv6 is not available so I don’t think it was an IPv6 issue.

I’m not sure, there appears to be intermittent network issues. But without more solid network traces I don’t have any suggestions on how to proceed. If it happens again run that curl command so we can at least concretely see which IP your client is trying to connect to.

1 Like

On my connection in the UK I only saw IPv6 issues accessing the US mirror, everything else appeared OK, but apparently it differs based on where you’re accessing it from.