On a vanilla LXD set up where i have where i have host and guest both at ubuntu 16_04 LTS
I managed to get the public ip assigned to the container and was able to successfully ping the Container’s public IP from my local desktop over public internet. - GREAT!
BUT , there seems to be something missing… this is what I am facing right now
H1 can ping the C1’s internal ip
H1 can ping the C1’s public ip
BUT C1’s public ip is not reachable from local desktop
So couple of things that i observed to finally get the C1’s public IP accessible over internet from my desktop
- This command - lxc network set lxdbr0 ipv4.routes 149.56.148.6/32 which is supposed to route traffic to the bridge works fine from the host while referring to the containers public ip but does not result in a successful ping response from container if we ping the container’s ip from our desktop via public internet.
- Had to remove the route manually and add it again on the host after the container is started to get this work .
- order of when the step 1 performed and if the container’s public ip is set in the containers interfaces.d/c50xxxxx file made the difference
Here is some notes i took
List containers
root@ubuntu:~# lxc list
+-----------+---------+-------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------+---------+-------------------+------+------------+-----------+
| webserver | RUNNING | 10.0.8.100 (eth0) | | PERSISTENT | 0 |
+-----------+---------+-------------------+------+------------+-----------+
Display lxc visible network interfaces and what is used by lxc
root@ubuntu:~# lxc network list
+--------+----------+---------+-------------+---------+
| NAME | TYPE | MANAGED | DESCRIPTION | USED BY |
+--------+----------+---------+-------------+---------+
| eno1 | physical | NO | | 0 |
+--------+----------+---------+-------------+---------+
| eno2 | physical | NO | | 0 |
+--------+----------+---------+-------------+---------+
| lxdbr0 | bridge | YES | | 1 |
+--------+----------+---------+-------------+---------+
Display lxdbr01 as seen lxc sees
root@ubuntu:~# lxc network show lxdbr0
config:
ipv4.address: 10.0.8.1/24
ipv4.dhcp.ranges: 10.0.8.2-10.0.8.254
ipv4.nat: "true"
ipv6.address: fd42:614c:7ebe:916c::1/64
ipv6.dhcp.stateful: "true"
ipv6.nat: "true"
description: ""
name: lxdbr0
type: bridge
used_by:
- /1.0/containers/webserver
managed: true
status: Created
locations:
- none
Routes on Host
root@ubuntu:~# ip route show
default via 2xx.1xx.151.49 dev eno1 onlink
10.0.8.0/24 dev lxdbr0 proto kernel scope link src 10.0.8.1
2xx.1xx.151.48/28 dev eno1 proto kernel scope link src 2xx.1xx.151.50
Iptables rules on HOst
root@ubuntu:~# iptables --list
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:domain /* generated for LXD network lxdbr0 */
ACCEPT udp -- anywhere anywhere udp dpt:domain /* generated for LXD network lxdbr0 */
ACCEPT udp -- anywhere anywhere udp dpt:bootps /* generated for LXD network lxdbr0 */
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere /* generated for LXD network lxdbr0 */
ACCEPT all -- anywhere anywhere /* generated for LXD network lxdbr0 */
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp spt:domain /* generated for LXD network lxdbr0 */
ACCEPT udp -- anywhere anywhere udp spt:domain /* generated for LXD network lxdbr0 */
ACCEPT udp -- anywhere anywhere udp spt:bootps /* generated for LXD network lxdbr0 */
List containers.
root@ubuntu:~# lxc list
+-----------+---------+-------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------+---------+-------------------+------+------------+-----------+
| webserver | RUNNING | 10.0.8.100 (eth0) | | PERSISTENT | 0 |
+-----------+---------+-------------------+------+------------+-----------+
Pinging public ip to be assigned to container.
root@ubuntu:~# ping 2xx.1xx.151.51
PING 2xx.1xx.151.51 (2xx.1xx.151.51) 56(84) bytes of data.
^C
--- 2xx.1xx.151.51 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
No response for ping . -> We will use this Public ip to assign to this container.
**** this step should be performed after the container ip is assigned , container restarted. root@ubuntu:~# lxc network set lxdbr0 ipv4.routes 2xx.1xx.151.51/32 -------> This need to be set after the container ip is set in its config file.!!!!! Manually setting
Log in to the container
root@ubuntu:~# lxc exec webserver bash
Check the interfaces on container.
root@webserver:~# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3e:28:0f:17
inet addr:10.0.8.100 Bcast:10.0.8.255 Mask:255.255.255.0
inet6 addr: fe80::216:3eff:fe28:f17/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:138 errors:0 dropped:0 overruns:0 frame:0
TX packets:115 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:398164 (398.1 KB) TX bytes:9371 (9.3 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
First set the containers public ip to persist during reboot by placing the below line in "50-cloud-init.cfg" file at location "/etc/network/interfaces.d"
post-up ip -4 addr add dev eth0 2xx.1xx.151.51/32 preferred_lft 0
Pinging the local and public ip addresses from within container.
root@webserver:~# ping 10.0.8.100
PING 10.0.8.100 (10.0.8.100) 56(84) bytes of data.
64 bytes from 10.0.8.100: icmp_seq=1 ttl=64 time=0.055 ms
64 bytes from 10.0.8.100: icmp_seq=2 ttl=64 time=0.022 ms
^C
--- 10.0.8.100 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.022/0.038/0.055/0.017 ms
root@webserver:~# ping 2xx.1xx.151.51
PING 2xx.1xx.151.51 (2xx.1xx.151.51) 56(84) bytes of data.
64 bytes from 2xx.1xx.151.51: icmp_seq=1 ttl=64 time=0.048 ms
64 bytes from 2xx.1xx.151.51: icmp_seq=2 ttl=64 time=0.022 ms
^C
--- 2xx.1xx.151.51 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.022/0.035/0.048/0.013 ms
Confirming that the web pages are accessible via public and internal ip's from within Container.
root@webserver:~# cd
root@webserver:~# service apache2 restart
root@webserver:~# wget http://10.0.8.100
--2019-01-21 23:06:10-- http://10.0.8.100/
Connecting to 10.0.8.100:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11321 (11K) [text/html]
Saving to: ‘index.html.3’
index.html.3 100%[============================================================================>] 11.06K --.-KB/s in 0s
2019-01-21 23:06:10 (105 MB/s) - ‘index.html.3’ saved [11321/11321]
root@webserver:~# wget 2xx.1xx.151.51
--2019-01-21 23:06:30-- http://2xx.1xx.151.51/
Connecting to 2xx.1xx.151.51:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11321 (11K) [text/html]
Saving to: ‘index.html.4’
index.html.4 100%[============================================================================>] 11.06K --.-KB/s in 0s
2019-01-21 23:06:30 (118 MB/s) - ‘index.html.4’ saved [11321/11321]
Great
from the host Tried to ping the containers public and internal ip addresses
From Internal address successful
From Public address was not successful - Understood.
Executed the command
lxc network set lxdbr0 ipv4.routes 2xx.1xx.151.51/32
This resulted in successful response from the public ip 2xx.1xx.151.51 while run on the HOST.
WAIT - Container's Ip was not reachable from local desktop.
root@ubuntu:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 2xx.1xx.151.49 0.0.0.0 UG 0 0 0 eno1
10.0.8.0 0.0.0.0 255.255.255.0 U 0 0 0 lxdbr0
2xx.1xx.151.48 0.0.0.0 255.255.255.240 U 0 0 0 eno1
root@ubuntu:~# ip route show
default via 2xx.1xx.151.49 dev eno1 onlink
10.0.8.0/24 dev lxdbr0 proto kernel scope link src 10.0.8.1 linkdown
2xx.1xx.151.48/28 dev eno1 proto kernel scope link src 2xx.1xx.151.50
root@ubuntu:~# lxc start webserver
root@ubuntu:~# lxc list
+-----------+---------+-----------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------+---------+-----------------------+------+------------+-----------+
| webserver | RUNNING | 2xx.1xx.151.51 (eth0) | | PERSISTENT | 0 |
| | | 10.0.8.100 (eth0) | | | |
+-----------+---------+-----------------------+------+------------+-----------+
root@ubuntu:~# ping 2xx.1xx.151.51
PING 2xx.1xx.151.51 (2xx.1xx.151.51) 56(84) bytes of data.
^C
--- 2xx.1xx.151.51 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
root@ubuntu:~# ping 10.0.8.100
PING 10.0.8.100 (10.0.8.100) 56(84) bytes of data.
64 bytes from 10.0.8.100: icmp_seq=1 ttl=64 time=0.069 ms
64 bytes from 10.0.8.100: icmp_seq=2 ttl=64 time=0.029 ms
^C
--- 10.0.8.100 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.029/0.049/0.069/0.020 ms
root@ubuntu:~# lxc network set lxdbr0 ipv4.routes 2xx.1xx.151.51/32
root@ubuntu:~# ping 2xx.1xx.151.51
PING 2xx.1xx.151.51 (2xx.1xx.151.51) 56(84) bytes of data.
64 bytes from 2xx.1xx.151.51: icmp_seq=1 ttl=64 time=0.103 ms
64 bytes from 2xx.1xx.151.51: icmp_seq=2 ttl=64 time=0.030 ms
^C
--- 2xx.1xx.151.51 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.030/0.066/0.103/0.037 ms
root@ubuntu:~# wget http://10.0.8.100
--2019-01-21 19:02:06-- http://10.0.8.100/
Connecting to 10.0.8.100:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11321 (11K) [text/html]
Saving to: ‘index.html.13’
index.html.13 100%[============================================================================>] 11.06K --.-KB/s in 0.002s
2019-01-21 19:02:06 (6.49 MB/s) - ‘index.html.13’ saved [11321/11321]
root@ubuntu:~# wget http://2xx.1xx.151.51
--2019-01-21 19:02:18-- http://2xx.1xx.151.51/
Connecting to 2xx.1xx.151.51:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11321 (11K) [text/html]
Saving to: ‘index.html.14’
index.html.14 100%[============================================================================>] 11.06K --.-KB/s in 0s
2019-01-21 19:02:18 (195 MB/s) - ‘index.html.14’ saved [11321/11321]
At this point Cannot ping public ip from local container *******
So added this route on HOST - this resulted in the container being visible from Local desktop
root@ubuntu:~# ip address add 2xx.1xx.151.51/32 dev lxdbr0
root@ubuntu:~# lxc list
+-----------+---------+-----------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------+---------+-----------------------+------+------------+-----------+
| webserver | RUNNING | 2xx.1xx.151.51 (eth0) | | PERSISTENT | 0 |
| | | 10.0.8.100 (eth0) | | | |
+-----------+---------+-----------------------+------+------------+-----------+
root@ubuntu:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 2xx.1xx.151.49 0.0.0.0 UG 0 0 0 eno1
10.0.8.0 0.0.0.0 255.255.255.0 U 0 0 0 lxdbr0
2xx.1xx.151.48 0.0.0.0 255.255.255.240 U 0 0 0 eno1
2xx.1xx.151.51 0.0.0.0 255.255.255.255 UH 0 0 0 lxdbr0
root@ubuntu:~# ip route show
default via 2xx.1xx.151.49 dev eno1 onlink
10.0.8.0/24 dev lxdbr0 proto kernel scope link src 10.0.8.1
2xx.1xx.151.48/28 dev eno1 proto kernel scope link src 2xx.1xx.151.50
2xx.1xx.151.51 dev lxdbr0 proto static scope link
root@ubuntu:~# wget http://2xx.1xx.151.51
--2019-01-21 19:04:15-- http://2xx.1xx.151.51/
Connecting to 2xx.1xx.151.51:80... failed: Connection refused. -**------> ISSUE HERE!!!**
root@ubuntu:~# wget http://10.0.8.100
--2019-01-21 19:06:19-- http://10.0.8.100/
Connecting to 10.0.8.100:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11321 (11K) [text/html]
Saving to: ‘index.html.15’
index.html.15 100%[============================================================================>] 11.06K --.-KB/s in 0s
2019-01-21 19:06:19 (232 MB/s) - ‘index.html.15’ saved [11321/11321]
At this point, container is visible from local desktop , we cannot get the pages being served by the public ip of container from the HOST .
Back to deleting and adding the route .
root@ubuntu:~# lxc list
+-----------+---------+-----------------------+------+------------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-----------+---------+-----------------------+------+------------+-----------+
| webserver | RUNNING | 2xx.1xx.151.51 (eth0) | | PERSISTENT | 0 |
| | | 10.0.8.100 (eth0) | | | |
+-----------+---------+-----------------------+------+------------+-----------+
root@ubuntu:~# cd
root@ubuntu:~# wget http://2xx.1xx.151.51
--2019-01-21 19:09:23-- http://2xx.1xx.151.51/
Connecting to 2xx.1xx.151.51:80... failed: Connection refused.
root@ubuntu:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 2xx.1xx.151.49 0.0.0.0 UG 0 0 0 eno1
10.0.8.0 0.0.0.0 255.255.255.0 U 0 0 0 lxdbr0
2xx.1xx.151.48 0.0.0.0 255.255.255.240 U 0 0 0 eno1
2xx.1xx.151.51 0.0.0.0 255.255.255.255 UH 0 0 0 lxdbr0
root@ubuntu:~# ip route show
default via 2xx.1xx.151.49 dev eno1 onlink
10.0.8.0/24 dev lxdbr0 proto kernel scope link src 10.0.8.1
2xx.1xx.151.48/28 dev eno1 proto kernel scope link src 2xx.1xx.151.50
2xx.1xx.151.51 dev lxdbr0 proto static scope link
root@ubuntu:~# route del -net 2xx.1xx.151.51 gw 0.0.0.0 netmask 255.255.255.255 dev lxdbr0
root@ubuntu:~# ip address del 2xx.1xx.151.51/32 dev lxdbr0
root@ubuntu:~# ping 2xx.1xx.151.51
PING 2xx.1xx.151.51 (2xx.1xx.151.51) 56(84) bytes of data.
^C
--- 2xx.1xx.151.51 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
root@ubuntu:~# lxc network set lxdbr0 ipv4.routes 2xx.1xx.151.51/32
root@ubuntu:~# wget http://2xx.1xx.151.51
--2019-01-21 19:11:10-- http://2xx.1xx.151.51/
Connecting to 2xx.1xx.151.51:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 11321 (11K) [text/html]
Saving to: ‘index.html.16’
index.html.16 100%[============================================================================>] 11.06K --.-KB/s in 0s
2019-01-21 19:11:10 (237 MB/s) - ‘index.html.16’ saved [11321/11321]
root@ubuntu:~#
At his point ,it is all as expected. Containers public ip is visible from local desktop , from host, containers apache web pages are accessible from HOST , and from the local desktop.
********* IF the host is restarted , had to remove and re add the routes.