LXD cluster on Raspberry Pi 4

This suggests the profile doesn’t exist with this name?

I use other command and now I have IP address. Thanks a lot for hint.
Where I can find best documentation for lxc and lxd containers. I think this is future not Vmware. Is light and offer same functionality. It is strange that have never hear about it befor.

root@localhost:~# lxc config device add proxis eth0 nic network=lxdfan0 name=eth0
Device eth0 added to proxis
root@localhost:~# lxc list
±-------±--------±-----±-----±----------±----------±----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
±-------±--------±-----±-----±----------±----------±----------+
| proxis | STOPPED | | | CONTAINER | 0 | localhost |
±-------±--------±-----±-----±----------±----------±----------+
| web1 | RUNNING | | | CONTAINER | 0 | pi2 |
±-------±--------±-----±-----±----------±----------±----------+
| web2 | RUNNING | | | CONTAINER | 0 | localhost |
±-------±--------±-----±-----±----------±----------±----------+
root@localhost:~# lxc start proxis
root@localhost:~# lxc list
±-------±--------±------------------±-----±----------±----------±----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
±-------±--------±------------------±-----±----------±----------±----------+
| proxis | RUNNING | 240.5.0.82 (eth0) | | CONTAINER | 0 | localhost |
±-------±--------±------------------±-----±----------±----------±----------+
| web1 | RUNNING | | | CONTAINER | 0 | pi2 |
±-------±--------±------------------±-----±----------±----------±----------+
| web2 | RUNNING | | | CONTAINER | 0 | localhost |
±-------±--------±------------------±-----±----------±----------±----------+
root@localhost:~#

root@localhost:~# lxc exec proxis bash
root@proxis:~# apt update
Hit:1 http://ports.ubuntu.com/ubuntu-ports focal InRelease
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease [111 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease [107 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports focal-updates/main arm64 Packages [539 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports focal-updates/universe arm64 Packages [638 kB]
Fetched 1395 kB in 1s (947 kB/s)
Reading package lists… Done
Building dependency tree
Reading state information… Done
All packages are up to date.
root@proxis:~#

Thanks Stephane,

So, sounds like I need to test with a specific a application stack(s) to guestimate number of containers that can be reliably supported.

Do you have suggested methods/protocols for such testing?

I’m looking toward supporting multiple web applications. I’m assuming low traffic per container and storage per container less than 2GB with some outlyers.

Thanks again,

LRP

Hey would you by chance know how to setup kubernetes inside lxd/lxc environment across multiple machine nodes? Ive had success with one machine running 3 instances and kubernetes. Maybe networking issue? thanks in advance

Great article and shows off exactly how lightweight and easy to use LXD is when compared to other very heavyweight orchestration methods.

Thames for sharing

@stgraber - that was a remarkably interesting article. I am interested in this idea, of setting up a ‘lab’ based on Raspberry Pi primarily motivated from interest in learning rather than needing lots of compute power or with a specific technical use-case in mind.

I’ve set up a single machine Ubuntu appliance before (AdGuard), so I’m familiar with the basic idea. But I’ve never worked with multiple Raspberry Pi in cluster before. Can you elaborate a bit at all on the practicalities of doing this?

From reading your tutorial, it looks like :

  • each device would be flashed with its own image
  • each device would be independently connected to the LAN
  • the clustering process would then join together the 3 (or whatever devices), but they would still all need their own connections to LAN.

Just from a pure physical footprint perspective, not to mention the fact I do not have many LAN ports, I assume some kind of cluster rack mount and network switch would be needed? (or at least would be very helpful).

Any suggestions on where to start with that? Like recommended brands or mistakes to avoid.

Something that could ‘start small’ and grow sounds good. Like rather than start with x3 with 8GB as you suggest, would starting at x2 with 4GB also work, with the extra $ going toward a cluster rack set-up. Would that still give enough oomph to start with, and would it be easy to add more resources later? I like the idea of adding some faster storage, especially if it can just slot into a rack.

You’re correct about what’s needed for the setup.

My own lab is quite overkill for most people as I have far more than a few raspberry pi in there, so I’ve got a 24 ports managed gigabit switch, 24 ports IP KVM and 16 ports managed PDU so I can easily restart anything I want remotely, change the network config or even access the display and keyboard of anything in there from anywhere.

For a more minimal environment, you could go with the simplest 8-port gigabit switch which should be very cheap these days. If you want to play with more complex networking, VLANs and the like down the line, then getting an older managed switch off eBay shouldn’t be much more expensive. Just note that those tend to be far bulkier, more power hungry and a lot more noisy than a tiny consumer grade switch :wink:

1 Like

Thanks @stgraber !

That lab set-up sounds cool!

I did manage to find some other walk-throughs on clustering RPis which give some good hints on simpler switches and multi-port PSU which will solve some of my constraints (few wall outlets and even fewer LAN ports!).

When you say 16-port PDU – is that a Power Distribution unit ? I take it that is quite a specialised (== $$) piece of kit?

Thanks again, this is looking like a very interesting ‘second project’ on Ubuntu appliance.

Yeah, PDU is server rack speak for a big powerbar but usually one that you can remotely control on a per-outlet basis. They’re expensive new but you can get some very old ones that still support web/ssh/snmp for not all that much money on eBay or similar sites.

Looking for a specific such model on eBay “APC MasterSwitch” gets me some viable options (including the SNMP/WEB card) for less than $100.

Wow, that APC MasterSwitch looks neat. Definitely something to keep in mind for ‘Phase 2’. Thanks for all your help here, hopefully will make some progress and have some positive report back in due course (may not be quick!).

hello, I tried to install the LXD appliance and when I do snap refresh to update the snap and the kernel appears this error. Can you help me?
"snap refresh
error: cannot perform the following tasks:

  • Update assets from kernel “pi-kernel” (475) (volume pi is not compatible with disk /dev/mmcblk0: cannot find disk partition /dev/mmcblk0p2 (starting at 269484032) in gadget: start offsets do not match (disk: 269484032 (257 MiB) and gadget: 1048576 (1 MiB)))"

As mentioned by e-mail, this should be reported at https://forum.snapcraft.io as that looks like an Ubuntu Core bug.

thank you so much, greetings

Error: Failed to join cluster: Failed to update cluster trust: Existing server certificate with different name “localhost” already in trust store

Unable to join on my second Pi.
Alex Meseguer