Table of Contents

Intro

(to the main content)

Working as a DevOps Engineer, I really appreciate virtual machines. Need to test another OS or distro? Run it in a VM! Need to mimic live infrastructure? VM and virtual networking! Need to run something sketchy? You already know the answer. And cut out network access. Just in case.

I required a disposable system for testing various scripts. Something lighter than VW, to be able to run like 3-7 of them at a same time.
At work, we use Docker extensively. At some point I thought "Why not?", and quickly put together a simple container with OpenSSH. As a container ran, it started sshd and sure enough, I was able to connect and even run Ansible scripts against it.
There was one problem, though: Docker is really geared towards the "one container - one process" paradigm. Containers itself lack a proper INIT system. Even if I stick something into the container, I couldn't test that something starting properly. It won't start. Not without kludges.

It's time to investigate what the heck LXC and LXD is, because, apparently, I need it.

What is LXC and LXD

That was wiki, not me. But still helpful. To some extent.

Preparations

There is no hard requirements. Some third party components will provide additional capabilities, but nothing LXC and LXD will not run without.
Let's begin.

Install and Initialize

LXC and LXD can be installed using package manager:

# install
sudo pacman -S lxd

# enable and start service
sudo systemctl enable --now lxd

# start initialisation process
sudo lxd init

During initialization LXD tried to bury me under a ton of questions. I made it through. Here they are, with answers, for convenience of those who reading this:

• Would you like to use LXD clustering? (yes/no)
  Usually no. However, it's possible to combine several hosts into a cluster to manage containers and even migrate them from one host to another. This is actually a deep rabbit hole. I'll dig it out someday.

• Do you want to configure a new storage pool? (yes/no)
  A storage pool is a place for LXD to keep images, snapshots, and other data related to containers. Initially, we don't have any, so yes.

• Name of the storage backend to use (btrfs, dir, lvm)
  This is another interesting choice. If you have a BTRFS volume set up, you can use its sub-volumes like storage pool. If ZFS support is present - it will be listed here too. Dir will create a plain directory. I'm trying to make a simple setup - answer will be dir. At least for this time.

• Would you like to connect to a MAAS server? (yes/no)
  Answer no.

• Would you like to create a new local network bridge? (yes/no)
  I don't have any, so Yes.

• What should the new bridge be called?
  Leave it default - lxdbr0

• What IPv4 address should be used?
  Auto

• What IPv6 address should be used?
  Auto

• Would you like the LXD server to be available over the network? (yes/no)
  For local interaction - No. You may want "yes" if you wish to control LXD from another host or use something like LXD Dashboard.

• Would you like stale cached images to be updated automatically? (yes/no)
  I settled on No. Images will still be updated eventually when I create a new container. If the answer is yes, LXD will check for updates from time to time and download updated versions as soon as they become available.

• Would you like a YAML "lxd init" preseed to be printed? (yes/no)
  No, but if answered Yes - LXD will save yml file with answers which may be reused next time.

More info can be found here

Access from an Unprivileged User

By default, only root can use lxc command and manipulate containers. Similar to Docker, to use lxc without sudo, let's add current user to lxd user group:

$ sudo usermod -a -G lxd $USER

Re-login or reboot to apply the changes.

Unprivileged Containers

Using unprivileged containers is recommended.

To begin with unprivileged containers, it's required to do one of two things:

• either use usermod to add subuid and subgid to root like this:

  $ sudo usermod -v 1000000-1000999999 -w 1000000-1000999999 root
  

• or manually edit /etc/subuid and /etc/subgid, adding this line:

  root:1000000:1000000000
  

Container images

Ok, preparations done, let's experiment.
I want an Alpine Linux container, so which tag to use?

$ lxc image list images:alpine

There are plenty of them! I need an Edge version of Alpine, so the filter will be:

$ lxc image list images:alpine/edge

That's better. Now the platform. My laptop has an AMD64 CPU, so:

$ lxc image list images:alpine/edge/amd64

This level of precision isn't necessary, though. It's ok to use images:alpine/edge and let LXD figure out rest.

First container

Now, when correct image was found, launching a new container is a matter of executing:

# run container
$ lxc launch images:alpine/edge test

# list running containers
$ lxc list --format compact

      NAME       STATE          IPV4          IPV6    TYPE     SNAPSHOTS           
  test          RUNNING  10.38.47.237 (eth0)        CONTAINER  0  

To manage containers use:

# start
$ lxc start test

# stop
$ lxc stop test

# drop into container's shell
$ lxc exec test ash

# run arbitrary command
$ lxc exec test -- apk update

# remove
$ lxc delete test

This commands list is far from complete. lxc --help is your guide.

Networking

No network access

Networking supposed to "just work" from the beginning. Not in my case. As usual. Let's see...

# create container
$ lxc launch images:alpine/edge test

# drop into it
$ lxc exec test ash

# ping Google DNS
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
# No answer was received

Long story short: I found this post on Linux containers forum. Proposed solution indeed worked, although I needed to rewrite some firewall rules.

First, the bridge (lxdbr0 in my case) need to be added to the trusted firewall zone:

# add lxdbr0 bridge to trusted zone
$ firewall-cmd --zone=trusted --add-interface=lxdbr0

# reload firewall
$ firewall-cmd --reload

# check if "interfaces" item has "lxdbr0" in it
$ firewall-cmd --zone=trusted --list-interfaces

Second, we need to forward traffic from and to the bridge:

# add rules
$ firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -i lxdbr0 -j ACCEPT
$ firewall-cmd --direct --add-rule ipv4 filter FORWARD 0 -o lxdbr0 -j ACCEPT

# reload firewall
$ firewall-cmd --reload

# display all active rules
$ firewall-cmd --direct --get-all-rules

Ok, now let's test it:

# drop into container (again)
$ lxc exec test ash

# ping Google DNS
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=117 time=29.5 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=117 time=29.6 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=117 time=30.8 ms

...And the Ping Goes On!
Now, when network's alive, and nothing broken let's apply all rules again but adding --permanent argument to make them survive system restart.

Wrapping Up

Content of this article should be enough to set up and run LXC next time. There are still a lot of things I need to figure out. Expect follow-ups.