LXC on OpenWRT

Intro

LXC stands for LinuX Container. Those containers are some kind of chroot images but on steroids. But you can find a more appropriate definition and info on the internet! 😀

The popular Docker, under the hood, makes use of this technology in order to work and that is what makes this technology so popular nowadays.

LXC is lightweight and it can run on openwrt without any problem. To get it working, btw, you have to enable it in the kernel. In this article i will show you how to build a openwrt image with LXC enabled.


 

Let’s getting started!

The documentation availabe at wiki.openwrt.org explains to you how to download the source code from git of the current openwrt release (“Caos Calmer” as for now) . Avoid to download the packages, so skip the step:

./scripts/feeds install -a       

That will make the compilation a lot faster. Since you are building the current release, you can always get the packages you need from the official repo.

Once cloned the git repository, you can proceed getting the diff config file from the official repository and add your changes to it.

The diff config file contains all the configuration settings needed to build openwrt for your target board (RPi in my case).

For a raspberry pi this step would look like this:

cp config.diff .config
cat LXC_PATCH >> .config
make defconfig
make menuconfig

Make menuconfig is optional and it can be used to validate the configuration.

LXC_PATCH files contains the following lines, which add lxc support to the openwrt kernel.

CONFIG_KERNEL_BLK_CGROUP=y
# CONFIG_KERNEL_CC_STACKPROTECTOR_NONE is not set
CONFIG_KERNEL_CC_STACKPROTECTOR_REGULAR=y
CONFIG_KERNEL_CFQ_GROUP_IOSCHED=y
CONFIG_KERNEL_CGROUPS=y
CONFIG_KERNEL_CGROUP_CPUACCT=y
CONFIG_KERNEL_CGROUP_DEVICE=y
CONFIG_KERNEL_CGROUP_FREEZER=y
CONFIG_KERNEL_CGROUP_SCHED=y
CONFIG_KERNEL_CPUSETS=y
CONFIG_KERNEL_DEVPTS_MULTIPLE_INSTANCES=y
CONFIG_KERNEL_FREEZER=y
CONFIG_KERNEL_IOSCHED_DEADLINE=m
CONFIG_KERNEL_IPC_NS=y
# CONFIG_KERNEL_KALLSYMS is not set
CONFIG_KERNEL_LXC_MISC=y
CONFIG_KERNEL_MEMCG=y
CONFIG_KERNEL_MEMCG_SWAP=y
CONFIG_KERNEL_MM_OWNER=y
CONFIG_KERNEL_NAMESPACES=y
CONFIG_KERNEL_NETPRIO_CGROUP=y
CONFIG_KERNEL_NET_CLS_CGROUP=y
CONFIG_KERNEL_NET_NS=y
CONFIG_KERNEL_PID_NS=y
CONFIG_KERNEL_POSIX_MQUEUE=y
CONFIG_KERNEL_RESOURCE_COUNTERS=y
CONFIG_KERNEL_USER_NS=y
CONFIG_KERNEL_UTS_NS=y

The configuation is now in good shape and you can run “make world” ti build a flash able image which can be found in the bin directory

In case you need to re-compile everything from scratch, please make use of make dirclean and make clean.

If you wanna purge the config too, then run make distclean.

The compilation will take a while. Once done you can flash it to your device or SD card in my case according to

the instructions related to your target board.

After flashing the image to the device, SSH onto the device and  install the following package:

sudo -s

opkg update
opkg install iptables-mod-extra kmod-ipt-extra lxc-start lxc-execute lxc-info getopt xz
lxc lxc-attach lxc-autostart lxc-cgroup lxc-checkconfig lxc-clone lxc-common lxc-config lxc-configs lxc-console lxc-create lxc-destroy lxc-execute lxc-freeze lxc-hookslxc-info lxc-init lxc-ls lxc-lua lxc-monitor lxc-monitord lxc-snapshot lxc-start lxc-stop lxc-templates lxc-unfreeze lxc-unsharelxc-user-nic lxc-usernsexec lxc-wait lxc-unshare bash debootstrap

Bash is needed by all lxc-templates. Debootstrap is needed by some Debian based distribution templates.

Once you have the lxc userspace tools installed, you should check that lxc support in the kernel is fine by running the following command:

root@opwenrt:/usr/lib/lua/luci# lxc-checkconfig
— Namespaces —
Namespaces: enabled
Utsname namespace: enabled
Ipc namespace: enabled
Pid namespace: enabled
User namespace: enabled
Network namespace: enabled
Multiple /dev/pts instances: enabled

— Control groups —
Cgroup: enabled
Cgroup clone_children flag: enabled
Cgroup device: enabled
Cgroup sched: enabled
Cgroup cpu account: enabled
Cgroup memory controller: enabled

— Misc —
Veth pair device: enabled
Macvlan: enabled
Vlan: enabled
File capabilities: enabled

Note : Before booting a new kernel, you can check its configuration
usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig

If you get all options enabled they you get make use of all functionalities that LXC has to offer and you can then proceed  creating your first container:

  lxc-create -t cirros –name mycontainer

Remember that the containers are saved in /var/cache/lxc/

Please note that the lxc community package currently comes with a wide set of almost broken templates

root@Honeypot:/usr/lib/lua/luci# opkg files lxc-templates
Package lxc-templates (1.1.1-1) is installed on root and has the following files:

/usr/share/lxc/templates/lxc-oracle <– BAD ARCHITECTURE. NO ARM??
/usr/share/lxc/templates/lxc-plamo <– depends on “flock”. It may work. I got “Failed to download”
/usr/share/lxc/templates/lxc-busybox <– creates the containers, but doesn’t start (busybox no statically linked)
/usr/share/lxc/templates/lxc-fedora <– fails to download /releases/20/Fedora/armhfp/os
/usr/share/lxc/templates/lxc-sshd <— requires ssh-keygen.
/usr/share/lxc/templates/lxc-ubuntu-cloud <– no idea what’s broken
/usr/share/lxc/templates/lxc-openmandriva
/usr/share/lxc/templates/lxc-gentoo <– requires tar. maybe works >220MB. Ran out of disk on /tmp
/usr/share/lxc/templates/lxc-download <– no idea what’s wrong. broken?
/usr/share/lxc/templates/lxc-archlinux <– fails require pacman. not available?
/usr/share/lxc/templates/lxc-cirros <– WORKS
/usr/share/lxc/templates/lxc-debian <– requires debootstrap. maybe broken?
/usr/share/lxc/templates/lxc-ubuntu <– requires debootstrap. maybe broken?
/usr/share/lxc/templates/lxc-centos <– requires yum. fails container creation
/usr/share/lxc/templates/lxc-altlinux <– requires apt-get. fails container creation
/usr/share/lxc/templates/lxc-alpine <– requires sha256sum.
/usr/share/lxc/templates/lxc-opensuse <– requires zipper. not available?

Feel free to check overtime if any of those templates have been fixed, but for the time being only the smallest distro work for me and maybe some of you would prefer to create containers elsewhere and get openwrt download them.

For sure it would be nice to have a openwrt specific containers hub.

Beside those CLI tools,  OpenWRT community repo offers a LuCI web interface package.

This interface offers two main options:

  1. Management of the current images available. (start, stop, info, config editor)
  2. Download new images from the internet if you have a repository from where you can download images.

For me 1 works as a charme, but I have not managed to get 2 working yet.

4 comments on “LXC on OpenWRT

  1. Hi,
    I’m not able to find the LXC_PATCH file anywhere after I cloned the git project. Where is it located?

  2. Copy the lines CONFIG_KERNEL_ written above in the post.. and here you go.. 😀 you have the LXC_PATCH file. Hopefully it’s still valid since this post is a little bit outdated.
    Cheers

  3. Thank you for your research 🙂 I am currently working in a raspi3 implementing openwrt with a 200G ssd attached with usb. Attempting to create a mobile AP that will provide services like team speak, web server, files, Wireless monitoring, usable mini virtual machines that are available to users via LXC. The idea is to create a mobile meeting space 🙂 Thank you for pointing me in the right direction 🙂

    I have built a remoteable ubuntu vm with 8 cores and 16 GB of ram for remote compile of openwrt. Ssh is used to remote to it for commands. Using screen, the make -j7 V=s command is used for pretty quick compiles . That way at any time and location i can screen -r to reconnect to the terminal session and check progress 🙂

  4. The kernel modules seem to be default like in your blog in at least since this commit efffba34097ff75d12365fb863621d893f974623.

Leave a Reply

Your email address will not be published. Required fields are marked *