Note: This HOWTO is out of date. For now it is here only for historical reasons.
Last updated 2009-12-27 07:40:43 CDT

HOW-TO for Linux containers, somewhat Arch Linux specific.

Linux Containers


Linux Containers (LXC) are an operating system-level virtualization method for running multiple isolated server installs (containers) on a single control host. LXC does not provide a virtual machine, but rather provides a virtual environment that has its own process and network space. It is similar to a chroot, but offers much more isolation.


This document is intended as an overview on setting up and deploying containers, and is not an in depth detailed instruction by instruction guide. A certain amount of prerequisite knowledge and skills are assumed (running commands as root, kernel configuration, mounting filesystems, shell scripting, chroot type environments, networking setup, etc).

Kernel Setup

Configuration Options

LXC has been in the mainstream kernel since 2.6.29 but there are several configuration options that need to be set so that it can be used.


Build and install your kernel as is needed. Check your /proc/config.gz and your /proc/filesysystems first as you might already have support in your 2.6.29+ kernel.

Setup of the Controlling Host

Control group filesystem

LXC depends on the control group filesystem being mounted.


mkdir -p /cgroup
mount none -t cgroup /cgroup

In /etc/fstab

none /cgroup cgroup defaults 0 0

Userspace tools

LXC depends on the LXC package for your distribution: Arch Linux Debian Ubuntu Fedora

aptitude install lxc # or whatever you need to do to install the package

Bridge device setup

Arch Linux specific (also see bridge section of for more information)

config_br0="brctl setfd br0 0"
MODULES=(... bridge ...) # I did not need to do this
eth0="eth0 up" # I had to do the, "eth0 up" was not sufficient.
br0="dhcp" # or however you set your address
INTERFACES=(eth0 br0)
Bridge forward delay

In order for br0 to dhcp quickly (and for container network devices to be available quickly) one must set the forward delay of the bridge device to zero.

brctl setfd br0 0
Patch for the default Arch linux /etc/rc.d/network script to do this automatically
--- network.0 2009-10-13 13:05:40.924603683 -0500
+++ network 2009-10-13 13:18:59.534523717 -0500
@@ -172,6 +172,15 @@
          /usr/sbin/brctl addif $br $brif || error=1
+     eval brconfig="\$config_${br}"
+     if [ -n "${brconfig}" ]; then
+       if ${brconfig}; then
+         true
+       else
+         echo config_${br}=\"${brconfig}\" \<-- invalid configuration statement
+         error=1
+       fi
+     fi

See also: FS#16625

Container install


Bootstrap an install ( mkarchroot Debootstrap Rinse ). You can also just copy/use an existing installation’s complete root filesystem.

Download existing

You can download a base install tar ball. OpenVZ templates work just fine.

Using the lxc tools

/usr/bin/lxc-debian {create|destroy|purge|help}
/usr/bin/lxc-fedora {create|destroy|purge|help}

Container definition setup

Configuration files

Main config file

lxc.utsname = {CONTAINER_NAME} = veth = up = br0 # Use same bridge device used in your controlling host setup =  {a1:b2:c3:d4:e5:f6} # As appropiate (line only needed if you wish to dhcp later) = {} # (Use if you wish to dhcp later) = eth0 # could likely be whatever you want
lxc.mount = {/path/to/fstab/for/CONTAINER_NAME}
lxc.rootfs = {/path/to/rootfs/of/CONTAINER_NAME}

Container fstab

none {/path/to/rootfs/of/CONTAINER_NAME}/dev/pts devpts defaults 0 0
none {/path/to/rootfs/of/CONTAINER_NAME}/proc    proc   defaults 0 0
none {/path/to/rootfs/of/CONTAINER_NAME}/sys     sysfs  defaults 0 0
none {/path/to/rootfs/of/CONTAINER_NAME}/dev/shm tmpfs  defaults 0 0
You can bind mount your controlling host’s /dev into the container, but that can cause problems on your controlling host and is not recommended unless you really know what you are doing.
Specifically, you do not want /dev/initctl from the host to be mapped into your container, unless you want your containers to be able to shut down or reboot your controlling host.

/etc/fstab of the controlling host

# I map in shared directories from my controlling host's /etc/fstab
/home/{USER}/workspace /path/to/rootfs/of/{CONTAINER_NAME}/home/{USER}/workspace bind 0 0
# of course that assumes that the controlling host's UIDs and GIDs
# are the same in the container for those shares.

Container /dev

This can be done afer the container is created, but should be before starting it for the first time. Script to generate static devices for the container

udevd will not work in the container.

cd into /path/to/rootfs/of/CONTAINER_NAME first or change script as needed.

mv ${DEV} ${DEV}.old
mkdir -p ${DEV}
mknod -m 666 ${DEV}/null c 1 3
mknod -m 666 ${DEV}/zero c 1 5
mknod -m 666 ${DEV}/random c 1 8
mknod -m 666 ${DEV}/urandom c 1 9
mkdir -m 755 ${DEV}/pts
mkdir -m 1777 ${DEV}/shm
mknod -m 666 ${DEV}/tty c 5 0
mknod -m 600 ${DEV}/console c 5 1
mknod -m 666 ${DEV}/tty0 c 4 0
mknod -m 666 ${DEV}/full c 1 7
mknod -m 600 ${DEV}/initctl p
mknod -m 666 ${DEV}/ptmx c 5 2

Container Creation and destruction


lxc-create -f /path/to/{CONTAINER_NAME}/main/config/file -n {CONTAINER_NAME}

Once created editing the main config file and fstab file won’t work. You could track down where the container is created and change those files. In my experience it is just easier to destroy the container and create it again. Container creation and destruction are both very fast.


lxc-destroy -n {CONTAINER_NAME}

Init setup


Important if you plan to run the container’s /sbin/init.

Find the initial startup script for your container install:

Arch Linux: rc::sysinit:/etc/rc.sysinit
Centos 5: si::sysinit:/etc/rc.d/rc.sysinit
Debian (Lenny): si::sysinit:/etc/init.d/rcS
OpenSuse (10.3): si::bootwait:/etc/init.d/boot

The entry will need to be changed to point to a custom container init script.

rc.sysinit replacement


# Whatever is needed to clearn out old daemon/service pids from your container
rm -f $(find /var/run -name '*pid')
rm -f /var/lock/subsys/*
# you could use a dhcp client here
# or you could get your system network scripts to work
# (Which I've ran into troubles with in containers)
# (Edit gateway address, domain, and nameserver as need be)

route add default gw
echo > /etc/resolv.conf search your-domain
echo >> /etc/resolv.conf nameserver

# Initally we don't have any container originated mounts
rm -f /etc/mtab
touch /etc/mtab

I recommend testing this script first in a container bash session before running it from init. See if package managment can see the internet and install sshd (ssh-server) and make sure you can start and stop it. You’ll need to add sshd (ssh, ssh-server, whatever) to the runlevels you plan to run inside the container.

/etc/inittab cleanup

Comment out getty (or equivalent) entries in /etc/inittab (they do work if you have the /dev/ttyX entries for them)

other rc.d/runlevel cleanup

  • Remove any clock setting/saving from your runlevels (they will likely fail anyways)

  • Remove any starting of udevd as it will not work properly inside a container.

  • Remove any thing from your runlevels that uses /proc/mounts to determine what needs to be unmounted.

  • Remove any thing that tries to remount / as read only (or add mount -o rw,remount / to the rc.sysinit replacement script)

  • For Arch Linux I needed to edit /etc/rc.shutdown and put an exit 0 before stat_busy “Saving System Clock” as the clock and remaining items are not needed.

  • Debian and CentOS just required some runlevel clearing out (halt removed/replaced in levels 0 and 6) and some /etc/inittab editing.

  • OpenSUSE I’ve not completely figured out everything needed to clean it up, short of editing several startup starts or rewriting them.

/etc/rc.d/runlevel cleanup can be skipped if you need a simple setup

You can just append something like: /etc/init.d/sshd start to the above rc.sysinit replacement.

rc.d script for starting/stopping containers

Adapt as needed for your setup, this one is quite simplistic and does not invoke shutdown/halt or init 0 in the containers. Also, it might hang on waiting for a container to start.


(Arch Linux specific)

. /etc/rc.conf
. /etc/rc.d/functions

[ -f /etc/conf.d/lxc.conf ] && source /etc/conf.d/lxc.conf

case "$1" in
    for container in "${CONTAINERS[@]}"; do
       if [ "${container}" = "${container#!}" ]; then
         stat_busy "Starting Linux container: ${container}"
         /usr/bin/screen -dmS init-${container} /usr/bin/lxc-start -n ${container}
         lxc-wait --name=${container} --state=RUNNING
         if [  $? -gt 0 ]; then
    add_daemon lxc

    for container in "${CONTAINERS[@]}"; do
      if [ "${container}" = "${container#!}" ]; then
       stat_busy "Stopping Linux container: ${container}"
       /usr/bin/lxc-stop -n ${container}
       if [  $? -gt 0 ]; then
    rm_daemon lxc

    $0 stop
    sleep 1
    $0 start

    echo "usage: $0 {start|stop|restart}"




# use the ! prefix to disable starting/stopping a container


You may need to set your timezone in the container:

rm -f /etc/localtime
cp /usr/share/zoneinfo/YOUR_TIME_ZONE /etc/localtime

If you cleaned up your inittab and your rc/runlevels enough the you should be able to start the container without any errors. You will also be able to halt or reboot in the container, and it will act accordingly without errors, but will not actually restart or halt. To shutdown the container for real /sbin/init would have to exit, or an lxc-stop -n CONTAINER_NAME would have to be invoked from the controlling host. The terminal you start a container in will often lose control-C (you can restore that) but you can start the container in a screen session.

/usr/bin/screen -dmS init-CONTAINER_NAME /usr/bin/lxc-start -n CONTAINER_NAME

With Debian and Arch Linux I’ve had not problem installing packages inside a container and having the proper architecture (x86_32 or x86_64) detected for packages. However, with CentOS and OpenSUSE this has not been the case, and 32 bit containers often wanted to install 64 bit packages. (I need to look into starting the container’s init with setarch to fix this)

I’ve not had much success with lxc-debian. Additionally, it installs more packages than I would like. (For instance, I don’t need apache in most of my debian containers). I set LC_ALL to POSIX in /etc/profile so that I can do debian package management.


keychain behavior when running in a private pid namespace

If you use lxc-execute to run a shell or X session and you use keychain, you’ll need to pass an environment variable, say CONTAINER_ID, through lxc-execute to use in your private PID namespace and setup your keychain invocation as follows:

[ -z "${HOSTNAME}" ] && HOSTNAME=$(uname -n) # same as before

keychain --host "${CONTAINER_ID}${HOSTNAME}" keys-to-load
     . ~/.keychain/${CONTAINER_ID}${HOSTNAME}-sh

Reference Documentation

Useful Resources

Note: This HOWTO is out of date. For now it is here only for historical reasons.
Last updated 2009-12-27 07:40:43 CDT