Mathew McBride's website

HOWTO: Read only rootfs, writes to USB on Linux

embeddedraspberrypilinuxTue 26 Mar 2013 09:49:48No comments

What you will need:

  • Kernel sources for the system you are booting on
  • AUFS patches for the kernel
  • A USB drive, preferrably one with reasonably fast write speeds

The goal of this excercise is to create a Linux system that has a read-only root filesystem, with all write activites performed on a USB drive. In this instance, we will boot a Raspberry Pi, with the SDcard used as the read only rootfs, and a USB drive used for all file writes (system logs etc.). This is done to improve reliability, as during our access system project, we found that the SDcard can be a bit fragile if the system is reset often.

To achieve this, we use a "union filesystem", which allows to file system mounts to be joined under a single virtual file system, along with the ability for one of underlying filesystems to be read-only. It also allows any changes made to the read-only filesystem to be overriden by the read-write filesystem.

Step 1: Apply the aufs patch to the kernel sources
Instructions to do this are included in the aufs readme (and on the homepage linked above).

Step 2: Build an initramfs for your kernel
This is the real meat in the solution. An initial system image (initrd) is embedded in the kernel, which the Linux kernel hands control to at the end of the boot process. Normal Linux systems use initram/initrd (a seperate file image) as well, often to preload any modules (filesystem, RAID controllers) etc before handing over to the 'real' rootfs.

On the Raspberry Pi, the emergency kernel image uses the same approach.

To start, we need to assemble a statically compiled BusyBox image. (It needs to be static so all dependent libraries are embedded inside). There are already prebuilt packages in some distribtions for this (or as an emerge option in source-based distributions such as Gentoo), or you can compile one yourself.

Once you have a BusyBox binary, create an empty directory for your initramfs, and place your busybox in bin/. Then, create the symlinks like so:

mkdir -p initramfs/bin
cd /bin/busybox .
for i in $(./busybox --list); do ln -s busybox $i; done

We also need to create a few other directories under the initramfs root so Linux will boot. /dev is a little tricky, we need a skeleton device set so the console is activated. This needs to be done as root:

# From /bin above
cd ..
mkdir dev
mknod console c 5 1
mknod tty1 c 4 1
mknod tty2 c 4 2
mknod tty3 c 4 3
mknod tty4 c 4 4

We don't actually need to care about all the other devices under /dev, as devtmpfs in the Linux kernel will take care of that for us (make sure you have CONFIG_DEVTMPFS enabled in your kernel!). But even if you have the automount option for devtmpfs set, the init process still needs the /dev/console file present before it starts.

We also need a skeleton /etc/fstab file:

proc		/proc	proc	defaults    0	0
sysfs		/sys	sysfs	defaults    0	0

And we also need a few other directories present for this to work:

#From initramfs root
mkdir proc
mkdir rw
mkdir rootfs
mkdir aufs
mkdir sbin
ln -s bin/busybox sbin/init

Now, this all goes together under a custom init script (/init). This is the bare minimum to get going, adjust accordingly for your target environment:

#!/bin/sh

# Wait 10 seconds for the USB controller to sort itself out
echo "Waiting"
sleep 10

echo "Mounting dev proc sys"

mount -t devtmpfs none /dev
mount -t proc proc /proc
mount -t sysfs sysfs /sys

echo "Mount rootfs on sdcard on /rootfs"
mount -o noatime,ro -t ext4 /dev/mmcblk0p2 /rootfs

echo "Mounting USB on /dev/sda"
mount -o noatime -t ext4 /dev/sda1 /rw

# Remove old stuff
rm -rf /rw/tmp
rm -rf /rw/dev
 
mount -t aufs -o noatime,br=/rw=rw:/rootfs=ro none /aufs

umount /sys
umount /proc
mount -t devtmpfs none /aufs/dev

# This switches to your root filesystem

exec switch_root /aufs /sbin/init

Make sure you set execute (chmod +x init) on the init script

Step 3: Build a kernel with the initramfs embedded
Rather handily, the kernel build process will handle embeddeding the initramfs into the kernel image for us. To do this, you need to set the location of your initramfs directory in the kernel source configuration in your kernel .config. This can be relative to the location of your kernel sources. Here is an example from my .config:

CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="../initramfs"
CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0
CONFIG_RD_GZIP=y
# CONFIG_RD_BZIP2 is not set
# CONFIG_RD_LZMA is not set
# CONFIG_RD_XZ is not set
# CONFIG_RD_LZO is not set
CONFIG_INITRAMFS_COMPRESSION_NONE=y
To make sure any initramfs changes are embedded into your kernel image, I suggest removing the initramfs image made by the kernel build process first:
rm usr/initramfs_data.cpio

You can then build a kernel as usual and install it onto your target

Prebuilt images for the Raspberry Pi:
This is based on the Raspberry Pi kernel development git repository from early March 2012.


Your email address will not be published

Please retry reCAPTCHA

Welcome to my site

Mathew McBride, telecoms hardware access engineer, programmer, gamer and all round nerd

Warning: contents of blog may not make any sense whatsoever.

ipv6 ready

You are accessing this page over IPv6!

(C) Mathew McBride, 2006-2017
Creative Commons License
Unless specified, the content on this website is licensed under a Creative Commons Attribution-ShareAlike 3.0 Australia License.