Colin.Guthr.ie Illegitimi non carborundum

26Jun/116

Network Boot Mageia: PXE + NFS Root goodness

So a while back I wrote about setting up Mandriva for network boot and I'm still running a similar configuration albeit upgraded every six months or so as new Mandriva releases come out.

Well as I'm now mostly running Mageia, I decided it was worth posting a follow up article. Read on if you're interested in the nitty gritty of how such a system works.

So most of the details in that previous article still stand true, although thankfully it's now much simpler and requires very little in the way of patching of system files etc.

The Mandriva and Mageia rc.sysinit files both support the "early local" script (like rc.local, but executed much earlier in the boot process) which allows us to inject the necessary tweaks.

What are those necessary tweaks? Well primarily it's for setting up some tmpfs files and some unionfs or aufs2 mounts. The standard system init does actually do quite a lot towards getting this working automatically, but I've never had much success with that and there are a still a few things missing (related to dbus) so I prefer to keep things simple!

When I first upgraded my chroot to Mageia, the first hurdle was that mkinitrd-net was not available for Mageia. Thankfully I already had this installed from Mandriva days and the package still works fine. I'll make sure we import this package into Mageia so this won't be a problem ongoing. The second problem was that when booting with the latest kernel, the kernel panicked quite quickly relating to unionfs. It seems unionfs+nfs don't really agree that much. So I reverted back to the old Mandriva kernel which worked fine. So with a little research, I found out that aufs2 is basically replacing unionfs these days. With a couple of checks I was able to add aufs2 support to my rc.early.local script easily enough. This allowed the boot to work without any issues.

In actual fact, even with unionfs and the old kernel, things didn't work perfectly and relied on various tmpfs mounts done by the standard rc.sysinit. Without these, I found various issues relating to kernel panics and general slow performance. With aufs2 these problems don't seem to exist anymore and I can bypass these mounts for a simpler mount setup which is all rather nice.

So once things booted the second problem turned out to be related to keyboard support... I can't quite remember what this was, but I suspect it just related to me not having evdev installed... my memory is sketchy tho'. It certainly wasn't a big problem. What was a big problem was that my user had no rights to various hardware devices. I quickly realised that this related to console kit not marking my login (which was via the autologin package) as "active". This took a little more sleuthing but I finally realised this was down to not bringing the autologin package up to date with the recent(ish) changes in console kit, namely the pam ck connector. With a little bit of patching of the /etc/pam.d/autologin file, I was up and running again! I've reported this to the Mageia devel list. Hopefully someone who knows this stuff better than me can comment and we can push an update soon now the updates and QA processes are in place.

So without further ado, here is the contents of my rc.early.local. The comments should mean the file kinda speaks for itself.

#!/bin/bash

# This file is sourced early on in rc.sysinit and this means
# we can do some very early setup for the read only / filesystem.

# Firstly, we need some writable directories.
# We do this using unionfs or aufs to combine the readonly versions with
# a tmpfs location to provide writable space.

USE_AUFS=$(uname -r | grep -v 2.6.33 | wc -l)
UFS=unionfs
if [ $USE_AUFS -gt 0 ]; then
UFS=aufs
fi

# Mount somewhere we can use in ram
TMPFS=/mnt/tmpfs
mount -t tmpfs none $TMPFS -o size=2g

for fs in etc var tmp root; do
echo "Mounting $UFS on /$fs to make it writable (any changes will not be saved)"
mkdir -p $TMPFS/$UFS/$fs
mount -t $UFS -o dirs=$TMPFS/$UFS/$fs=rw:/$fs=ro none /$fs
done

# Rather than have the stupid mounts done by rc.sysinit
# our aufs stuff above is much cleaner. But we need to
# tell rc.sysinit that we are READONLY all the same.
echo "READONLY=yes" >>/etc/sysconfig/readonly-root

if [ $USE_AUFS -gt 0 ]; then
# If we use aufs all is well and we don't need stateful dirs
# Or the complexity of copying over files from the underlying filesystem
# to our stateful directories. So we just nuke rwtab and rc.sysinit
# will do mostly nothing.
echo -n "">/etc/rwtab

# We also fudge the code in rc.sysinit to not mount it's own tmpfs
# space seeing as we already do that above. There is no combo of
# options to let this happen so we just create a fake mount point
. /etc/sysconfig/readonly-root
[ -z "$RW_MOUNT" ] && RW_MOUNT=/var/lib/stateless/writable

mkdir -p $TMPFS/ignore-me $RW_MOUNT
echo "$TMPFS/ignore-me $RW_MOUNT noauto bind 0 0" >>/etc/fstab
else
# Sadly if we are using unionfs, the module is simply not
# reliable enough to work properly, thus we must use the tmpfs
# approach built in to rc.sysinit.

# Sadly, the official version of this file forgets about keeping
# the dbus machine-id file consistent (which is used by PulseAudio
# to store setup info) so we have to manually add this here.
echo "files /var/lib/dbus" >>/etc/rwtab
fi

# We need dev to have ACL support for Console Kit, but
# unionfs does not support this, so we just mount a clean
# /dev. When aufs is used rather than unionfs, we can just
# include this in the above.
mkdir -p $TMPFS/dev
mount -o bind $TMPFS/dev /dev

# Mark /tmp as temp
chmod ugo+rwxt /tmp

# Mount the correct config directory for our hostname
HN=$(hostname -s)
if [ -d /var/configs/$HN ]; then
mount /var/configs/$HN /var/config -o bind
else
echo "Warning: Config directory /var/configs/$HN does not exist"
fi

# Fake an eth0 configuration
cat >/etc/sysconfig/network-scripts/ifcfg-eth0 < DEVICE=eth0
ONBOOT=yes
NOZEROCONF=yes
EOF

If anyone has any suggestions for improvements or wants to discuss any of the bits in detail, just leave a comment!

Share and Enjoy:
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks
  • Identi.ca
  • Slashdot
Comments (6) Trackbacks (1)
  1. Any specific reason to use NOZEROCONF, or it just came this way for legacy issues? Or it interferes in some manner with remote boot?

    • (sorry for the late reply – seems I missed a few comment notifications)
      To be honest that’s probably legacy… but as this is a network boot, you don’t really want to use any kind of link detection at all on the network port. I’ve not tested recently with out the flag tho’, so it could be it works fine.

      That’s kind of the trouble with a rolling config like this, some of the quirks you don’t really need any more will just live on and never get tested :D

      Roll on systemd, as I’ll have to change *everything* then :p

      • Or perhaps not, systemd simplifies most of those things for you with the services and their dependencies and such. But I haven’t seen any of such systemd-based environments deployed yet, it would be really interesting to get it working.

  2. It would be good if you could describe how to get around Ethernet firmware netboot annoyances. To my dismay I found out a few old my older machines have e100 and tg3, both of which require firmware.

    How (and from where) is the firmware for ethernet loaded if you netboot? I tried putting the firmware images in the initrd under /lib/firmware but the driver complained it couldn’t find the firmware files. Unfortunately this is not documented for either Mageia or Mandriva

    • To be perfectly honest, I have no idea! I’d have thought that putting them in the initrd would be the right thing to do, but I guess that process will need to be tweaked a bit. The initrc created by mkinitrd-net is a bit simplistic, so it’s entirely possible that it’ll need to have some commands injected into it when loading the actual driver. Quite what those commands are I’m not too sure :s

      • There are two ways I know of to overcome this issue:
        1. Use a PXE NIC that does not require (non-free) firmware to be loaded at boot-time (least painful)
        2. Recompile kernel to “include in-kernel firmware blobs in kernel binary (FIRMWARE_IN_KERNEL=Y)”; and the offending driver be built into the kernel (i.e., not as module). You’d also have to recompile any third party drivers (such as nvidia) if you happen to need that.

        I can’t be sure but I don’t think any amount of initrc fu is going to allow you to netboot if the module requires non-free firmware (chicken-egg problem). I would love however to be proven wrong.

        Incidentally I was mistaken earlier regarding tg3, the firmware is GPL with that driver so at least with that NIC there is no netboot annoyance.


Leave a comment