Colin.Guthr.ie Illegitimi non carborundum

NFS Root Media Centre v2.0

So I've been running my home media centre on Mageia for the last year or so and Mandriva before that. Rather than just a simple setup with a normal disk, I like to try and make things complicated for myself. I don't necessarily do this because it's the best and most efficient setup (it's very likely NOT!), but I do it for the sake of learning how things work and glue together, maintainability and, in theory at least, replication to more hardware as I expand my house and the media player "outlets".

So a little over a year ago I wrote about how to setup Mageia 1 with network booting and an NFS root, so with all the changes in Mageia 2's init system (switch to systemd and dracut), and because I wanted to switch the architecture to x86_64, I figured a re-image was called for and of course I tried to take appropriate notes so others can learn from this experience - perhaps putting it to more practical use than just a single home media centre!!

Server Side

So I've covered the server side before, and there isn't really too much to it. The basic setup is to ensure you have a dhcpd configuration that works, and (ideally) provides a static IP to your media centre. Combine this with a tftpd service to dish up the kernel and initrd and appropriate NFS exports and you're all good.

DHCPD

For me my config in /etc/dhcpd.conf is as follows:

group {
  filename "pxelinux.0";
  next-server 192.168.99.1;

  host mediacentre {
        hardware ethernet 01:23:45:67:89:ab;
        fixed-address 192.168.99.2;
  }
}

In the past, I also added a root-path extra variable to this config, it was needed when used with mkinitrd-net to know how to mount the rootfs, but now that I'm using dracut, it is no longer needed and I prefer to keep things simple.

TFTPD

The TFTPD setup is pretty simple. It's just a matter of pointing it at the folder you want to use for your your boot. I chose the folder /home/netboot/loader as my folder which will hold the PXE bootloader and the kernel+initrd for the actual system along with a few config files. So all that was needed here was to edit the file: /etc/xinetd.d/tftp to have the following line:

  server_args = -s /home/netboot/loader -r blksize

I then added the following files to that folder:

-rw-r--r--  1 colin colin 7057124 Sep  2 02:24 initrd
-rw-r--r--  1 colin colin 2725632 Sep  1 18:42 kernel
-rw-rw-r--  1 colin colin   13148 Sep  3  2005 pxelinux.0
drwxrwxr-x  2 colin colin    4096 Sep  2 01:04 pxelinux.cfg/
lrwxrwxrwx  1 colin colin      20 Dec  5  2009 pxelinux.cfg/01-01-23-45-67-89-ab -> mediacentre.pxe.conf
-rw-r--r--  1 colin colin     273 Sep  2 01:04 pxelinux.cfg/mediacentre.pxe.conf

The contents of the .conf file is as follows:

DEFAULT default
LABEL default
    KERNEL kernel
    APPEND ro rd.ip=dhcp ip=dhcp splash quiet initrd=initrd root=nfs:192.168.99.1:/home/netboot/fs:ro,rsize=32768,wsize=32768,exec,hard,proto=tcp

This is enough to make the system boot happily via PXE. The config file name needed bu pxelinux is just the MAC address with a 01- prefix. I used a symlink to make this a bit clearer.

The kernel and the initrd generation I'll come back to later.

NFS Exports

The NFS exports are fairly basic, but there is one big gotcha that took me a while to figure out.

My /etc/exports is as follows:

/home/netboot/fs  192.168.99.2(fsid=root,ro,no_root_squash,no_subtree_check,async,insecure)

The key bit here is fsid=root. Without this, the client side would use idmapd to map remote uids to local uids. Sadly with NFS Root, you need to do various setup things (which primarily involves mounting tmpfs mounts to make certain areas writable) using the "mount" binary. When idmapping is used, but rpc.idmapd is not running, then the mount binary will be owned by the uid -1, and as it is a setuid binary, refused to run. This presents a problem as the readonly root setup stuff needs to run before rpc.idmapd is run! The classic Chicken and Egg conundrum. Thankfully, using fsid=root in the export seems to fix this problem.

That's pretty much it from the server side, let's install the fs now!

Installing the File System

So as was hinted about above, but I chose the path /home/netboot/fs as the folder to contain my rootfs. Installing the rootfs is pretty simple. Assuming your host OS is running the same version you want to install, just run:

rpm --root /home/netboot/fs --initdb
urpmi --root /home/netboot/fs basesystem-minimal

Once that was complete, I installed some of the packages I knew I wanted:

urpmi --root /home/netboot/fs kernel-desktop-latest locales-en xbmc mythtv-frontend nfs-clients-utils bash-completion colorprompt openssh-clients task-x11

This was sufficient to get things booting, but other packages will ultimately need to be installed as an when you need them. If you are looking for a more general purpose system, you can of course use any of the various task-* packages to get a more complete install.

Configuring the Client Side

With the basics of the filesystem installed, there needs to be a few tweaks here and there to ensure things run correctly. These are soemtimes things that happen on first boot on a normal system but, because we'll be using a read-only root, have to be performed manually. None of these are really critical, but they will result in warnings and such like on boot and it's always nice to be clean and tidy.

Random Seed

touch /home/netboot/fs/var/lib/random-seed

Create /etc/hostname

echo "mediacentre" > /home/netboot/fs/etc/hostname

Create /etc/sysconfig/network

echo "NETWORKING=yes" > /home/netboot/fs/etc/sysconfig/network

Configure DNS Loopups

Edit /home/netboot/fs/etc/resolvconf/resolv.conf.d/base and set appropriate DNS settings. This can likely be made to work seamlessly but at the moment the network is brought up by the initrd and then doesn't change after transitioning to the real filesystem, therefore DNS settings are never written. For my setup, a static file is simple. I'm sure it wouldn't be too hard to hack things up to do this dynamically in a neat way.

Configure NFS

• Edit /home/netboot/fs/etc/sysconfig/nfs-common and set NEED_IDMAP=yes
• Check that an "rpc" user exists inside the filesystem. I've not investigated as to why, but it wasn't created on my install.

Configure Read-only Root

The next stage is actually quite simple as 99% of the setup is included out of the box.

• Edit /home/netboot/fs/etc/sysconfig/readonly-root and set READONLY=yes
• Edit /home/netboot/fs/etc/rwtab and remove the line that mentions /var/lib/dbus. This will remove the "machine-id" file on every boot giving your machine a new "identity" each time. Programs like PulseAudio use this information to save settings.
• Create the file /home/netboot/fs/etc/rwtab.d/custom with the following contents:

empty   /root
files   /var/run
files   /var/lock
files   /var/lib/dbus

That's mostly it for the client side setup, we're almost ready to boot!

Generating the Ramdisk

This is a two part process. We will first generate a generic initrd that should work on most hardware. This initrd will be quite large and thus slow to download over TFTP when booting, so the second stage will be to generate a new initrd ON the machine that is tailored to the specific hardware and is thus smaller and will make booting quicker.

Generate the Generic Ramdisk

In order to generate the generic ramdisk I added the following to to /home/netboot/fs/etc/dracut.conf.d/52-netboot.conf:

hostonly="no"
omit_dracutmodules=
omit_dracutmodules+=" mdadmin "
add_dracutmodules+=" network nfs "
add_drivers+=" af-packet "
omit_drivers+=" nouveau "
compress=xz
push fstab_lines "tmpfs /var/lib/stateless/writable tmpfs defaults"
push fstab_lines "tmpfs /var/lib/stateless/state tmpfs defaults"

This overrides some of the defaults and ensures the maxium compression format is used. As my media centre has an nVidia card and as I want to use the horrible proprietary driver (blearch!) I've excluding nouveau. The tmpfs mounts are an needed part of readonly root. As I'm not really bothered about long-term state, I'm just using tmpfs here, but another approach would be to use NFS here to stave state longer term. For some reason we also needed af_packet otherwise dhcpclient in the initrd doesn't work. I'm not sure if this is a candidate to include in the kernel statically (rather than as a module) but if not I'll send a patch upstream to dracut to ensure this is included by default in the future.

With this in place I could generate my initrd with the following command:

dracut -c /home/netboot/fs/etc/dracut.conf --confdir /home/netboot/fs/etc/dracut.conf.d -f /home/netboot/loader/initrd 3.3.6-desktop-2.mga2

This generated the initrd as needed and along with the rest of the PXE setup mentioned earlier was enough to get my system booting!

First Boot

Obviously when I did this myself, the boot failed miserable several times, but hopefully by following the above instructions you should be OK and get a smooth boot. If it doens't work out that great you should be able to use the environment in the initrd itself to debug a little. If you don't have the typical command line stuff you need, just add a install_items+= line in your dracut config to install all the binaries and such you need to use (bash, vi, ifconfig, route and ps are good starting points!)

Generating the Host-specific Ramdisk

After things are booted you'll want to regenerate your initrd so it becomes sleek and trimmed down! To do this, just run:

dracut --host-only --modules "nfs network plymouth kernel-modules base" net.img

This should give you a host-specific initrd with only the absolutely needed modules to boot your box. You can then scp or use NFS to copy the initrd out of your net-booted machine and back to the host. Use it to replace the /home/netboot/loader/initrd file (although keep the original as a backup of course).

 

Summary

That's it! You now have a network booting machine. You can rubber stamp this and get a whole cluster of machines really easily now! Of course you may want to configure other things, such as ensuring graphical target is started rather than the default multi-user target, configure autologin and maybe LDAP for network wide authentication other such things, but these are all left as an exercise for the reader!

For me, this install is a massive improvement over the previous one using sysvinit and mkinitrd-net. It's significantly faster to boot, a very large element of which I place on systemd - by avoiding so many shell scripts during startup, we avoid reading a whole bunch of stuff over NFS repeatedly and thus get really good startup times. I'm also happy to say it shuts down cleanly too, which my old system never did (although as it's stateless, I usually just kill the power in these situations anyway - no need to worry about corrupt filesystems and such like after all!

Anyway, happy hacking. Hope this helps a few people!