Diskless Debian Linux booting via dhcp/pxe/nfs/tftp/aufs

Want to boot a (possibly minimal) installation of Debian off the network using a read-only NFS share as the root filesystem, such that each netbooted machine has / mounted read-only over NFS and all writes are done to memory? Read on!

This assumes you are using a Linux computer as your router, which will be running Debian and hosting the local version of Debian we will be serving to clients which are PXE booting. This could be seen as a second part of my tutorial on making a Debian box a router , as it assumes your local network is still 10.0.0.0/24 and the dhcp/nfs/tftp server’s IP is 10.0.0.1

First off, we’ll need deboostrap, nfs, tftpd, and syslinux. Install them:

apt-get install tftp-hpa nfs-kernel-server debootstrap syslinux

We will store our initrd and boot loader under /srv/tftp and our NFS root filesystem under /srv/nfsroot

mkdir -p /srv/tftp /srv/nfsroot

Our nfsroot needs to be mountable via NFS. Export it read-only to our local network by putting the following in /etc/exports

/srv/nfsroot 10.0.0.0/24(ro,no_root_squash,no_subtree_check)

We will be booting to a custom Debian install. Install it in /srv/nfsroot using Debootstrap:

debootstrap stable /srv/nfsroot http://ftp.us.debian.org/debian

Now we need to install some packages in the NFS installation of Debian:

chroot /srv/nfsroot apt-get update
chroot /srv/nfsroot apt-get install initramfs-tools linux-image-2.6.32-5-amd64

Configure its initramfs to generate NFS-booting initrd’s

sed 's/BOOT=local/BOOT=nfs/' -i /srv/nfsroot/etc/initramfs-tools/initramfs.conf

We’ll need the aufs module

echo aufs >> /srv/nfsroot/etc/initramfs-tools/modules

Create the file /srv/nfsroot/etc/initramfs-tools/scripts/init-bottom/aufs give it executable permissions and fill it with the following

modprobe aufs
mkdir /ro /rw /aufs
mount -t tmpfs tmpfs /rw -o noatime,mode=0755
mount --move $rootmnt /ro
mount -t aufs aufs /aufs -o noatime,dirs=/rw:/ro=ro
mkdir -p /aufs/rw /aufs/ro
mount --move /ro /aufs/ro
mount --move /rw /aufs/rw
mount --move /aufs /root
exit 0

Generate initrd

update-initramfs -k

Copy generated initrd, kernel image, and pxe bootloader to tftp root and create folder for pxe config

cp /srv/nfsroot/boot/initrd.img-2.6.32-5-amd64 /srv/tftp/
cp /srv/nfsroot/boot/vmlinuz-2.6.32-5-amd64 /srv/tftp/
cp /usr/lib/syslinux/pxelinux.0 /srv/tftp
mkdir /srv/tftp/pxelinux.cfg

Configure boot loader. Put the following into /srv/tftp/pxelinux.cfg/default

default Debian
prompt 1
timeout 10
label Debian
kernel vmlinuz-2.6.32-5-amd64
append ro initrd=initrd.img-2.6.32-5-amd64 root=/dev/nfs ip=dhcp nfsroot=10.0.0.1:/srv/nfsroot

Configure tftp’s /etc/default/tftpd-hpa

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/srv/tftp"
TFTP_ADDRESS="0.0.0.0:69"
TFTP_OPTIONS="--secure"

Add these lines to your dhcp config file /etc/dhcp/dhcpd.conf

next-server 10.0.0.1;
allow bootp;
allow booting;

Restart some services:

/etc/init.d/isc-dhcp-server restart
/etc/init.d/tftpd-hpa restart
exportfs -ra

At this point, configuration is done and you should be good to go. You might want to reset the root password on the nfs debian install:

chroot /srv/nfsroot passwd root

 

Linux as a router with iptables, bind9, and dhcpd

There are some benefits to using a Linux box as a router. You get full access to the power of iptables, can host stuff directly on the box itself rather than having forwarding ports to other machines on your network, can torrent with way more peers as the box will support more connections than a usual home router, use the router itself as a fileserver/seedbox, etc.

The network setup this entails is as follows: [Modem] – [Linux box/router] – [switch] – [other machines on your network]

For the box itself you will need two network interfaces, one for your modem and one for your switch. Throughout this tutorial, we will be referring to the one connected to your modem as eth0 and the one connected to your switch as eth1.

Additionally, the network range I will be using for your local network will be 10.0.0.0/24

This tutorial is intended for Debian/Ubuntu but porting it to CentOS is trivial.

Step 0 – Configure network interfaces

Debian uses /etc/network/interfaces for assigning IP addresses and so on to its network interfaces. You can use the following and tweak it to your needs.

# Loopback interface. Omitting this will cause weird problems
auto lo
iface lo inet loopback

# The interface connected to the modem. This implies you do not
# have a static IP address from your ISP. If you do, you can
# use the same notation eth1 uses below, with the addition of a 
# gateway clause
auto eth0
iface eth0 inet dhcp

# Interface bound to local network. 
auto eth1
iface eth1 inet static
address 10.0.0.1
netmask 255.255.255.0

Step 1 – Install packages

We will need dhcpd to provide DHCP to our local network and bind9 to provide DNS lookups

apt-get install isc-dhcp-server bind9

Step 2 – Configure dhcpd

As mentioned earlier, we’ll be using 10.0.0.0/24 as our IP range. Additionally, we’ll use 10.0.0.1 for the IP of our router on the local network.

The configuration file for dhcpd is /etc/dhcp/dhcpd.conf. You can configure it as follows for our purposes:

default-lease-time 600;
max-lease-time 7200;

subnet 10.0.0.0 netmask 255.255.255.0 {
        range 10.0.0.100 10.0.0.200;
        option domain-name-servers 10.0.0.1;
        option routers 10.0.0.1;

}

This will hand out IP addresses between 10.0.0.100 and 10.0.0.200 for your local network. When/if they run out, old addresses will be reused.

Step 3 – Configure bind9 to provide DNS for your network

Debian uses /etc/bind for its bind9 named configuration files. The one we care about in this case is /etc/bind/named.conf.options

At some point the file will contain the directive allow-recursion, inside the options block. The act of allowing a DNS server to provide DNS for domains other than ones it hosts is referred to as recursion, as it is recursively contacting other DNS servers to carry out the client’s request. Allow recursion for your local network as follows:

allow-recursion { 10.0.0.0/24; };

Step 4 – Allow packet forwarding in the kernel

Make sure the following two lines are either present or not commented in /etc/sysctl.conf

net.ipv4.conf.all.forwarding=1
net.ipv4.conf.default.forwarding=1

Then reload sysctl:

# sysctl -p

Step 5 – iptables packet fowarding/masquerading

We need to have iptables route packets from eth0 to eth1. For this we will use an init script. Create this file: /etc/init.d/iptables

#!/bin/bash

### BEGIN INIT INFO
# Provides: iptablesrules
# Required-Start:
# Required-Stop:
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description:
# Description:
### END INIT INFO

iptables -F
iptables -t nat -F

iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT

iptables -A FORWARD -i eth1 -s 10.0.0.0/255.255.255.0 -j ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

The most important lines are the last two. The first is accepting all packets that forward from eth1 (the local network) and the second masquerades them out eth0 (the internet).

That big comment at the top is to avoid warnings from Debian’s new dependency boot system.

Now enable the script:

# update-rc.d iptables enable

Step 6 – Restart services (or reboot)

/etc/init.d/iptables
/etc/init.d/bind9 restart
/etc/init.d/isc-dhcp-server restart

Conclusion

At this point, you’re essentially done. Restart the services and your machines on your local network will start receiving IP addresses and be able to connect to the internet, faster than if you were using a normal consumer-grade router.

Read on if you’d like more functionality.

Appendix 0 – Port forwarding

As tempting as hosting all the services you want on your router may be, you will invariably want to forward ports to machines behind your router.  Simply add this line to the iptables init script we made:

iptables -t nat -I PREROUTING -p tcp --dport 2080 -i eth0 -j DNAT --to 10.0.0.169:80

This will forward all requests coming from eth0 (the internet) on port TCP 2080 to port 80 on machine 10.0.0.169. If you need to use UDP rather than TCP, replace tcp with udp in that command.

Appendix 1 – Static IP’s on the local network

Having all of the computers on your network get a random dhcp address can be inconvenient if you want to export NFS shares to a single machine, among other reasons. DHCP can assign IP addresses based on MAC addresses. You can add lines such as the following to the dhcpd.conf file we referred to earlier:

host adore {
 hardware ethernet f4:6d:04:44:11:fc;
 fixed-address 10.0.0.40;
}

What you provide for the hostname can be anything you feel like making up, really. Make sure that the IP you give it does not overlap the range you are having dhcpd provide.

Appendix 2 – Well, what if I want WiFi?

A nice use for your old Linksis wifi router would be to use it as a hotspot. Simply log in to its admin interface, disable its built-in DHCP server, configure its WiFi settings as you’d prefer, and plug one of its switch ports into your switch which is connected to your Linux router. Leave the Linksys’ WAN port unplugged.

At this point it will essentially serve as a wireless “switch” of sorts. So you’ll have all the benefits of using a computer running Linux as a router, and still have WiFi for your place using the old Linksys as a hotspot.

Another way of providing WiFi connectivity is adding a wireless card to your Linux router. Unfortunately, that isn’t something I’ve felt like dealing with yet, so I’m not going to write an article on it.

Installing php-gtk on Debian

So you want to install PHP’s gtk extension. Compared to GTK’s bindings for Perl and Python, PHP’s apparently is under-maintained and is a pain to install as the developers have not accommodated changes in libtool. We will need to install various development packages, temporarily tweak libtool, and then attempt compiling PHP-GTK and enabling it, provided that didn’t fail.

This tutorial was performed on vanilla 64-bit Debian Squeeze 6.0.2 successfully. Something similar will hopefully work for Ubuntu and other Debian derivatives.

First off, become root and install these packages. We’ll be snagging the latest version via subversion and compiling it. We need pear to install cairo, which apparently is a dependency of compiling php-gtk

Assuming you use sudo like myself, you’ll use sudo -i to drop yourself to a root shell. If you have the root account enabled, either log in as root directly or use su - as a normal user to become root.

apt-get install build-essential php5-cli php5-dev libgtk2.0-dev libglade2-dev subversion php-pear

Now get cairo via pecl

pecl install cairo-beta

Unfortunately we temporarily need to hack our libtool configuration. Don’t worry; we’ll restore the old version later. As we’re running as root, we do not need to tweak the file permissions at all here, as other tutorials seem to mention.

cd /usr/share/aclocal
cp libtool.m4 libtool.m4.bak
cat lt~obsolete.m4 ltoptions.m4 ltsugar.m4 ltversion.m4 >> libtool.m4
cd

Check it out, run buildconf, compile, etc. This is the most important part. If buildconf or configure do not finish successfully, undo the above change (move on to my next step) and reply demanding me to update this tutorial. Please give as much information regarding your distro’s setup and the steps you have taken as possible.

svn co http://svn.php.net/repository/gtk/php-gtk/trunk php-gtk
cd php-gtk
./buildconf
./configure
make
make install
cd

Now undo that ugly libtool hack we used:

mv /usr/share/aclocal/libtool.m4.bak /usr/share/aclocal/libtool.m4

Enable cairo and php-gtk extensions by adding the necessary lines to the php.ini for command-line. We are not creating .ini’s for each .so in the /etc/php5/conf.d/ folder (as php extensions normally do on Debian) because then php5-cgi will attempt loading them and web pages will generate error 500’s as they try to connect to X-Windows

echo 'extension=php_gtk2.so' >> /etc/php5/cli/php.ini
echo 'extension=cairo.so' >> /etc/php5/cli/php.ini

Ensure that gtk is properly installed. We shouldn’t need to worry about cairo being broken since pecl shouldn’t have failed.

php -m | grep gtk

This should give something such as the following if all goes well. If it does not list php-gtk, we have failed.

root@adore:~# php -m | grep gtk
php-gtk
root@adore:~#

To further verify everything is working correctly, you should probably test a sample GTK hello world as described here: http://gtk.php.net/manual/en/tutorials.helloworld.php or just run the php-gtk software you sought this tutorial for anyway. This is unnecessary if you are installing php-gtk just to get a tool such as the Phoronix Test Suite to work.

You should be done! Comments are much appreciated!

Uninstalling php-gtk

Uninstall by killing the shared library/header files and then removing the line from the cli php.ini. (Due to your shell’s noclobber possibly being enabled, we are first outputting a gtk-less php.ini to a separate file and then moving it on top of the one including gtk)

rm -rf /usr/lib/php5/20090626/php_gtk2.so /usr/include/php5/ext/php_gtk2/
grep -v gtk /etc/php5/cli/php.ini > orig_php.ini
mv orig_php.ini /etc/php5/cli/php.ini

Fixing a dpkg io error

I encountered a dpkg related error a little while ago while upgrading packages on my Ubuntu Lucid server. I couldn’t find a fix on the internet and spent a little while investigating the cause. You can see from the command output that dpkg failed to properly install the Linux kernel package:

root@aeroplane:~# apt-get dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
linux-image-2.6.32-33-generic
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
3 not fully installed or removed.
Need to get 0B/31.6MB of archives.
After this operation, 0B of additional disk space will be used.
Do you want to continue [Y/n]? y
(Reading database ... 178303 files and directories currently installed.)
Preparing to replace linux-image-2.6.32-33-generic 2.6.32-33.70 (using .../linux-image-2.6.32-33-generic_2.6.32-33.71_i386.deb) ...
Done.
Unpacking replacement linux-image-2.6.32-33-generic ...
dpkg-deb: subprocess paste killed by signal (Broken pipe)
dpkg: error processing /var/cache/apt/archives/linux-image-2.6.32-33-generic_2.6.32-33.71_i386.deb (--unpack):
short read in buffer_copy (backend dpkg-deb during `./lib/modules/2.6.32-33-generic/kernel/drivers/ata/sata_mv.ko')
No apport report written because the error message indicates a dpkg I/O error
Running postrm hook script /usr/sbin/update-grub.
Generating grub.cfg ...
Found initrd image: /boot/initrd.img-2.6.32-33-generic
Found linux image: /boot/vmlinuz-2.6.32-32-generic
Found initrd image: /boot/initrd.img-2.6.32-32-generic
Found linux image: /boot/vmlinuz-2.6.32-31-generic
Found initrd image: /boot/initrd.img-2.6.32-31-generic
Found linux image: /boot/vmlinuz-2.6.32-30-generic
Found initrd image: /boot/initrd.img-2.6.32-30-generic
Found linux image: /boot/vmlinuz-2.6.32-29-generic
Found initrd image: /boot/initrd.img-2.6.32-29-generic
Found linux image: /boot/vmlinuz-2.6.32-28-generic
Found initrd image: /boot/initrd.img-2.6.32-28-generic
Found linux image: /boot/vmlinuz-2.6.32-27-generic
Found initrd image: /boot/initrd.img-2.6.32-27-generic
Found linux image: /boot/vmlinuz-2.6.32-26-generic
Found initrd image: /boot/initrd.img-2.6.32-26-generic
Found linux image: /boot/vmlinuz-2.6.32-25-generic
Found initrd image: /boot/initrd.img-2.6.32-25-generic
Found linux image: /boot/vmlinuz-2.6.32-24-generic
Found initrd image: /boot/initrd.img-2.6.32-24-generic
Found linux image: /boot/vmlinuz-2.6.32-21-generic
Found initrd image: /boot/initrd.img-2.6.32-21-generic
Found memtest86+ image: /boot/memtest86+.bin
done
Errors were encountered while processing:
/var/cache/apt/archives/linux-image-2.6.32-33-generic_2.6.32-33.71_i386.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@aeroplane:~#

Somehow the package archive file /var/cache/apt/archives/linux-image-2.6.32-33-generic_2.6.32-33.71_i386.deb became corrupt. Cleaning the local cache of deb packages and then upgrading again fixed the issue:

root@aeroplane:~# apt-get clean
root@aeroplane:~# apt-get dist-upgrade