Diskless Operation with PXE - Advanced Networking

http://www.freebsd.org/doc/handbook/

Chapter 30.8. Advanced Networking

30.8. Diskless Operation with PXE

Updated by Jean-François Dockès.

Reorganized and enhanced by Alex Dupre.

The Intel® Preboot eXecution Environment (PXE) allows an operating system to boot over the network. For example, a FreeBSD system can boot over the network and operate without a local disk, using file systems mounted from an NFS server. PXE support is usually available in the BIOS. To use PXE when the machine starts, select the Boot from networkoption in the BIOS setup or type a function key during system initialization.

In order to provide the files needed for an operating system to boot over the network, a PXE setup also requires properly configured DHCP, TFTP, and NFS servers, where:

• Initial parameters, such as an IP address, executable boot filename and location, server name, and root path are obtained from the DHCP server.

• The operating system loader file is booted using TFTP.

• The file systems are loaded using NFS.

When a computer PXE boots, it receives information over DHCP about where to obtain the initial boot loader file. After the host computer receives this information, it downloads the boot loader via TFTP and then executes the boot loader. In FreeBSD, the boot loader file is /boot/pxeboot. After /boot/pxeboot executes, the FreeBSD kernel is loaded and the rest of the FreeBSD bootup sequence proceeds, as described in Chapter 12, The FreeBSD Booting Process.

This section describes how to configure these services on a FreeBSD system so that other systems can PXE boot into FreeBSD. Refer to diskless(8) for more information.

Caution: 

As described, the system providing these services is insecure. It should live in a protected area of a network and be untrusted by other hosts.

30.8.1. Setting Up the PXE Environment

Written by Craig Rodrigues.

The steps shown in this section configure the built-in NFS and TFTP servers. The next section demonstrates how to install and configure the DHCP server. In this example, the directory which will contain the files used by PXE users is /b/tftpboot/FreeBSD/install. It is important that this directory exists and that the same directory name is set in both/etc/inetd.conf and /usr/local/etc/dhcpd.conf.

1. Create the root directory which will contain a FreeBSD installation to be NFS mounted:

   # export NFSROOTDIR=/b/tftpboot/FreeBSD/install
   # mkdir -p ${NFSROOTDIR}

2. Enable the NFS server by adding this line to /etc/rc.conf:

   nfs_server_enable="YES"

3. Export the diskless root directory via NFS by adding the following to /etc/exports:

   /b -ro -alldirs

4. Start the NFS server:

   # service nfsd start

5. Enable inetd(8) by adding the following line to /etc/rc.conf:

   inetd_enable="YES"

6. Uncomment the following line in /etc/inetd.conf by making sure it does not start with a # symbol:

   tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /b/tftpboot

   Note: 

   Some PXE versions require the TCP version of TFTP. In this case, uncomment the second tftp line which contains stream tcp.

7. Start inetd(8):

   # service inetd start

8. Rebuild the FreeBSD kernel and userland (refer to Section 23.6, “Rebuilding World” for more detailed instructions):

   # cd /usr/src
   # make buildworld
   # make buildkernel

9. Install FreeBSD into the directory mounted over NFS:

   # make installworld DESTDIR=${NFSROOTDIR}
   # make installkernel DESTDIR=${NFSROOTDIR}
   # make distribution DESTDIR=${NFSROOTDIR}

10. Test that the TFTP server works and can download the boot loader which will be obtained via PXE:

    # tftp localhost
    tftp> get FreeBSD/install/boot/pxeboot
    Received 264951 bytes in 0.1 seconds

11. Edit ${NFSROOTDIR}/etc/fstab and create an entry to mount the root file system over NFS:

    # Device                                         Mountpoint    FSType   Options  Dump Pass
    myhost.example.com:/b/tftpboot/FreeBSD/install       /         nfs      ro        0    0

    Replace myhost.example.com with the hostname or IP address of the NFS server. In this example, the root file system is mounted read-only in order to prevent NFSclients from potentially deleting the contents of the root file system.

12. Set the root password in the PXE environment for client machines which are PXE booting :

    # chroot ${NFSROOTDIR}
    # passwd

13. If needed, enable ssh(1) root logins for client machines which are PXE booting by editing ${NFSROOTDIR}/etc/ssh/sshd_config and enabling PermitRootLogin. This option is documented in sshd_config(5).

14. Perform any other needed customizations of the PXE environment in ${NFSROOTDIR}. These customizations could include things like installing packages or editing the password file with vipw(8).

When booting from an NFS root volume, /etc/rc detects the NFS boot and runs /etc/rc.initdiskless. In this case, /etc and /var need to be memory backed file systems so that these directories are writable but the NFS root directory is read-only:

# chroot ${NFSROOTDIR}
# mkdir -p conf/base
# tar -c -v -f conf/base/etc.cpio.gz --format cpio --gzip etc
# tar -c -v -f conf/base/var.cpio.gz --format cpio --gzip var

When the system boots, memory file systems for /etc and /var will be created and mounted and the contents of the cpio.gz files will be copied into them.

30.8.2. Configuring the DHCP Server

The DHCP server does not need to be the same machine as the TFTP and NFS server, but it needs to be accessible in the network.

DHCP is not part of the FreeBSD base system but can be installed using the net/isc-dhcp42-server port or package.

Once installed, edit the configuration file, /usr/local/etc/dhcpd.conf. Configure the next-server, filename, and root-path settings as seen in this example:

subnet 192.168.0.0 netmask 255.255.255.0 {
   range 192.168.0.2 192.168.0.3 ;
   option subnet-mask 255.255.255.0 ;
   option routers 192.168.0.1 ;
   option broadcast-address 192.168.0.255 ;
   option domain-name-servers 192.168.35.35, 192.168.35.36 ;
   option domain-name "example.com";

   # IP address of TFTP server
   next-server 192.168.0.1 ;

   # path of boot loader obtained via tftp
     filename "FreeBSD/install/boot/pxeboot" ;

   # pxeboot boot loader will try to NFS mount this directory for root FS
   option root-path "192.168.0.1:/b/tftpboot/FreeBSD/install/" ;

}

The next-server directive is used to specify the IP address of the TFTP server.

The filename directive defines the path to /boot/pxeboot. A relative filename is used, meaning that /b/tftpboot is not included in the path.

The root-path option defines the path to the NFS root file system.

Once the edits are saved, enable DHCP at boot time by adding the following line to /etc/rc.conf:

dhcpd_enable="YES"

Then start the DHCP service:

# service isc-dhcpd start

30.8.3. Debugging PXE Problems

Once all of the services are configured and started, PXE clients should be able to automatically load FreeBSD over the network. If a particular client is unable to connect, when that client machine boots up, enter the BIOS configuration menu and confirm that it is set to boot from the network.

This section describes some troubleshooting tips for isolating the source of the configuration problem should no clients be able to PXE boot.

1. Use the net/wireshark package or port to debug the network traffic involved during the PXE booting process, which is illustrated in the diagram below.

   Figure 30.1.  PXE Booting Process with  NFS Root Mount

   

   	Client broadcasts a DHCPDISCOVER message.
   	The DHCP server responds with the IP address, next-server, filename, and root-path values.
   	The client sends a TFTP request to next-server, asking to retrieve filename.
   	The TFTP server responds and sends filename to client.
   	The client executes filename, which is pxeboot(8), which then loads the kernel.  When the kernel executes, the root file system specified by root-path is mounted over NFS.

   
   

2. On the TFTP server, read /var/log/xferlog to ensure that pxeboot is being retrieved from the correct location. To test this example configuration:

   # tftp 192.168.0.1
   tftp> get FreeBSD/install/boot/pxeboot
   Received 264951 bytes in 0.1 seconds

   The BUGS sections in tftpd(8) and tftp(1) document some limitations with TFTP.

3. Make sure that the root file system can be mounted via NFS. To test this example configuration:

   # mount -t nfs 192.168.0.1:/b/tftpboot/FreeBSD/install /mnt