Install EL Linux via PXE and UEFI

Overview

This HowTo guide documents how to install EL/RHEL Linux using PXE on a client host booting by UEFI.

We will show how to support UEFI booting with PXE, downloading files from your TFTP server. See also our network PXE network booting page for Linux OS installation, and also these useful pages:

Setting up the DHCP, TFTP and PXE services

Enable UEFI support in the DHCP server

We use an ISC_DHCP Linux server on EL/RHEL Linux. The ISC_DHCP server has actually been superceded by the ISC_KEA server, but we do not consider it here. On EL Linux ISC_KEA can be installed (in EL8/EL9 from EPEL) with dnf install kea kea-hooks kea-doc kea-keama.

Install the ISC_DHCP packages:

dnf install dhcp-server dhcp-common grub2-efi-x64

To get started with configuration the packages contain an example file /usr/share/doc/dhcp-server/dhcpd.conf.example. It is also recommended to consult examples on the internet, or to read the DHCP_Handbook for complete coverage of the ISC_DHCP server.

Add the following to the configuration file /etc/dhcp/dhcpd.conf in the top (global) section:

# These settings are required for UEFI boot:
option arch code 93 = unsigned integer 16; # RFC4578

The Client System Architecture Type Option 93 (EFI x86-64) is defined in RFC4578.

Add these options only if you need to support MTFTP (Multicast TFTP) as recommended (but undocumented) in many places:

option space PXE;
option PXE.mtftp-ip    code 1 = ip-address;
option PXE.mtftp-cport code 2 = unsigned integer 16;
option PXE.mtftp-sport code 3 = unsigned integer 16;
option PXE.mtftp-tmout code 4 = unsigned integer 8;
option PXE.mtftp-delay code 5 = unsigned integer 8;

In the DHCP subnet section(s) define UEFI RFC4578 or PXE (legacy) boot image types in the /tftpboot/uefi/ subdirectory:

# UEFI x86-64 boot (RFC4578 architecture types 7, 8 and 9)
if option arch = 00:07 {
      filename "uefi/BOOTX64.EFI";
} else if option arch = 00:08 {
      filename "uefi/BOOTX64.EFI";
} else if option arch = 00:09 {
      filename "uefi/BOOTX64.EFI";
} else {
      # PXE boot
      filename "pxelinux.0";
}

NOTES:

  • The BOOTX64.EFI file name is upper case in the EL8 installation images.

  • Having the boot file in a subdirectory such as uefi/BOOTX64.EFI will cause the client host PXE to download all further files also from that same uefi/ subdirectory, so you need to place other files there.

  • Probably obsolete: The shimx64.efi bootloader may be required in stead of BOOTX64.EFI in the above /etc/dhcp/dhcpd.conf.

When you have completed configuring the dhcpd.conf file, open the firewall for DHCP (port 67):

firewall-cmd --add-service=dhcp --permanent
firewall-cmd --reload

and start the DHCP service:

systemctl enable dhcpd
systemctl restart dhcpd

Configure the TFTP service

Your DHCP server should also run a TFTP service for file downloads. Install these packages:

dnf install tftp-server tftp shim-x64

Copy the service file to make local customizations:

cp /usr/lib/systemd/system/tftp.service /etc/systemd/system/tftp.service

Edit the file /etc/systemd/system/tftp.service to add the in.tftpd options --secure --ipv4:

ExecStart=/usr/sbin/in.tftpd -v --secure --ipv4 /var/lib/tftpboot

Open the firewall for TFTP (port 69):

firewall-cmd --add-service=tftp --permanent
firewall-cmd --reload

and start the service:

systemctl enable tftp
systemctl restart tftp

Download UEFI boot files

Create a special directory for UEFI boot files on the TFTP server:

mkdir /var/lib/tftpboot/uefi
ln -s /var/lib/tftpboot/uefi /tftpboot/uefi

NOTE: The OS installation *.efi files must be copied from the OS installation image, since the versions contained in EL8 shim-x64 RPM package seem to be buggy, see for example https://forums.rockylinux.org/t/pxe-boot-uefi-mode/4852. Symptoms may be that TFTP download of large vmlinuz or initrd.img files during Kickstart fail with a message error: timeout reading ….

Download all .efi files from a mirror site, for example the AlmaLinux mirror at https://mirror.fysik.dtu.dk/linux/almalinux/8/BaseOS/x86_64/kickstart/EFI/BOOT/ to the TFTP server’s folder /tftpboot/uefi/.

Download Linux boot images

For each EL/RHEL Linux (and other OS) version you should copy Linux boot images to a separate directory on the TFTP server, for example, for AlmaLinux 8.10:

mkdir /var/lib/tftpboot/AlmaLinux-8.10-x86_64/

In this directory create the following Makefile:

OS=almalinux
VERSION=8.10
MIRROR=<your-favorite-mirror>
default:
      @echo "NOTE: Boot images are from ${OS} version ${VERSION}"
      @wget --timestamping ${MIRROR}/${OS}/${VERSION}/BaseOS/x86_64/os/images/pxeboot/initrd.img
      @wget --timestamping ${MIRROR}/${OS}/${VERSION}/BaseOS/x86_64/os/images/pxeboot/vmlinuz

and run a make command to download the boot image files.

Create a grub.cfg file in /tftpboot/uefi/

The uefi/BOOTX64.EFI boot file will be looking for a GRUB2/Grub configuration file uefi/grub.cfg in the same subdirectory. Create the file /var/lib/tftpboot/uefi/grub.cfg with the contents:

set default="0"
function load_video {
  insmod efi_gop
  insmod efi_uga
  insmod video_bochs
  insmod video_cirrus
  insmod all_video
}
load_video
set gfxpayload=keep
insmod net
insmod efinet
insmod tftp
insmod gzio
insmod part_gpt
insmod ext2
set timeout=60
menuentry 'AlmaLinux 8.10 minimal Kickstart' --class centos --class gnu-linux --class gnu --class os --unrestricted {
  # Note: IPv6 disable during initial boot:
  linuxefi (tftp)/AlmaLinux-8.10-x86_64/vmlinuz ip=dhcp inst.ks=nfs:nfsvers=3:130.225.86.3:/u/kickstart/ks-rockylinux-8-minimal-x86_64.cfg ipv6.disable=1
  initrdefi (tftp)/AlmaLinux-8.10-x86_64/initrd.img
}

Additional menu entries may be appended to the above, for example:

menuentry 'AlmaLinux 9.6 minimal Kickstart' --class centos --class gnu-linux --class gnu --class os --unrestricted {
  linuxefi (tftp)/AlmaLinux-9.6-x86_64/vmlinuz ip=dhcp inst.ks=nfs:nfsvers=3:130.225.86.3:/u/kickstart/ks-rockylinux-9-minimal-x86_64.cfg ipv6.disable=1
  initrdefi (tftp)/AlmaLinux-9.6-x86_64/initrd.img
}

It is useful to have a grub.cfg menu item from the TFTP server which allows to boot the system from an existing OS installation on disk. This should be the default menu item. To boot a system with grubx64.efi (provided by the grub2-efi-x64 package) in the 1st partition of the first disk hd0:

menuentry 'Useless: Boot from local disk' {
  # Undocumented "exit" command.  Returns to BIOS boot menu on Dell 9020
  exit
}

In other Linux distributions such as Fedora the .efi files will be in different subdirectories of /boot/efi/EFI.

If there are multiple disks in the server, Grub will label them as hd0, hd1, hd2, etc. It seems that the numbering of such disks may vary, and if the OS installation is suddenly in disk hd1 in stead of hd0, it is useful to define a fallback boot menu item as in this example:

set default=0
set fallback=1
menuentry 'Boot from local disk hd0' {
 set root=(hd0,1)
 chainloader /efi/centos/grubx64.efi
}
menuentry 'Boot from local disk hd1' {
 set root=(hd1,1)
 chainloader /efi/centos/grubx64.efi
}

Automated network installation with pxeconfig

You can automate the PXE network booting process completely using the pxeconfig_toolkit written by Bas van der Vlies. Download the pxeconfig_toolkit and read the pxeconfig_installation page.

NOTE: We assume throughout the use of client UEFI booting, since the old BIOS booting is more or less deprecated.

Installation on EL Linux

Briefly, the installation steps for EL Linux are:

dnf install autoconf make gcc telnet
tar xf pxeconfig-5.1.3.tar.gz
cd pxeconfig-5.1.3/
/usr/bin/autoconf
./configure --prefix=/usr/local
make install DESTDIR=/

Configure the default boot method as UEFI in /usr/local/etc/pxeconfig.conf:

[DEFAULT]
boot_method=uefi

This configures the pxeconfig command to create grub.cfg files in the /tftpboot/uefi/ directory which was created in the Create a grub.cfg file in /tftpboot/uefi/ section.

Add the pxeconfigd service to the services file /etc/services:

pxeconfigd 6611/tcp   # pxeconfig daemon

Open port 6611 in the firewall:

firewall-cmd --permanent --zone=public --add-port=6611/tcp --reload

After installation setup the pxeconfigd service with Systemd:

restorecon -v /usr/local/sbin/pxeconfigd
cp /usr/local/share/doc/pxeconfig/examples/pxeconfigd@.socket /etc/systemd/system/
cp /usr/local/share/doc/pxeconfig/examples/pxeconfigd@.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable pxeconfigd.socket
systemctl start pxeconfigd.socket

Note that it is pxeconfigd.socket which handles the pxeconfigd service, similar to the normal telnet service, and not the .service file.

Hexadecimally encoded IP-addresses

To understand the client’s hexadecimally encoded IP-address, which the pxeconfig_toolkit manipulates in the server’s /tftpboot/uefi/ directory, we show some examples:

0A018219 decodes as 10.1.130.25

You can use the gethostip command from the syslinux package to convert hostnames and IP-addresses to hexadecimal, for example:

$ gethostip -f s001
s001.(domainname) 10.2.130.21 0A028215
$ gethostip -x s001
0A028215

The pxeconfig command

To use pxeconfig you should create any number of configuration files named default.<something> which contain different PXELINUX commands that perform the desired actions, for example, BIOS updates, firmware updates, hardware diagnostics, or network installation. See the above Create a grub.cfg file in /tftpboot/uefi/ section.

Use the pxeconfig command to configure those client nodes that you wish to install (the remaining nodes will simply boot from their hard disk). An example is:

$ pxeconfig c150
Which pxe config file must we use: ?
1 : default.rockylinux-8-sr850v3-x86_64
2 : default.rockylinux-8-x86_64

The pxeconfig command creates soft-links in the /tftpboot/uefi/ directory named as the hexadecimally encoded IP-address of the clients, pointing to one of the files default.*. As designed, the PXE network booting process will download the file given by the hexadecimal IP-address, and hence network installation of the node will take place.

If desired you can remove the soft-link:

$ pxeconfig -r c150

The hexls command

To list the soft links created by pxeconfig use the tool hexls and look for the IP-addresses and/or hostnames. An example output is:

$ hexls /tftpboot/uefi/
default.rockylinux-8-x86_64
grub.cfg
grub.cfg-0A028396 => 10.2.131.150 => c150.nifl.fysik.dtu.dk -> default.rockylinux-8-x86_64

The pxeconfigd service

The pxeconfigd service will remove the hexadecimally encoded IP-address soft-link on the server when contacted on port 6611 by the client node. In order for this to happen, you must create the client’s post-install script to make an action such as this example:

#!/bin/sh
# To be used with the pxeconfigd service:
# Remove the <hex_ipaddr> file from the pxelinux.cfg directory so the client will boot from disk.
telnet <IMAGESERVER> 6611
sleep 1
exit 0

When this script is executed on the node in the post-install phase, the telnet command connects to the pxeconfigd service on the image server, and this daemon will remove the hexadecimally encoded IP-address soft-link in /tftpboot/uefi/ corresponding to the client IP-address which did the telnet connection.

Configuring Kickstart automated install

EL Linux installation with Kickstart

RHEL Linux and EL clones such as AlmaLinux or RockyLinux, as well as Fedora, can be installed using Kickstart. See a general description from the Fedora page:

  • Many system administrators would prefer to use an automated installation method to install Fedora or RHEL on their machines. To answer this need, Red Hat created the Kickstart installation method. Using Kickstart, a system administrator can create a single file containing the answers to all the questions that would normally be asked during a typical installation.

  • A Kickstart_file can be kept on a server system and read by individual computers during the installation. This installation method can support the use of a single Kickstart_file to install Fedora or RHEL on multiple machines, making it ideal for network and system administrators.

Please read the documentation of the Kickstart_file syntax.

A Kickstart installation can be made using PXE network booting or PXE_and_UEFI network booting.

Automated installation using Anaconda is possible with UEFI as well as PXE legacy booting. In the above grub.cfg file use the inst.ks parameter to specify the location of a Kickstart_file.

For example, the following menu item may be added to grub.cfg to download a Kickstart_file ks-almalinux-8.10-minimal-x86_64.cfg from the NFS (version 3) server at IP address <server-IP>:

menuentry 'AlmaLinux 8.10 minimal Kickstart' --class centos --class gnu-linux --class gnu --class os --unrestricted {
  linuxefi (tftp)/AlmaLinux-8.10-x86_64/vmlinuz ip=dhcp inst.ks=nfs:nfsvers=3:<server-IP>:/u/kickstart/ks-almalinux-8.10-minimal-x86_64.cfg
  initrdefi (tftp)/AlmaLinux-8.10-x86_64/initrd.img
}

A Legacy PXE BIOS boot file /tftpboot/pxelinux.cfg/default example using the same Kickstart_file is:

label AlmaLinux8.10 minimal-x86_64
      menu label Clean AlmaLinux-8.10-x86_64, minimal install
      kernel AlmaLinux-8.10-x86_64/vmlinuz
      append load_ramdisk=1 initrd=AlmaLinux-8.10-x86_64/initrd.img network inst.ks=nfs:nfsvers=3:<server-IP>:/u/kickstart/ks-almalinux-8.10-minimal-x86_64.cfg vga=792

(Setting up an NFS server at <server-IP> is not discussed here.)

In the following sections we discuss relevant sections of the Kickstart_file.

Bootloader command

The bootloader command (required) specifies how the boot loader should be installed.

You should always use a password to protect your boot loader. An unprotected boot loader can allow a potential attacker to modify the system’s boot options and gain unauthorized access to the system:

  • --password If using GRUB2 as the boot loader, sets the boot loader password to the one specified with this option. This should be used to restrict access to the GRUB2 shell, where arbitrary kernel options can be passed. If a password is specified, GRUB2 will also ask for a user name, and the user name is always root.

  • --iscrypted Normally, when you specify a boot loader password using the --password= option, it will be stored in the Kickstart_file in plain text. If you want to encrypt the password, use this option and specify an encrypted password. To generate an encrypted password use the command:

    grub2-mkpasswd-pbkdf2

    Enter the password you want to use, and copy the command’s output (the hash starting with grub.pbkdf2) into the Kickstart_file. An example bootloader Kickstart entry with an encrypted password will look similar to the following:

    bootloader --iscrypted --password=grub.pbkdf2.sha512.10000.5520C6C9832F3AC3D149AC0B24BE69E2D4FB0DBEEDBD29CA1D30A044DE2645C4C7A291E585D4DC43F8A4D82479F8B95CA4BA4381F8550510B75E8E0BB2938990.C688B6F0EF935701FF9BD1A8EC7FE5BD2333799C98F28420C5CC8F1A2A233DE22C83705BB614EA17F3FDFDF4AC2161CEA3384E56EB38A2E39102F5334C47405E

Some systems require a special partition for installing the boot loader. The type and size of this partition depends on whether the disk you are installing the boot loader to uses the Master Boot Record (MBR) or a GUID Partition Table (GPT) schema. For more information, see the bootloader page.

Installation screen resolution

If you have an old server or PC where the VGA graphics adapter only supports screen resolutions up to 1024x768 or 1280x1024, then the kernel in EL8 Linux may select a higher, unsupported screen resolution which gives a flickering monitor with no image! See these pages:

You can add a vga= directive to the kernel line in the GRUB file, something like the following:

linuxefi /vmlinuz-X.Y.Z vga=792

You will, of course, see something specific in place of X.Y.Z and you can use numbers other than 792, which gives 1024×768 with 65,536 possible colors. This is a partial list of GRUB VGA Modes:

Colour depth  640x480 1024x768
8 (256)       769     773
15 (32K)      784     790
16 (65K)      785     791
24 (16M)      786     792

Linux kernel with 16-bit boot protocol

From https://www.systutorials.com/configuration-of-linux-kernel-video-mode/ we see:

  • Switching VESA modes of Linux kernel at boot time can be done by using the “vga=…“ kernel boot parameter. This parameter accept the decimal value of Linux video mode numbers instead of VESA video mode numbers.

The video mode number of the Linux kernel is the VESA mode number plus 0×200:

Linux_kernel_mode_number = VESA_mode_number + 0x200

So the table for the Kernel mode numbers are:

    | 640x480  800x600  1024x768 1280x1024
----+-------------------------------------
256 |  0x301    0x303    0x305    0x307
32k |  0x310    0x313    0x316    0x319
64k |  0x311    0x314    0x317    0x31A
16M |  0x312    0x315    0x318    0x31B

The decimal value of the Linux kernel video mode number can be passed to the kernel in the form “vga=YYY“, where YYY is the decimal value.

The parameter vga=ask is often mentioned, but is not supported by GRUB2.

Last, calculate the decimal value of the Linux video mode number. This simple python command can be used to convert a hex-number 0xYYY:

python -c "print 0xYYY"

Capture the %pre logfile

The %pre command can create a logfile:

# Start of the %pre section with logging into /root/ks-pre.log
%pre --log=/root/ks-pre.log

However, this file exists only in the memory file system during installation, and the logfile will be lost after the system has rebooted.

There are methods to get a copy of the %pre logfile:

Boot disk device selection

The server or PC computer may have multiple disk devices, and each device may have different bus interfaces to the system such as NVME or SATA.

When the Kickstart installation starts up, the file given by inst.ks must select, format and partition the system boot disk. However, you do not want to install the Linux OS on a large disk device which might be used only for data storage! Another problem is that NVME and SATA devices have different device names in the Linux kernel, for example:

and the correct device name must be given to Kickstart.

A nice and flexible solution to this issue is given in the thread https://access.redhat.com/discussions/3144131. You configure a Kickstart_file %include line where you would traditionally partition the disk:

# The file /tmp/part-include is created below in the %pre section
%include /tmp/part-include
%packages
%end

Then you define a pre-install section with %pre, here adding a number of improvements:

# Start of the %pre section with logging into /root/ks-pre.log
%pre --log=/root/ks-pre.log
# pick the first drive that is not removable and is over MINSIZE
DIR="/sys/block"
# minimum and maximum size of hard drive needed specified in GIGABYTES
MINSIZE=100
MAXSIZE=1999
# The loop first checks NVME then SATA/SAS drives:
for d in $DIR/nvme* $DIR/sd*
do
  DEV=`basename "$d"`
  if [ -d $DIR/$DEV ]; then
    # Note: the removable file may have an incorrect value:
    if [[ "`cat $DIR/$DEV/removable`" = "0" ]]
    then
      # /sys/block/*/size is in 512 byte chunks
      GB=$((`cat $DIR/$DEV/size`/2**21))
      echo "Disk device $DEV has size $GB GB"
      if [ $GB -gt $MINSIZE -a $GB -lt $MAXSIZE -a -z "$ROOTDRIVE" ]
      then
        ROOTDRIVE=$DEV
        echo "Select ROOTDRIVE=$ROOTDRIVE"
      fi
    fi
  fi
done

if [ -z "$ROOTDRIVE" ]
then
      echo "ERROR: ROOTDRIVE is undefined"
else
      echo "ROOTDRIVE=$ROOTDRIVE"
      cat << EOF > /tmp/part-include
zerombr
clearpart --drives=$ROOTDRIVE --all --initlabel
ignoredisk --only-use=$ROOTDRIVE
reqpart --add-boot
part swap --size 32768 --asprimary
part pv.01 --fstype xfs --size=1 --grow --asprimary
volgroup VolGroup00 pv.01
logvol / --fstype xfs --name=lv_root --vgname=VolGroup00 --size=32768
EOF
fi
%end

WARNING: We have some old Intel Xeon Nehalem servers with SATA disks where /sys/block/sda/removable contains an incorrect value of 1!

Disk partitions

With UEFI systems it is required to configure a special /boot/efi partition in your Kickstart_file, see also:

It is most convenient to configure boot partitions using reqpart:

  • Automatically create partitions required by your hardware platform. These include a /boot/efi for x86_64 and Aarch64 systems with UEFI firmware, biosboot for x86_64 systems with BIOS firmware and GPT, and PRePBoot for IBM Power Systems.

An example Kickstart_file section specifying disk partitions and using reqpart may be:

reqpart --add-boot
part swap --size 50000 --asprimary
part pv.01 --fstype xfs --size=1 --grow --asprimary
volgroup VolGroup00 pv.01
logvol / --fstype xfs --name=lv_root --vgname=VolGroup00 --size=32768

Disable Secure Boot in BIOS

If the PXE client system BIOS is configured for UEFI Secure_Boot then the PXE boot will fail with an error about an invalid signature. See also What is UEFI Secure Boot and how it works?.

As explained in Installation of RHEL8 on UEFI system with Secure Boot enabled fails with error ‘invalid signature’ on vmlinuz RedHat is currently working on a solution for RHEL 8.

Workaround: Disable secureboot from BIOS settings.

efibootmgr - manipulate the EFI Boot Manager

efibootmgr is a userspace application used to modify the Intel Extensible Firmware Interface (EFI) Boot Manager. This application can create and destroy boot entries, change the boot order, change the next running boot option, and more.

To show the current boot order:

efibootmgr -v

Some useful command options (see the efibootmgr page):

-n | --bootnext XXXX   set BootNext to XXXX (hex)
-N | --delete-bootnext delete BootNext
-o | --bootorder XXXX,YYYY,ZZZZ,...     explicitly set BootOrder (hex)
-O | --delete-bootorder   delete BootOrder

UEFI network boot process

In this section we describe how a computer doing an UEFI PXE boot will download a GRUB2 bootfile from the network server and execute it. Please note:

  • This GRUB2 information has been copied from the local Linux grub.html manual’s Network section in /usr/share/doc/grub2-common/grub.html because the original manual from gnu.org is frequently inaccessible. Make sure that the package grub2-common containing the grub.html file has been installed on your PC.

  • The grub.cfg file is placed in the same directory as the path output by grub-mknetdir hereafter referred to as (FWPATH). Note: Our setup uses FWPATH=/tftpboot/uefi.

The PXE bootloader image /tftpboot/uefi/BOOTX64.EFI executing in the computer’s NIC adapter will search for GRUB2 configuration files in order using the following rules, where the appended value corresponds to a value on the client machine:

(FWPATH)/grub.cfg-(UUID OF NIC)
(FWPATH)/grub.cfg-(MAC ADDRESS OF NIC)
(FWPATH)/grub.cfg-(IPv4 OR IPv6 ADDRESS)
(FWPATH)/grub.cfg

Hint: Use the gethostip command from the syslinux RPM package to convert hostnames and IP-addresses to hexadecimal, for example:

$ gethostip -f s001
s001.(domainname) 10.2.130.21 0A028215
$ gethostip -x s001
0A028215

The client will only attempt to look up an IPv6 address config once, however, it will try the IPv4 address multiple times. The first file in this list which can be downloaded successfully will be used for network booting. The concrete example below shows what would happen under the IPv4 case:

The GRUB2 bootloader will attempt TFTP download of this list of configuration files in order:

(FWPATH)/grub.cfg-7726a678-7fc0-4853-a4f6-c85ac36a120a
(FWPATH)/grub.cfg-52-54-00-ec-33-81
(FWPATH)/grub.cfg-0A000082
(FWPATH)/grub.cfg-0A00008
(FWPATH)/grub.cfg-0A0000
(FWPATH)/grub.cfg-0A000
(FWPATH)/grub.cfg-0A00
(FWPATH)/grub.cfg-0A0
(FWPATH)/grub.cfg-0A
(FWPATH)/grub.cfg-0
(FWPATH)/grub.cfg

After GRUB2 has started, files on the TFTP server will be accessible via the (tftp) device.

The server IP_address can be controlled by changing the (tftp) device name to (tftp,server-ip). Note that this should be changed both in the prefix and in any references to the device name in the configuration file.