Modifying the Kernel to Improve Performance

Modifying the Kernel to Improve Performance

Introduction

Like a government that rules a nation and all its provinces, the Linux kernel is the central program that not only governs how programs interact with one another, but also provides the guidelines on how they should use the computer's core infrastructure, such as memory, disks, and other input/output (I/O) devices for the user's benefit.

Linux drivers, the programs that manage each I/O device, are the staff that keeps all the government departments running. Continuing with the analogy, the more departments you make the kernel manage, the slower Linux becomes. Large kernels also reduce the amount of memory left over for user applications. These may then be forced to juggle their memory needs between RAM and the much slower swap partitions of disk drives, causing the whole system to become sluggish.

The Fedora installation CDs have a variety of kernel RPMs, and the installation process autodetects the one best suited to your needs. For this reason, the Fedora Linux kernel installed on your system is probably sufficient. The installation process chooses one of several prebuilt kernel types depending on the type of CPU and configuration you intend to use (Table 33-1).

Table 33-1: Kernels Found On Fedora Installation CDs

The Pros And Cons Of Kernel Upgrades

Despite this best fit installation, you may want to rebuild the kernel at times. For example, there is no installation RPM for multiprocessor systems with large amounts of memory. You may also want to experiment in making a high-speed Linux router without support for SCSI, USB, Bluetooth, and sound but with support for a few NIC drivers, an IDE hard drive, and a basic VGA console. This would require a kernel rebuild.

Rebuilding the kernel in a small business environment is usually unnecessary. If your system starts to slow down and you can't afford to replace hardware or are unable to add more RAM, however, you may want to tune the kernel by making it support only necessary functions or updating built-in parameters to make it perform better. Sometimes new features within the new kernel are highly desirable; for example, the version 2.6 kernel has much more efficient data handling capabilities than the older version 2.4, providing new life for old hardware.

Kernel tuning on a production server shouldn't be taken lightly, because the wrong parameters could cause your system to fail to boot, software to malfunction, or hardware peripherals to become unavailable. Always practice on a test system and keep a backup copy of your old kernel. Whenever possible, hire a consultant with kernel experience to help, and use this chapter and other references as a guide to prepare you for what to expect.

This chapter provides only an overview of the steps to take. It won't make you an expert, but it will expose you to the general process and provide greater confidence when you need to research the task with a specialized guide.

The Kernel Sources Package

You will need to install the kernel source code on your system prior to modifying your kernel. As of Fedora Core 3, the kernel sources come as a source RPM package that matches the version of the kernel you are running. In Fedora Core 2 and earlier, the Kernel sources came as a generic RPM package called kernel-source, the installation of which is covered in Appendix III, 'Fedora Version Differences'.

The newer method is more complicated as it requires a number of post installation steps. Though the process is well documented in the release notes section of the Fedora website (https://fedora.redhat.com/docs/release-notes/) there are some clarifications that are needed. These are explained in the following section.

Installing Kernel Sources

The installation process for the kernel sources is long, but not very complicated. Here is how it's done:

1. Determine the version of your kernel with the uname command. In this case it is version 2.6.14-1.1644.

[root@bigboy tmp]# uname -r

2.6.14-1.1644_FC4smp

[root@bigboy tmp]#

2. Visit your favorite Fedora operating system download mirror and get the corresponding source RPM package. If you are running the original version of the kernel that came with your installation discs, then the sources will be located in the /core/<version>/i386/os/SRPMS/ directory. If the kernel has been updated using yum or some other method, the sources will be located in the /core/updates/<version>/SRPMS/ directory. In this example the sources of an updated Core 4 kernel is downloaded from the /core/updates/4/SRPMS/ directory of https://download.fedora.redhat.com using wget.

[root@bigboy tmp]# wget

https://download.fedora.redhat.com/pub/fedora/linux/core/updates/4/SRPMS/kernel-2.6.14-1.1644_FC4.src.rpm

https://download.fedora.redhat.com/pub/fedora/linux/core/updates/4/SRPMS/kernel-2.6.14-1.1644_FC4.src.rpm

=> `kernel-2.6.14-1.1644_FC4.src.rpm'

Resolving download.fedora.redhat.com 66.187.224.20,

Connecting to

download.fedora.redhat.com|66.187.224.20|:80

connected.

HTTP request sent, awaiting response 200 OK

Length: 40,454,218 (39M) [application/x-rpm]

100%[=========================>] 40,454,218 862.89K/s ETA 00:00

15:33:10 (842.22 KB/s) -

`kernel-2.6.14-1.1644_FC4.src.rpm' saved

[root@bigboy tmp]#

3. Install the contents of the RPM file into the /usr/src/redhat/SOURCES and /usr/src/redhat/SPECS directories using the rpm command.

[root@bigboy tmp]# rpm -Uvh kernel-2.6.14-1.1644_FC4.src.rpm

1:kernel ########################################### [100%]

[root@bigboy tmp]#

4. The kernel source directory tree will have to be created next. Enter the /usr/src/redhat/SPECS directory and create the tree using the rpmbuild command with the -bp option.

[root@bigboy tmp]# cd /usr/src/redhat/SPECS

[root@bigboy SPECS]# ls

kernel-2.6.spec

[root@bigboy SPECS]# rpmbuild -bp --target $(arch) kernel-2.6.spec

Building target platforms: i686

Building for target i686

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.44004

+ umask 022

+ cd /usr/src/redhat/BUILD

removed `./init/Kconfig.orig'

removed `./init/main.c.orig'

+ find . -name '*~' -exec rm -fv '' ';'

+ exit 0

[root@bigboy SPECS]#

5. The tree has now been created in the /usr/src/redhat/BUILD/kernel-<version> directory. You can link, move or copy this directory to become /usr/src/linux depending on your needs and the likelihood of you compiling multiple kernel versions in future. In this case, the tree is moved and then linked.

[root@bigboy SPECS]# cd /usr/src/redhat/BUILD/kernel-2.6.14/

[root@bigboy kernel-2.6.14]# ls

linux-2.6.14 vanilla

[root@bigboy kernel-2.6.14]# mv linux-2.6.14 /usr/src/

[root@bigboy kernel-2.6.14]# cd /usr/src

[root@bigboy src]# ln -s ./linux-2.6.14 linux

[root@bigboy src]# ls

kernels linux linux-2.6.14 redhat

[root@bigboy src]#

6. The configuration files for specific kernels shipped in Fedora Core will be located in the configs/ directory. In this case the uname -a command is used to determine the systems CPU type (686) and the kernel type (SMP, Symmetrical Multi Processor), and the relevant configuration file is then copied to become the new /usr/src/linux/.config file to be used during the kernel compilation.

[root@bigboy src]# cd /usr/src/linux

[root@bigboy linux]# uname -a

Linux bigboy.my-web-site.org 2.6.14-1.1644_FC4smp #1 SMP

Sun Nov 27 03:39:31 EST 2005 i686 i686 i386 GNU/Linux

[root@bigboy linux]# cp configs/kernel-2.6.14-i686-smp.config .config

cp: overwrite `.config'? y

[root@bigboy linux]#

7. You can also automatically copy the correct file from the /usr/src/linux/configs/ directory to /usr/src/linux/.config using the make oldconfig command.

[root@bigboy linux]# cd /usr/src/linux

[root@bigboy linux]# make oldconfig

You should now be ready to compile a customized kernel at your leisure, but first I'll discuss Kernel modules.

Kernel Modules

Over the years the Linux kernel has evolved, in the past device drivers were included as part of this core program, whereas now they are loaded on demand as modules.

Reasons For Kernel Modules

There are a number of advantages to this new architecture:

Updates to a driver, such as a USB controller module, don't require a complete recompilation of the kernel; just the module itself needs recompiling. This reduces the likelihood of errors and ensures that the good, working base kernel remains unchanged.

An error in a device driver is less likely to cause a fault that prevents your system from booting. A faulty device driver can be prevented from loading at boot time by commenting out its entry in the /etc/modprobe.conf file, or by using the rmmod command after boot time. In the past, the kernel would have to be recompiled.

Updates to a driver don't require a reboot either, just the unloading of the old and reloading of the new module.

You can add new devices to your system without requiring a new kernel, only the new module driver is needed. This adds a great deal of flexibility without a lot of systems administrator overhead.

There are some drivers that will always need to be compiled into the kernel to make sure your system boots correctly. For example, routines for basic system functions used in reading and writing files, are an indispensable integrated part of any kernel.

Loadable kernel modules now include device drivers to manage various types of filesystems, network cards, and terminal devices to name a few. As they work so closely with the kernel, the modules need to be compiled specifically for the kernel they are intended to support. The kernel always looks for modules for its version number in the /lib/modules/<kernel version> directory and permanently loads them into RAM memory for faster access. Some critical modules are loaded automatically, others need to be specified in the /etc/modprobe.conf file.

The kernel recompilation process provides you with the option of compiling only the loadable modules. I won't specifically cover this, but simultaneous recompilation of all modules will be covered as part of the overall recompilation of your kernel.

How Kernel Modules Load When Booting

One question that must come to mind is 'How does the kernel boot if the disk controller modules reside on a filesystem that isn't mounted yet?'

As stated in Chapter 7, 'The Linux Boot Process', the GRUB boot loader resides on its own dedicated partition and uses the /boot/grub/grub.conf file to determine the valid kernels and their locations. The grub.conf file not only defines the available kernels, but also the location of the root partition and an associated ramdisk image that is automatically loaded into memory and that contains just enough modules to get the root filesystem mounted.

Note: In Fedora Linux, the /boot/grub/grub.conf file can also be referenced via the symbolic link file named /etc/grub.conf.

Modules And The grub.conf File

In this example of the /boot/grub/grub.conf file, the kernel in the /boot directory is named vmlinuz-2.6.8-1.521, its RAM disk image file is named initrd-2.6.8-1.521.img, and the root partition is (hd0,0).

# File: /boot/grub/grub.conf

default=0

timeout=10

splashimage=(hd0,0)/grub/splash.xpm.gz

title Fedora Core (2.6.8-1.521)

root (hd0,0)

kernel /vmlinuz-2.6.8-1.521 ro root=LABEL=/

initrd /initrd-2.6.8-1.521.img

The .img file is created as part of the kernel compilation process, but can also be created on demand with the mkinitrd command.

The (hd0,0).disk definition may seem strange, but there is a file that maps the GRUB device nomenclature to that expected by Linux in the /boot/grub/device.map file.

# File: /boot/grub/device.map

(fd0)  /dev/fd0

(hd0)  /dev/hda

During the next phase of the boot process, the loaded kernel executes the init program located on the RAM disk, which mounts the root filesystem and loads the remaining modules defined in the /etc/modprobe.conf file before continuing with the rest of the startup process.

Loading Kernel Modules On Demand

It is possible to load add-on modules located under the /lib/modules/<kernel version> directory with the modprobe command. For example, the iptables firewall application installs kernel modules that it uses to execute NAT and pass FTP traffic. In this example: these modules are loaded with the modprobe command with the aid of the /etc/rc.local script.

# File: /etc/rc.local

# Load iptables FTP module when required

modprobe ip_conntrack_ftp

# Load iptables NAT module when required

modprobe iptable_nat

Kernel module drivers that are referenced by the operating system by their device aliases are placed in the /etc/modprobe.conf file and are loaded automatically at boot time. In the example, you can see that devices eth1 and eth0 use the natsemi and orinoco_pci drivers respectively.

# /etc/modprobe.conf

alias eth1 natsemi

alias eth0 orinoco_pci

Linux has a number of commands to help you with modules. The lsmod command lists all the ones loaded. In the example, you can see that iptables, NFS, and the Orinoco drivers are all kernel modules. You can use the modprobe command to load and unload modules or use the insmod and rmmod commands. See the man pages for details.

[root@bigboy tmp]# lsmod

Module  Size Used by

iptable_filter  2048 0

ip_tables  13440 1 iptable_filter

exportfs  4224 1 nfsd

nfs  142912 0

lockd  47944 3 nfsd,nfs

autofs4  10624 1

sunrpc  101064 20 nfsd,nfs,lockd

natsemi  18016 0

orinoco_pci  4876 0

orinoco  31500 1 orinoco_pci

hermes  6528 2 orinoco_pci,orinoco

[root@bigboy tmp]#

Finally, when in doubt about a device driver, try using the lspci command to take a look at the devices that use your PCI expansion bus. Here you can see that the natsemi module listed in the lsmod command has a high probability of belonging to the 01:08.0 Ethernet controller: device made by National Semiconductor.

[root@bigboy tmp]# lspci

01:07.0 Network controller: Intersil Corporation Prism 2.5 Wavelan chipset (rev 01)

01:08.0 Ethernet controller: National Semiconductor Corporation DP83815 (MacPhyter) Ethernet Controller

01:0c.0 Ethernet controller: 3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)

[root@bigboy tmp]#

Creating A Custom Kernel

The installation of the kernel sources creates a file called README in the /usr/src/linux directory that briefly outlines the steps needed to create a new kernel. Take a look at a more detailed explanation of the required steps.

Make Sure Your Source Files Are In Order

Cleaning up the various source files is the first step. This isn't so important for a first time rebuild, but it is vital for subsequent attempts. You use the make mrproper command to do this; it must be executed in the Linux kernel version's subdirectory located under /usr/src. In this case, the subdirectory's name is /usr/src/linux-2.6.5-1.358.

[root@bigboy tmp]# cd /usr/src/linux

[root@bigboy linux]# make mrproper

[root@bigboy linux]#

The '.config' File

You next need to run scripts to create a kernel configuration file called /usr/src/linux/.config. This file lists all the kernel options you wish to use.

Backup Your Configuration

The .config file won't exist if you've never created a custom kernel on your system before, but fortunately, RedHat stores a number of default .config files in the /usr/src/linux/configs directory. You can automatically copy the .config file that matches your installed kernel by running the make oldconfig command in the /usr/src/linux directory.

[root@bigboy tmp]# cd /usr/src/linux

[root@bigboy linux]# ls .config

ls: .config: No such file or directory

[root@bigboy linux]# make oldconfig

[root@bigboy linux]#

If you've created a custom kernel before, the .config file that the previous custom kernel build used will already exist. Copy it to a safe location before proceeding.

Customizing The '.config' File

Table 33-2 lists three commands that you can run in the /usr/src/linux directory to update the .config file.

Table 33-2 Scripts For Modifying The .config File

Each command prompts you in different ways for each kernel option, each of which generally provides you with the three choices shown in Table 33-3. A brief description of each kernel option is given in Table 33-4.

Table 33-3 Kernel Option Choices

Table 33-4 Kernel Configuration Options

Configure Dependencies

As I mentioned before, the .config file you just created lists the options you'll need in your kernel. In version 2.4 of the kernel and older, the make dep command was needed at this step to prepare the needed source files for compiling. This step has been eliminated as of version 2.6 of the kernel.

Edit The Makefile To Give The Kernel A Unique Name

Edit the file Makefile and change the line 'EXTRAVERSION =' to create a unique suffix at the end of the default name of the kernel.

For example, if your current kernel version is 2.6.5-1.358, and your EXTRAVERSION is set to -6-new, your new additional kernel will have the name vmlinuz-2.6.5-6-new.

Remember to change this for each new version of the kernel you create.

Compile A New Kernel

You can now use the make command to create a compressed version of your new kernel and its companion .img RAM disk file. This could take several hours on a 386 or 486 system. It will take about 20 minutes on a 400MHz Celeron.

[root@bigboy linux-2.6.5-1.358]# make

[root@bigboy linux-2.6.5-1.358]#

Note: In older versions of Fedora the command to do this would have been make bzImage.

Build The Kernel's Modules

You can now use the make modules_install command to copy all the modules created in the previous step to the conventional module locations.

[root@bigboy linux]# make modules_install

[root@bigboy linux]#

Note: In versions of Fedora before Core 3, this was a two step process. The make modules command would compile the modules, but locate them within the Linux kernel source directory tree under the directory /usr/src/. The make modules_install command would then relocates them to where they should finally reside under the /lib/modules/<kernel version> directory.

Copy The New Kernel To The /boot Partition

The kernel and the .img you just created needs to be copied to the /boot partition where all your systems active kernel files normally reside. This is done with the make install command.

This partition has a default size of 100MB, which is enough to hold a number of kernels. You may have to delete some older kernels to create enough space.

[root@bigboy linux]# make install

[root@bigboy linux]#

Here you can see that the new kernel vmlinuz-2.6.5-1.358-new is installed in the /boot directory.

[root@bigboy linux]# ls -l /boot/vmlinuz*

lrwxrwxrwx 1 root root 22 Nov 28 01:20 /boot/vmlinuz -> vmlinuz-2.6.5-1.358-new

-rw-r--r-- 1 root root 1122363 Feb 27 2003 /boot/vmlinuz-2.6.5-1.358

-rw-r--r-- 1 root root 1122291 Nov 28 01:20 /boot/vmlinuz-2.6.5-1.358-new

[root@bigboy linux]#

Updating GRUB

You should now update your /etc/grub.conf file to include an option to boot the new kernel. The make install command does this for you automatically.

In this example, default is set to 1, which means the system boots the second kernel entry, which happens to be that of the original kernel 2.6.5-1.358. You can set this value to 0, which makes it boot your newly compiled kernel (the first entry).

default=1

timeout=10

splashimage=(hd0,0)/grub/splash.xpm.gz

title Red Hat Linux (2.6.5-1.358-new)

root (hd0,0)

kernel /vmlinuz-2.6.5-1.358-new ro root=LABEL=/

initrd /initrd-2.6.5-1.358-new.img

title Red Hat Linux (2.6.5-1.358)

root (hd0,0)

kernel /vmlinuz-2.6.5-1.358 ro root=LABEL=/

initrd /initrd-2.6.5-1.358.img

Kernel Crash Recovery

Sometimes the new default kernel will fail to boot or work correctly with the new kernel. A simple way of recovering from this is to reboot your system, selecting the old version of the kernel from the Fedora splash screen. Once the system has booted with this stable version, edit the grub.conf file and set the default parameter to point to the older version instead. If this fails, you may want to boot from a CD with the original kernel. You can then try to either reinstall a good kernel RPM or rebuild the failed one over again after fixing the configuration problem that caused the trouble in the first place.

How To Create A Boot CD

The kernel in Fedora Core 2 and higher is too big to fit on a floppy disk, so you'll have to create a boot CD instead. Here are the steps.

1. Each installed kernel has a dedicated subdirectory for its modules in the /lib/modules directory. Get a listing of this directory. Here there are two installed kernels; versions 2.6.5-1.358custom and 2.6.8-1.521.

[root@bigboy tmp]# ls /lib/modules/

2.6.5-1.358custom 2.6.8-1.521

[root@bigboy tmp]#

2. Select the desired kernel and use the mkbootdisk command to create a CD ISO image named /tmp/boot.iso of one of the kernels, in this case 2.6.8-1.521:

[root@bigboy tmp]# mkbootdisk --iso --device /tmp/boot.iso

2.6.8-1.521

3. Burn a CD using the image. This creates a boot CD with the specified kernel, named vmlinuz, and a scaled-down version of the grub.conf configuration file named isolinux.cfg, both located in the isolinux subdirectory of the CD. This example mounts the newly created CD-ROM and takes a look at the isolinux.cfg file to confirm that everything is okay.

[root@bigboy tmp]# mount /mnt/cdrom

[root@bigboy tmp]# ls /mnt/cdrom/isolinux/

boot.cat boot.msg initrd.img isolinux.bin isolinux.cfg TRANS.TBL vmlinuz

[root@bigboy tmp]# cat /mnt/cdrom/isolinux/isolinux.cfg

default linux

prompt 1

display boot.msg

timeout 100

label linux

kernel vmlinuz

append initrd=initrd.img ro root=/dev/hda2

[root@bigboy tmp]#

When you reboot your system with the CD, the boot process automatically attempts to access your files in the /root partition and boot normally. The only difference being that the kernel used is on the CD.

Updating The Kernel Using RPMs

It is also possible to install a new standardized kernel from an RPM file. As you can see, it is much simpler than creating a customized one.

To create an additional kernel using RPMs, use the command

[root@bigboy tmp]# rpm -ivh kernel-file.rpm

To replace an existing kernel using RPMs, you need only one line

[root@bigboy tmp]# rpm -Uvh kernel-file.rpm

Conclusion

Building a customized Linux kernel is probably something that most systems administrators won't do themselves. The risk of having a kernel that may fail in some unpredictable way is higher when you modify it, and, therefore, many system administrators hire experts to do the work for them. After reading this chapter, at least you will have an idea of what is going on when the expert arrives, which can help considerably when things don't go according to plan.