1. Introduction

In Linux device drivers can be implemented as kernel modules, which are object code files that can be loaded (and unloaded) dynamically by the Linux kernel, either at boot time or later. This makes it possible to install a device driver without having to recompile the kernel or even reboot the operating system.

The goal of this lab class is to develop a basic "char device driver" that in the next project lab will become a simple device driver for the standard serial port of a PC.

IMP.- A pre-requisite for this lab is that you have completed successfully the previous lab, i.e. that you have imported to VirtualBox the appliance with Linux that we have made available.

2. Steps

In order to achieve this goal, we will walk you through the following steps:

1. Develop a very simple "char device driver" that has only the initialization and the cleanup functions of a module.
2. Add the open() and close() system calls.
3. Add write() and read() calls to the device driver.

After each step, you will have to install the device driver in the kernel, and later remove it. Furthermore, in the last two steps, you will have to write, and run, a simple program for testing the device driver operations.

3. Setup (for the lab PCs)

If you are using one of the lab's PCs, you need to setup the VBox VM executing the following steps (assuming you have completed the previous project lab):

Remove the VM configuration

If you have imported the VirtualBox appliance in the last lab class, you have now to delete the corresponding configuration as described in Section 2.3 of the first DD handout (DD1).

$ cd ~/VirtualBox\ VMs/ $ rm -rf CICA2

Import the VirtualBox appliance

As described in Section 2.1 of the first DD handout (DD1).

Configure the shared folders

As described in Section 2.2 of the first DD handout (DD1).

You should not need to repeat these steps until you reboot the host OS.

4. The hello module

Before start writing code we will do a small warm-up, by compiling and installing a very simple module.

4.1 Source code

In this Section we'll use the Hello World module described in Chapter 2 of the Linux Device Drivers, 3rd Ed. book (LDD3).

The code for this module can be found in the archive with the example code of the LDD3 book available from the LDD3 O'Reilly's page. (Just click on the "Example Code" link on the list below the book's front cover image.) This archive is already in the home directory of the so user (of the Guest OS), and it was already uncompressed to the directory ~so/examples/.

In the guest (OS) create a directory in the folder shared between the guest and host OS's for this lab, which we'll assume to be /host/lab2/. In this directory, create a subdirectory named /host/lab2/hello. Copy the files hello.c and Makefile in ~so/examples/misc-modules/, to /host/lab2/hello/.

In the host (OS) open the hello.c file with your favorite text editor.

At a first glance this file is just an ordinary C source file. It includes two header files <linux/init.h> and <linux/module.h>, and defines two (static) functions hello_init() and hello_exit().

On a closer look, a few points are worth mentioning:

MODULE_LICENSE("Dual BSD/GPL")

is a macro that specifies the type of licensing of the source code. You can omit this. For more details check pg. 30 in Ch. 2 of the LDD3 book.

module_init(hello_init);

is a macro that declares hello_init() as the module's initialization function

module_exit(hello_exit);

is a macro that declares hello_exit() as the module's cleanup function

Furthermore, note that although the code of the module functions are C, there are a few restrictions that we'll present as they arise. At this point it is worth mentioning that module functions are part of the kernel, which is not linked with the standard C library. Hence module functions cannot call C library functions like printf(). Instead, they can only call functions such as printk() that belong to the kernel API.

The printk() function is similar to printf(). Note that the string KERN_ALERT is used to specify the priority of the message but is not an argument: it is not separated by a comma from the format string. For more details on printk() check pg. 76 in Ch. 4 of the LDD3 book.

4.2 Compilation

To compile the hello.c file with the source code of the hello kernel module, we'll use make, which is a program utility that was designed to build large programs and that is used by the Linux kernel.

The file Makefile that you have copied to the /host/lab2/hello directory is a "script" that tells the make utility how to compile a module. Note that this makefile is somewhat different from a standard makefile, because it takes advantage of the kernel build system to make things easier. For more details on module compilation check the Section Compiling Modules on pg. 23 of Ch. 2 of the LDD3 book.

In order to compile the hello module you have to change the 3rd line from the bottom:

to

i.e. change the module to be built from hellop.o to hello.o. Note that the string obj-m must be preceded by exactly one tab.

To make this change you can use your favorite text editor on the host OS. Afterwards, change to the guest OS and in the /host/lab2/hello directory give the command:

A successful compilation will create several output files including hello.ko the object file with the hello kernel module.

4.3 Installation and Removal

Once the kernel module has been compiled you can install it. This is done by invoking the insmod utility. However, an ordinary user such as so does not have permission to install a kernel module (why?). Thus, on the guest OS, you need to execute insmod as super user using the sudo command:

I.e., upon executing insmod, once you have typed in the password of user so, the initialization code is executed and the string Hello, world is printed. You can check that the module is indeed installed by invoking on the guest OS the command:

Use the man pages to find out what lsmod and grep do. If you want further information on the module you can invoke the command

Finally, you can unload the module by executing on the guest OS the rmmod command as super-user as follows:

I.e. upon unloading the module the hello_exit() function is executed printing the string Goodbye cruel world

5. The echo pseudo char device driver

In this section we'll outline the development of the echo pseudo char device driver that just writes to the console what is written to the pseudo char device. (We call this a pseudo char device driver because it does not control any I/O device, not physical not even virtual.)

5.0. Linux Kernel API

In the development of device drivers you cannot use the C library functions you usually use when developing user-level programs. For example, you cannot use the standard printf() function to print some string on the screen. Instead you have to use printk() (strings printed with this instruction are also written to the file /var/log/syslog, which only root can read).

The function printk() belongs to the kernel API, which includes many functions similar to those that you find in the standard C library. You should browse the above link to get an idea of which functions you can use.

In this lab, in addition to printk() you need to use also functions to manage dynamically allocated memory. Indeed, as you shall see below, you will have to allocate buffers in the kernel to transfer data between a user process and the echo device. For that you can use the kernel function declared in <linux/slab.h>:

Except for the second argument, this call is very similar to malloc() of the standard C library. A detailed explanation of the the kmalloc kernel function can be found in Ch. 8 of the LDD3 book, on pg. 213 and following. For now, it suffices to say that you can use the flag GFP_KERNEL, which is defined in <linux/mm.h>

Another interesting function for dynamically allocating memory is:

which not only dynamically allocates memory but also sets it to zero. This may be very important to prevent leaking data from the kernel, or other users, to user space.

Any memory dynamically allocated with kmalloc()/kzalloc() must be freed with kfree():

Failure to do that will create a memory leak and eventually the kernel will run out of memory. Note that unlike terminating an application, unloading a device driver will not automatically free all the memory it dynamically allocated and did not free. Indeed, a kernel module is not a program but part of the kernel. To recover the memory dynamically allocated by a module that was not freed before the module was unloaded, you will need to reset the operating system.

5.1. Allocation of a major device number and creation of the special file

5.1.1 Character and block devices, major and minor numbers

In Unix, most devices fall into one of two classes: character devices and block devices. Essentially, the difference is that whereas the latter allow accessing "blocks" of data independently of one another, the former can be accessed only as a stream of characters/bytes. Nevertheless, both char(acter) and block devices are accessed through names in the filesystem. These names are by convention under the /dev/ directory and the corresponding nodes in the file system are known as special files or device files.

The kernel also uses two numbers known as major and minor numbers to identify a device. Essentially, the major number identifies a device driver, whereas the minor number is driver specific, but usually specifies the device being referred to.

Kernel data structures use the dev_t type defined in <linux/types.h> to hold the major and minor device number, and to obtain the major and the minor numbers of a device from a dev_t value you should use the macros:

Inversely, to obtain the dev_t value from the major and the minor numbers you should use the macro:

Whereas in the past some major numbers have been assigned statically, nowadays major device numbers are being assigned dinamically only. Therefore, one of the first tasks of a device driver is to request the kernel to allocate one using the function:

Note that dev is an output-only parameter that will on success hold a dev_t struct for the first device in the allocated range. firstminor specifies the first minor number requested, usually 0.

In principle, in the module's cleanup function you should free the allocated device number region by invoking the function:

Dynamically allocating the major number has a disadvantage: it is not possible to create the device nodes in advance. One way around this is to write a script that immediately after loading the device driver creates the required device nodes. This is possible because the kernel provides via the /proc interface, more specifically the /proc/devices file the major number assigned to each device. On the guest check the major numbers that have been assigned so far, by typing:

The following bash script can be used to load in the kernel a module that uses a dynamically allocated major device number and create the respective device file in /dev/:

Likewise, the following bash script removes a module from the kernel and deletes the respective device file in /dev/:

Both scripts take as an argument the name of the module to load/unload (without the .ko extension). This name must be the last argument of the call to alloc_chrdev_region(), and is also used as the name of the respective device file. Remember, both scripts must be executed as super user, therefore you should use the sudo command.

5.1.2 Todo

The following list enumerates the steps that you should take to create and test the first version of the echo device driver. I suggest that you look at the code of the scullc (~so/examples/scullc/main.c) device driver mentioned above to perform steps 4 and 5. (Focus on the initialization and the cleanup functions only.) Note that all the steps on the guest OS but the last two should be done as user so.

1. Create directory /host/lab2/echo by executing the command:
   cp -r hello echo
   in /host/lab2
2. Clean up the newly created directory by executing the command:
   make clean
   This will execute the clean target specified in the file Makefile.
3. Change the Makefile's "obj-m line", to build the right module
4. Create the first version of file echo.c from file hello.c. E.g.
   1. Change the name of file hello.c to echo.c (e.g. use the command mv)
   2. Using a text editor replace the string hello by the string echo
5. Modify the initialization function to:
   1. dynamically allocate a major device number
   2. print the allocated major device number, instead of the salutation
   A few notes regarding this:
   1. The dev_t variable that you need to dynamically allocate a major device number will be needed later to free that major device number in the module cleanup function. Therefore, I suggest that you use a static global variable for that purpose.
   2. An alternative is to save just the device major number and later use the MKDEV() macro to obtain the corresponding dev_t value, as done in the scullc device driver. You can take a look at the initialization function of the scullc device driver in ~so/examples/scullc/main.c of the guest, although it has a lot more code than what you really need.
   3. The load script assumes that the device's name is the module's name. Therefore, you should use "echo" as the last argument of the function to dinamically allocate the major device number, alloc_chrdev_region()
   4. For the header files required check the Quick Reference section of Ch. 3 of the LDD3 book, on pg. 70.
   5. The echo.h header file includes a few definitions that can be helpful.
6. Modify the cleanup function to:
   1. free the allocated major device number
   2. print a different fairwell message, e.g. reporting the major device number being freed
   Again, if you wish, you can take a look at the cleanup function of the scullc device driver in ~so/examples/scullc/main.c of the guest, although it has a lot more code than what you really need.
7. On the guest, compile your first device driver by executing make
8. Use the above load script to load the echo kernel module. (If you wish, you may adapt it to your needs.)
   1. Ensure that the script has the appropriate permissions. (Use the chmod command if needed.)
   2. Check that the echo module is loaded
   3. Check that the script created the device files in the /dev on the guest
   4. Check yourself the major device number assigned by the kernel via the /proc file system
   Note As already mentioned, you should run the load script using the sudo command.
9. Use the above unload script to unload the echo kernel module. (If you wish, you may adapt it to your needs.)
   1. Check that the echo module is unloaded
   2. Check that the script has removed the device files from the /dev of the guest
   3. Check that major device number assigned by the kernel to echo has been released
   Note As already mentioned, you should run the unload script using the sudo command.

In Unix/Linux, before invoking any operation on a file, be it a regular file or a special file, an application must first open that file. Likewise, once an application is done with using a file it should close it. Therefore, in this section you will implement the file operations needed for the implementation of the open and close system calls.

The different operations required to implement system calls such as open() and read() supported by a device driver are maintained in a kernel data structure of type struct file_operations (<linux/fs.h>). Essentially, this data structure contains a set of pointers to functions, each of which implements the operations required for the implementation of a system call. You can find a detailed description of this structure in Ch. 3 of the LDD3 book, on pg. 49 and following.

5.2.1 Character device registration

Internally, the kernel uses the struct cdev (<linux/cdev.h>) to represent a char device. Thus, before the kernel invokes its operations, the device driver must allocate it, using the function:

The cdev struct has a member that points to a struct file_operations, which must be initialized before the kernel can invoke its operations. The cdev struct also has a a member named owner, which is used to point to the kernel module to which the drive belongs. These two members can be initialized as follows:

where fops is a pointer to a struct file_operations previously initialized.

After initializing struct cdev, the device driver must register it in the kernel, by invoking:

As mentioned above, the main purpose of device registration is to "tell" the kernel which operations are supported by the device, thus the right place to do it is in the driver's initialization function.

Likewise, the driver's cleanup function should remove from the system all cdev structs it has previously registered, by calling the kernel function:

5.2.2 More data structures

In addition to the data structures described in the previous sections there are 2 other kernel data structures defined in <linux/fs.h> that you need to be aware of when developing a device driver (actually, these data structures are not specific to device drivers, but rather related to the file system):

struct file

It represents a file descriptor, which is returned on an open system call. Among other members, this struct includes the member void *private_data; that can be used by a device driver to maintain state information across system calls. You can find a detailed description of this structure in Ch. 3 of the LDD3 book, on pg. 53 and following.

struct inode

This represents a file, a regular or a special file. Note that even though a given file may be opened simultaneously by different processes, the kernel keeps a single struct inode. Among other members, this struct includes the member struct cdev *i_cdev that points to the cdev struct of the corresponding device. You can find important information on this structure in Ch. 3 of the LDD3 book, on pg. 55.

5.2.3 The open operation

The open operation:

is invoked on every open of the special file. It is used to do any initialization of the driver in preparation for later operations. This is usually device specific. However, often there is the need to find out which device is being opened. This can be easily obtained via the i_cdev member of the struct inode.

Because the read() and write() operations provided by the device driver do not take as argument the address of a struct inode, but only the address of a struct file, and usually these functions also need to know on which device to perform the read/write operation, usually the private_data member of the struct file is initialized in open() with the address of the corresponding struct cdev.

5.2.4 The release operation

The release operation

is invoked by the kernel on the last close() of the corresponding file. It usually undoes what the open operation has done, e.g. free data structures.

Note that not every invocation of the close() system call leads to the invocation of the release operation. If you want some operation to be performed on every invocation of the close() system call, define a flush() file operation:

This operation is indeed invoked on every close() system call.

A more detailed discussion of these issues can be found in Ch. 3 of the LDD3 book, on pg. 59.

5.2.5 To do

1. Modify echo's initialization function to register one device after allocating the major and minor device numbers. A few notes regarding this step:
   1. Given that you'll need to unregister the device in the cleanup function, see next step, but that function does not take any argument, I suggest that you use a static global variable to store the address of the cdev struct returned by cdev_alloc()
   2. Do not forget to check the return value of cdev_add() and to pay attenttion to the remarks on cdev_add() in Ch. 3 of the LDD3 book, on pg. 56.
   In the general case, if the device driver controls more than one device, it must register each of them.
2. Modify echo's cleanup function to unregister the device registered in the previous step (the module's initialization function). In the general case, you should undo all actions performed in that step, e.g. free any data structures allocated in that step.
   1. If you wish, you can check the cleanup function in ~so/examples/scullc/main.c
3. Add operation open() to the echo device driver. It needs to do only the following:
1. Initialize the private_data member of struct file to point to struct cdev
2. Print a short message telling what it has just done.
4. Add operation release() to the echo device driver. It needs only print a short message telling that it was invoked.
5. Compile your new version of the echo device driver.
6. Load it, and check the messages it prints
7. Unload it, and check the messages it prints
8. Try to fix any problem that arises in the previous two points.
9. Write your own test program for opening and closing a device driver whose device file name is passed as a program argument. Test it, by:
   1. Loading the driver
   2. Running your program with /dev/echo as an argument, for example.
   3. Unloading the driver
10. Try to fix any problem that arises in the previous two points.

5.3 Writing to and Reading from a device

A device is usually used for data input/output, therefore we would expect a device driver to provide operations supporting data transfer. In the case of the echo device driver we would like it to support:

write()

to echo to the console whatever an application writes to it

read()

to find out how many characters have been written out by all applications to the device since the last time it was loaded

In other words, a write to an echo device will make it to print data on the console, whereas a read will return the number of characters written by the DD on the device since it was last loaded.

In this final section you will add the implementation for these file operations.

5.3.1 The read and the write operations

The prototypes of these operations are as follows:

filep is the file pointer, buff is a buffer used to transfer data either from user space to the kernel or vice-versa, count is the size of the requested transfer, offp is a pointer to a long offset type object that indicates the file position the user is accessing. The offp argument is needed because device drivers are not aware of the file position: it is maintained by the file system layer.

Nevertheless, so that that layer can update the file structure, usually the code should update the file position at *offp accordingly after the data transfer.

In the case of the echo device seeking does not make sense, hence there is no need to keep the file position. Because the default implementation of the operation allows seeking, the open() operation must invoke the function:

This way, the kernel will prevent an lseek() system call from succeeding. In addition, the llseek operation of the struct file_operations should be set to the special helper function no_llseek

Both read and write should return the number of bytes transferred, if the operation is successful. Otherwise, if no byte is successfully transferred, then they should return a negative number. However, if there is an error after successfully transferring some bytes, both should return the number of bytes transferred, and an error code in the following call of the function. This requires the DD to recall the occurrence of an error from a call to the next.

The echo DD needs not worry with partial success because it will always succeed in performing those operations.

5.3.2 Transferring data between user and kernel space

Both in read() and in write(), the buff argument is a pointer to user space, and should not be directly dereferrenced by kernel code. (An explanation of why this is so can be found in Ch. 3 of the LDD3 book, on pp. 63 and 64.) Instead, you can use the following kernel functions, which are defined in <asm/uaccess.h>:

Note that the pages containing the buffer may not be in memory, and the process may be put to sleep while the page is brought in. Therefore functions that invoke them must be reentrant and be able to execute concurrently with other driver functions.

5.3.3 To do

1. Change the open operation, to inform the kernel that the device is non seekable.
2. Add the write file operation. It should read the data from the user space buffer to a kernel space buffer, and then print it. Make sure that the last character in the buffer is code 0. Do not forget to free any buffer you allocate in the kernel.
3. Change the test program from Section 6, to test the write operation. IMP. Do not forget to change the value of the flags argument in the open system call to allow writing.
4. Modify the write operation so that it updates the number of characters it echoes. Add the read file operation to return the total number of characters echoed by the device.
5. Change the test program again, this time to test the read operation.

6. Some Improvements

In the previous sections, we have walked you through the development of a very simple (pseudo)device driver. You may wish to improve it in at least two aspects. First, your device driver rather than supporting a single device, might support several devices, say 2 or 4. Second you may want to embed the struct cdev in a device dependent structure. In the next two sections we will address these two improvements.

6.1. More than one device

Supporting only a device considerably simplifies the device driver. First, you only need to register/unregister one device. Furthermore, since there is only one device, when the DD functions are invoked, the device is implicitly known.

If the device driver supports more than one device you need to register each of the devices. Therefore, you need to allocate and initialize a struct cdev per device, and register each device by invoking cdev_add() with the address of the corresponding struct cdev as an argument. Note that the cdev_add() allows to register only one device at a time, although its third argument is named count. This argument allows a device to have more than one minor number associated with it. E.g., one could be used for control of the device and the other for data exchange.

Likewise, in the cleanup function, you need to unregister each device previously registered struct cdev.

6.2. Embedding the struct cdev

When a device driver supports more than one device, it is convenient to group the information specific to each device, including the struct cdev, in a single data structure. For example, in the case of the echo device driver we could use the following data structure:

In this case, you do not need to invoke cdev_alloc(), to allocate the struct cdev. Instead you need to use the following function:

to initialize the struct cdev. Note that you still need to invoke cdev_add() and cdev_del(), respectively, to register and unregister the device from the kernel.

This approach raises an issue: in open() the struct inode contains the address of the struct cdev, but we want that of the struct echo_dev to initialize the private_data member of struct inode . To address this, you can use the kernel macro container_of defined in <linux/kernel.h>:

which allows to obtain the address of a container structure given the address of one of its members. E.g., in the example above, assuming that inodep has the address of struct inode, we might use the following code snippet:

A more detailed discussion of this issue can be found in Ch. 3 of the LDD3 book, on pg. 58 and following.