Change log:

2017-04-30

Fixed broken links, by pointing to the Wayback Machine

2016-04-30

Added VBox serial port configuration to Section 2.

1. Introduction

In the previous lab you have developed a basic device driver (DD) that relies on polling. The use of polling severely limits the maximum bit-rate that can be used.

The UART is able to generate interrupts on some events, and by proper programming it can be used for interrupt driven I/O, which allows for a more efficient use of the processor.

In this lab, you will use interrupt driven I/O. Because, the control flow, and also the data flow, in an interrupt driven DD is very different from those in a DD that uses polling, you'll develop a separate device driver, whose name should be seri, for interrupt driven I/O.

This does not mean that you need to start a new DD from scratch. A device driver can use interrupts for receiving and polling for transmission, for example. Therefore, I suggest that first you implement and test interrupt driven reception, for example. Implementation of interrupt driven transmission should be done only once interrupt driven reception is working properly. Of course you can reverse the order. In either case, you can still use usocat to help you test your device drivers.

In this handout, we will use interrupt driven reception, whenever we need to be more concrete. In the last subsection, we will address issues specific to interrupt driven transmission.

2. Setup (for the lab PCs)

If you are using one of the lab's PCs, you need to setup the VBox VM executing the following steps (assuming you have completed the first project lab):

Remove the VM configuration

If you have imported the VirtualBox appliance in the last lab class, you have now to delete the corresponding configuration as described in Section 2.3 of the first DD handout (DD1).

$ cd ~/VirtualBox\ VMs/ $ rm -rf CICA2

Import the VirtualBox appliance

As described in Section 2.1 of the first DD handout (DD1).

Configure the shared folders

As described in Section 2.2 of the first DD handout (DD1).

Configure the Serial Port

As described in Section 3.0 of the third DD handout (DD3) (this step may not be required).

You should not need to repeat these steps until you reboot the host OS.

3. The UART and Interrupts

This subsection describes some aspects of the UART relevant for interrupt handling. For more details, you should read its data sheet.

3.1 Configuring the UART

By writing to the IER (Interrupt Enable Register), the UART can be programmed to generate an interrupt on events such as reception of a character, receive overrun, parity error or frame error. More specifically, by setting/clearing specific bits of the IER it is possible to enable/disable interrupts for each of these events. Furthermore, by reading this register it is possible to find out which events may generate an interrupt. Section 8.7 on pg. 19 of the data sheet describes the IER.

3.2 Determining the source of an interrupt

Upon occurrence of an event, the UART generates the corresponding interrupt, if enabled. To find out which event caused the interrupt, the interrupt handler can read the IIR (Interrupt Identification Register) and test its bits. Section 8.6 on pg. 18 of the data sheet describes the IIR.

If more than one of these events is pending, the UART prioritizes them as described in Section 8.6 and in Table IV, also in pg. 18 of the data sheet.

4. Interrupts and the Linux Kernel

4.1 Installing an Interrupt Handler in the Linux Kernel

In Linux, an interrupt handler can be a C function with the following prototype:

#include <linux/interrupt.h>
irqreturn_t short_interrupt(int irq, void *dev_id)

So that the IH is run upon an interrupt generated by the serial port, the seri module must register it invoking the function:

#include <linux/interrupt.h>
int request_irq(unsigned int irq, irqreturn_t (*handler)(int, void *), 
                unsigned long flags, const char *dev_name, void *dev_id)

The second argument is just the address of the interrupt handler being registered, and irq is the interrupt request number (IRQ) line associated with the serial port, and should have the value 4, as you can check in the settings of the VirtualBox. Both this argument and the last one are passed as arguments to the interrupt handler. You can find further details regarding this function in Ch. 10 of the LDD3 book, pp 259 and following.

In spite of the remarks made in those pages, you can install your handler in the module's initialization function. In that case, it should be freed in the module's exit function, by calling the function:

void free_irq(unsigned int irq, void *dev_id);

4.2 Interrupt Handler

As mentioned above, the prototype of the interrupt handler in Linux is:

#include <linux/interrupt.h>
irqreturn_t int_handler(int irq, void *dev_id)

where irq is the IRQ line that generated the interrupt and dev_id is the address that is passed in the last argument of the IH registration function request_irq(). To avoid using global variables, I suggest that you pass the address to the device specific data structure via these arguments.

Although IHs can be written as C functions there are two important restrictions on what they can do. First, they cannot transfer data from/to user level, because they do not execute in the context of any process. Second, they must hold onto the CPU until they are done, because they are not "scheduleable" entities. Therefore, IHs cannot block/sleep, either directly or by calling other functions that may block/sleep, neither call schedule() Nevertheless, IHs can be preempted by other IHs.

In principle, IHs should be as short as possible and the Linux kernel offers a few mechanisms, such as tasklets and workqueues, that can be used by IHs to defer work to a later time. These mechanisms are described in Ch. 7 of the LDD3 book. Workqueues are interesting in that they are associated with kernel threads and therefore are allowed to block/sleep.

The UART is a rather simple device and you need not defer work. The IH should be able to completely process an interrupt. Essentially, the IH has to deal with two issues: transferring data to/from the UART and signal any user process that may be blocked waiting for the event that caused the interrupt.

4.2.1 Data Transfer

Because the IH cannot access user-level data (neither do tasklets or the threads that serve the workqueues), the data must be transferred to kernel buffers. E.g. on an interrupt indicating that a character has been received, the IH should read the received character from the UART and put it into a receive buffer, from where it can be later read by the user process.

Because the user process can concurrently access that buffer, you must use appropriate synchronization mechanisms. The main synchronization mechanisms provided by the Linux kernel are described in Ch. 5 of the LDD3 book. Recall that an IH cannot block, therefore there are some synchronization mechanisms that cannot be used by IHs.

May be better is to use the kfifo data structure whose interface is specified in <linux/kfifo.h> (kernel header files can be found in /usr/src/linux-2.6.19.7/include/ in the guest OS). This is a generic FIFO implemented using a char array that uses a spinlock to prevent race conditions. (You may also wish to take a look at its implementation, which can be found in: /usr/src/linux-2.6.19.7/kernel/kfifo.c, also in the guest OS.)

The size of the buffer used should be large enough to allow for asynchronous operation of the DD and user processes. That is, it should be large enough to buffer enough characters read from the UART by the IH, so that user processes do not have to read from the device in a tight loop.

The IH needs to know the buffer where to put the received chars. One possibility is certainly to use a static global variable. However, this makes it hard to use the same IH for several devices. A better approach is to use the second argument of the interrrupt handler function.

4.2.2 Synchronization with user processes

With blocking I/O, user processes may be blocked waiting for data. E.g., the receiver may have invoked read() when there was no data in the buffer used for communication with the IH. In that case, it should block, unless it was asked explicitly not to do so, and the IH should wake it up later when a character is received.

The Linux kernel provides several synchronization mechanisms that can be used in this case. Perhaps the simplest one is the event queue (called wait queue in Linux.) It is described in Ch. 6 of the LDD3 book, in Subsection "Introduction to Sleep", in pp. 148-151, and in Subsection "Advanced Sleeping", in pp. 155-162.

5. The seri char device driver

The seri char device driver should support the file operations supported by the serp DD: open, close, read and write.

However, the DD file operations for interrupt driven operation need to be different from those for polled operation, especially those that concern data transfer. Before describing the issues related to data transfer in interrupt-driven operation, we'd like to call your attention to the design of data structures that we suggest you use in the seri DD.

5.1 Embedding struct cdev in another data structure

Whereas in the serp DD almost all the information needed to implement the file operations is in the struct cdev, already introduced Lab 2, in the case of an interrupt-driven DD you need buffers for communication between the IH and the user threads and wait queues for these threads to wait for data availability. Furthermore, you will need at least one semaphore to ensure that if multiple threads attempt to execute concurrent system calls to access the same device no race conditions occur. (Actually, the use of a semaphore is also needed in the case of polled operation.) It is therefore convenient to group all these variables, including the struct cdev, in a single data structure, such as seri_dev_t outlined in pg. 29 of the transparencies presented in the lab on concurrency in the Linux kernel.

Embedding the struct cdev in a device-specific data structure requires some changes to:

1. the code that allocates and initializes the struct cdev in the DD's initialization function
2. the open file operation

We will detail each of these changes next.

As described in Lab 2, when we use a standalone struct cdev, it should be allocated using the cdev_alloc() kernel function, which partially initializes it. However, when the struct cdev is embedded in another data structure, it will be allocated when that data structure is allocated. Therefore, you will have to initialize the struct cdev using the function cdev_init:

After initializing the struct cdev, you must register it as before, by invoking cdev_add().

When the struct cdev is embedded in a device-specific data structure, handling the open system call is a bit more tricky. Remember that the prototype of the open operation is:

and the struct inode has a member, i_cdev, that points to the corresponding struct cdev, not to the structure that contains it, which is what the DD needs. To solve this problem you can use the macro container_of defined in <linux/kernel.h>:

which allows to obtain the adddress of a structure that embeds another from the address of the latter. In principle, to simplify the implementation of other file operations, the code in open() should initialize the private_data member of the struct file with the address of the embedding structure rather than the address of the struct cdev.

A more detailed discussion of this issue, including some code, can be found in Ch. 3 of the LDD3 book, on pg. 58 and following.

5.2 read

The implementation of this function is closely related to that of the IH. Essentially, it needs to read data from the kernel buffer where the IH puts the characters received from the UART and copy it to the user-space buffer. Even if the device is used by at most one thread, access to such a buffer must be synchronized to prevent race conditions between the user thread and the IH.

If the kernel buffer is not empty, the user process may just copy the data from the kernel level to user level, and return immediately.

With blocking I/O, the default operating mode, if the kernel buffer is empty, the user process must block/sleep until data is received. To give the user control over the user process, it should use an "interruptible" block/sleep primitive. Later, when the user process is awoken by the IH, after receiving a character from the UART, it can copy that character to the user-space buffer and return to user level immediately.

Copying of data from kernel to user level can be done with the copy_to_user() function, already introduced in Lab 2. There are however some other functions described in Ch. 6 of the LDD3 book in pp. 142 and 143.

5.3 write

With exception of two small but important details, the implementation issues of this DD file operation are similar to those of the read.

First, it is OK to return immediately, if the kernel buffer is full, even if using blocking I/O. Of course, it must return the appropriate value. An alternative is to return only after all data is copied to the kernel buffer, blocking if necessary. This is likely to be what most application programmers expect, as they seldom check the return value of write for other than a failure. (And, if your system call should not return failure, just because there is no user space, unless it is asked not to block and it is not able to write a single char.)

The second issue is more tricky and is related to initiating data transmission. The UART generates an interrupt when the Transmitter Holding Register becomes empty, but this interrupt condition is cleared as soon as the IIR is read, which usually occurs when the IH runs. If there are no further characters to send, the UART will not generate further interrupts. Therefore, the first character after an "idle" period of UART transmitter must be sent by DD code other than the IH. A particular case of this is the first character in a stream.

What makes handling this tricky is that it is difficult to find whether the transmitter is idle. For example, just because the kernel buffer used for communication with the IH is empty, that does not mean that the transmitter is idle. It may be the case that it is still sending the character that was put last in that buffer. The fact that the IH, the UART and the user process all execute concurrently makes it hard to implement an efficient solution. So I suggest that you first try to get it right, and consider efficiency only later.

Acknowledgments

José Manuel Cruz provided very useful feedback regarding this handout. In particular, he suggested that a previous handout, covering not only interrupt-driven I/O, but also other aspects, be broken in two: the first on interrupt driven I/O, and the second on the remaining aspects.