% CVSId: $Id: intro_linux_device_drivers.tex,v 1.9 2005/05/28 20:17:08 muli Exp $

\documentclass[final, total, pdf, colorBG, slideColor]{prosper}

\usepackage{listings}

\title{Introduction to Linux Device Drivers} 
\subtitle{Recreating Life One Driver At a Time}
\author{Muli Ben-Yehuda}
\email{mulix@mulix.org, muli@il.ibm.com}
\institution{IBM Haifa Research Lab and Haifux - Haifa Linux Club}
\slideCaption{Linux Device Drivers, BIUX, April 2006}

\begin{document}

\lstset{
	language=C,
%	showspaces=no,	
%	showstringspaces=no,
	basicstyle=\tiny,
	keywordstyle=\tiny
}

\maketitle

\begin{slide}{Why Write Linux Device Drivers?} % 1

\begin{itemize}

\item For fun, 
\item For profit (Linux is {\red hot} right now, especially embedded Linux),
\item To scratch an itch.
\item Because you can!

\end{itemize} 

OK, but why {\red Linux} drivers? 

\begin{itemize} 

\item Because the source is available. 
\item Because of the community's cooperation and involvement.
\item Have I mentioned it's fun yet? 

\end{itemize} 

\end{slide}

\begin{slide}{klife - Linux kernel game of life} % 2 

klife is a Linux kernel Game of Life implementation. It is a software
device driver, developed specifically for this talk. 

\begin{itemize} 

\item The game of life is played on a square grid, where some of the
cells are alive and the rest are dead.
\item Each generation, based on each cell's neighbors, we mark the
cell as alive or dead.
\item With time, amazing patterns develop. 
\item The only reason to implement the game of life inside the kernel
is for demonstration purposes.

\end{itemize} 

Software device drivers are very common on Unix systems and provide
many services to the user. Think about /dev/null, /dev/zero,
/dev/random, /dev/kmem...

\end{slide} 

\begin{slide}{Anatomy of a Device Driver} % 2 

\begin{itemize} 

\item A device driver has three sides: one side talks to the rest of the
kernel, one talks to the hardware, and one talks to the user: 

\begin{figure}[ht]
  \centering
  \includegraphics[width=6cm]{device_file.eps}
%  \caption{Interfaces of a Device driver}
\end{figure}

\end{itemize} 

\end{slide} 


\begin{slide}{Kernel Interface of a Device Driver} % 2 

\begin{itemize} 

\item In order to talk to the kernel, the driver registers with subsystems to
respond to events. Such an event might be the opening of a file, a
page fault, the plugging in of a new USB device, etc.

\begin{figure}[ht]
  \centering
  \includegraphics[width=9cm]{event_list.eps}
%  \caption{Event List}
\end{figure}
\end{itemize} 

\end{slide} 


\begin{slide}{User Interface of a Device driver} % 2 

\begin{itemize} 
\item Since Linux follows the UNIX model, 
and in UNIX everything is a file, users talk
with device drivers through {\blue device files}.
\item Device files are a mechanism, supplied by the kernel, precisely
for this direct User-Driver interface.
\item klife is a {\blue character device}, and thus the user talks to
it through a {\blue character device file}. 
\item The other common kind of device file is a {\blue block device
file}. We will only discuss character device files today.

\end{itemize} 

\end{slide} 

\begin{slide}{Anatomy of klife device driver} % 2 

\begin{itemize} 

\item The user talks with klife through the {\blue /dev/klife} device
file. 
\begin{itemize}
\item When the user {\blue opens} /dev/klife, the kernel calls klife's
{\blue open routine}
\item When the user {\blue closes} /dev/klife, the kernel calls klife's
{\blue release routine} 
\item When the user {\blue reads or writes} from or to /dev/klife -
you get the idea\dots 
\end{itemize}
\item klife talks to the kernel through 
\begin{itemize}
\item its {\blue initialization
function} 
\item \dots and through {\blue register\_chrdev} 
\item \dots and through {\blue hooking into the timer interrupt} 
\end{itemize}
\item We will elaborate on all of these later 

\end{itemize} 

\end{slide} 

\begin{slide}{Driver Initialization Code} % 3

\begin{lstlisting}{}
static int __init klife_module_init(void)
{
        int ret;

        pr_debug("klife module init called\n");

        if ((ret = register_chrdev(KLIFE_MAJOR_NUM, "klife", &klife_fops)) < 0)
                printk(KERN_ERR "register_chrdev: %d\n", ret);

        return ret;
}
\end{lstlisting}

\end{slide} 

\begin{slide}{Driver Initialization} % 4
\begin{itemize}

\item One function ({\blue init}) is called on the driver's initialization.
\item One function ({\blue exit}) is called when the driver is removed from the
system.
\item Question: what happens if the driver is compiled into the
kernel, rather than as a module? 
\item The init function will register hooks that will get the driver's
code called when the appropriate event happens.
\item Question: what if the init function doesn't register any hooks? 
\item There are various hooks that can be registered: file operations,
pci operations, USB operations, network operations - it all depends on
what kind of device this is. 
\end{itemize} 

\end{slide}

\begin{slide}{Registering Chardev Hooks} % 5

\begin{lstlisting}{} 
struct file_operations klife_fops = {
        .owner = THIS_MODULE, 
        .open = klife_open, 
        .release = klife_release, 
        .read = klife_read, 
        .write = klife_write, 
        .mmap = klife_mmap, 
        .ioctl = klife_ioctl
};
...
if ((ret = register_chrdev(KLIFE_MAJOR_NUM, "klife", &klife_fops)) < 0)
        printk(KERN_ERR "register_chrdev: %d\n", ret); 
\end{lstlisting}

\end{slide} 

\begin{slide}{User Space Access to the Driver} % 6

We saw that the driver registers a character device tied to a given
{\red major number}, but how does the user create such a file?

\medskip
\begin{tt}\begin{verbatim}
# mknod /dev/klife c 250 0
\end{verbatim}\end{tt}
\medskip

And how does the user open it? 
\medskip
\begin{tt}\begin{verbatim}
if ((kfd = open("/dev/klife", O_RDWR)) < 0) { 
        perror("open /dev/klife"); 
        exit(EXIT_FAILURE); 
} 
\end{verbatim}\end{tt}
\medskip
And then what? 
\end{slide} 

\begin{slide}{File Operations} % 7

\dots and then you start talking to the device. 
klife uses the following device file operations: 
\begin{itemize}
\item {\blue open} for starting a game (allocating resources).
\item {\blue release} for finishing a game (releasing resources).
\item {\blue write} for initializing the game (setting the starting
positions on the grid).
\item {\blue read} for generating and then reading the next state of
the game's grid.
\item {\blue ioctl} for querying the current generation number, and
for enabling or disabling hooking into the timer interrupt (more on
this later). 
\item {\blue mmap} for potentially faster but more complex direct
access to the game's grid.
\end{itemize} 

\end{slide} 


\begin{slide}{The open and release Routines}
open and release are where you perform any setup not done in
initialization time and any cleanup not done in module unload time.
\end{slide}


\begin{slide}{klife\_open} %

klife's open routine allocates the klife structure which holds
all of the state for this game (the grid, starting positions, current
generation, etc).

\begin{lstlisting}{}
static int klife_open(struct inode *inode, struct file *filp)
{
        struct klife* k;
        int ret;

        ret = alloc_klife(&k);
        if (ret)
                return ret;

        filp->private_data = k;

        return 0;
}
\end{lstlisting}

\end{slide}

\begin{slide}{klife\_open - alloc\_klife} %

\begin{lstlisting}{}
static int alloc_klife(struct klife** pk)
{
        int ret;
        struct klife* k;

        k = kmalloc(sizeof(*k), GFP_KERNEL);
        if (!k)
                return -ENOMEM;

        ret = init_klife(k);
        if (ret) {
                kfree(k);
                k = NULL;
        }

        *pk = k;
        return ret;
}
\end{lstlisting}

\end{slide}

\begin{slide}{klife\_open - init\_klife} %
\begin{lstlisting}{}
static int init_klife(struct klife* k)
{
        int ret;

        memset(k, 0, sizeof(*k));

        spin_lock_init(&k->lock);

        ret = -ENOMEM;
        /* one page to be exported to userspace */
        k->grid = (void*)get_zeroed_page(GFP_KERNEL);
        if (!k->grid)
                goto done;

        k->tmpgrid = kmalloc(sizeof(*k->tmpgrid), GFP_KERNEL);
        if (!k->tmpgrid)
                goto free_grid;
\end{lstlisting}
\end{slide}

\begin{slide}{klife\_open - init\_klife cont'} %
\begin{lstlisting}{}
        k->timer_hook.func = klife_timer_irq_handler;
        k->timer_hook.data = k;
        return 0;

free_grid:
        free_page((unsigned long)k->grid);
done:
        return ret;
}
\end{lstlisting}
\end{slide}

\begin{slide}{klife\_release} 

klife's release routine frees the resource allocated during open
time.
\begin{lstlisting}{}
static int klife_release(struct inode *inode, struct file *filp)
{
        struct klife* k = filp->private_data;
        if (k->timer)
                klife_timer_unregister(k);
        if (k->mapped) {
                /* undo setting the grid page to be reserved */
                ClearPageReserved(virt_to_page(k->grid));
        }
        free_klife(k);
        return 0;
}
\end{lstlisting}
\end{slide} 

\begin{slide}{Commentary on open and release} 

\begin{itemize} 
\item Beware of races if you have any global data \dots many a driver
author stumble on this point.
\item Note also that release can fail, but almost no one checks errors
from close(), so it's better if it doesn't \dots
\item Question: what happens if the userspace program crashes while
holding your device file open? 
\end{itemize} 

\end{slide}


\begin{slide}{write} 
\begin{itemize}
\item For klife, I ``hijacked'' write to mean ``please initialize the
grid to these starting positions''.
\item  There are no hard and fast rules to
what write has to mean, but it's good to KISS (Keep It Simple,
Silly...)
\end{itemize} 

\end{slide}


\begin{slide}{klife\_write  - 1} %
\begin{lstlisting}{}
static ssize_t klife_write(struct file* filp, const char __user * ubuf, 
                           size_t count, loff_t *f_pos)
{
        size_t sz; 
        char* kbuf; 
        struct klife* k = filp->private_data; 
        ssize_t ret; 

        sz = count > PAGE_SIZE ? PAGE_SIZE : count; 

        kbuf = kmalloc(sz, GFP_KERNEL); 
        if (!kbuf)
                return -ENOMEM; 

\end{lstlisting}
Not trusting users: checking the size of the user's buffer
\end{slide} 

\begin{slide}{klife\_write - 2} %
\begin{lstlisting}{}
        ret = -EFAULT; 
        if (copy_from_user(kbuf, ubuf, sz))
                goto free_buf; 

        ret = klife_add_position(k, kbuf, sz); 
        if (ret == 0)
                ret = sz; 

free_buf: 
        kfree(kbuf); 
        return ret; 
} 
\end{lstlisting}
Use copy\_from\_user in case the user is passing a bad pointer.
\end{slide} 



\begin{slide}{Commentary on write} 
\begin{itemize}
\item Note that even for such a simple function, care must be
exercised when dealing with {\red untrusted} users.
\item  {\red Users are
always untrusted}.
\item {\red Always} be prepared to handle errors!
\end{itemize} 

\end{slide}

\begin{slide}{read}

\begin{itemize} 
\item For klife, read means ``please calculate and give me the next
generation''.
\item The bulk of the work is done in two other routines:
\begin {itemize}
\item klife\_next\_generation calculates the next generation based
on the current one, according to the rules of the game of life.
\item klife\_draw takes a grid and ``draws'' it as a single string in a page
of memory.
\end{itemize}
\end{itemize} 

\end{slide} 

\begin{slide}{klife\_read - 1} %
\begin{lstlisting}{}
static ssize_t 
klife_read(struct file *filp, char *ubuf, size_t count, loff_t *f_pos)
{
        struct klife* klife; 
        char* page; 
        ssize_t len; 
        ssize_t ret; 
        unsigned long flags; 

        klife = filp->private_data; 

        /* special handling for mmap */ 
        if (klife->mapped)
                return klife_read_mapped(filp, ubuf, count, f_pos); 
        
        if (!(page = kmalloc(PAGE_SIZE, GFP_KERNEL)))
                return -ENOMEM; 
\end{lstlisting}
\end{slide} 

\begin{slide}{klife\_read - 2} %
\begin{lstlisting}{}
    spin_lock_irqsave(&klife->lock, flags); 
    klife_next_generation(klife); 
    len = klife_draw(klife, page); 
    spin_unlock_irqrestore(&klife->lock, flags); 
    if (len < 0) { 
            ret = len; 
            goto free_page; 
    } 
    /* len can't be negative */
    len = min(count, (size_t)len);  
\end{lstlisting}
Note that the lock is held for the shortest possible time.

We will see later what the lock protects us against.
\end{slide} 

\begin{slide}{klife\_read - 3} %
\begin{lstlisting}{}
        if (copy_to_user(ubuf, page, len)) { 
                ret = -EFAULT; 
                goto free_page; 
        }

        *f_pos += len; 
        ret = len; 

free_page:
        kfree(page); 
        return ret; 
}
\end{lstlisting}
copy\_to\_user in case the user is passing us a bad page.
\end{slide} 

\begin{slide}{klife\_read - 4} %
\begin{lstlisting}{}
static ssize_t 
klife_read_mapped(struct file *filp, char *ubuf, size_t count, 
        loff_t *f_pos)
{ 
        struct klife* klife; 
        unsigned long flags; 

        klife = filp->private_data; 

        spin_lock_irqsave(&klife->lock, flags); 

        klife_next_generation(klife); 
        
        spin_unlock_irqrestore(&klife->lock, flags); 

        return 0; 
} 
\end{lstlisting}
Again, mind the short lock holding time.
\end{slide} 

\begin{slide}{Commentary on read}

\begin{itemize}
\item There's plenty of room for optimization in this code \dots can
you see where? 
\end{itemize} 

\end{slide} 

\begin{slide}{ioctl} 
\begin{itemize}
\item ioctl is a ``special access'' mechanism, for operations that do
not cleanly map anywhere else. 
\item It is considered extremely bad taste to use ioctls in Linux
where not absolutely necessary.
\item New drivers should use either sysfs (a /proc -like virtual file
system) or a driver specific file system (you can write a Linux file
system in less than a 100 lines of code).
\item In klife, we use ioctl to get the current generation number, for
demonstration purposes only \dots 
\end{itemize}
\end{slide}

\begin{slide}{klife\_ioctl - 1} %
\begin{lstlisting}{}
static int klife_ioctl(struct inode* inode, struct file* file, 
        unsigned int cmd,  unsigned long data)
{
        struct klife* klife = file->private_data; 
        unsigned long gen; 
        int enable; 
        int ret; 
        unsigned long flags; 
        ret = 0; 
        switch (cmd) { 
        case KLIFE_GET_GENERATION:
                spin_lock_irqsave(&klife->lock, flags); 
                gen = klife->gen; 
                spin_unlock_irqrestore(&klife->lock, flags); 
                if (copy_to_user((void*)data, &gen, sizeof(gen))) { 
                        ret = -EFAULT; 
                        goto done; 
                }
\end{lstlisting}
\end{slide} 

\begin{slide}{klife\_ioctl - 2} %
\begin{lstlisting}{}
                break; 
        case KLIFE_SET_TIMER_MODE: 
                if (copy_from_user(&enable, (void*)data, sizeof(enable))) { 
                        ret = -EFAULT; 
                        goto done; 
                }
                pr_debug("user request to %s timer mode\n", 
                         enable ? "enable" : "disable"); 
                if (klife->timer && !enable) 
                        klife_timer_unregister(klife); 
                else if (!klife->timer && enable) 
                        klife_timer_register(klife); 
                break; 
        }
done: 
        return ret;     
}
\end{lstlisting}
\end{slide} 

\begin{slide}{memory mapping}
\begin{itemize} 
\item The read-write mechanism, previously described, involves an
      overhead of a system call and related context switching and of
      memory copying.
\item mmap maps pages of a file into memory, thus enabling programs to
      directly access the memory directly and save the overhead, \dots
      but: 
  \begin{itemize}
  \item {\blue fast} synchronization between kernel space and
  user space is a pain (why do we need it?),
  \item and Linux read and write are really quite fast. 
  \end{itemize}
\item mmap is implemented in klife for demonstration purposes, with
      read() calls used for synchronization and triggering a
      generation update.
\end{itemize} 
\end{slide} 

\begin{slide}{klife\_mmap} %
%where are the said read calls? 
\begin{lstlisting}{}
	...
        SetPageReserved(virt_to_page(klife->grid));
        ret = remap_pfn_range(vma, vma->vm_start,
                              virt_to_phys(klife->grid) >> PAGE_SHIFT,
                              PAGE_SIZE, vma->vm_page_prot);

        pr_debug("io_remap_page_range returned %d\n", ret);

        if (ret == 0)
                klife->mapped = 1;

        return ret;
}
\end{lstlisting}
\end{slide} 

\begin{slide}{klife Interrupt Handler}
\begin{itemize}
\item What if we want a new generation on every raised interrupt?
\item Since we don't have a hardware device to raise interrupts for
us,  let's hook
into the one hardware every PC has - the clock - and steal its
interrupt! 
\end{itemize}
\end{slide}

\begin{slide}{Usual Request For an Interrupt Handler}
Usually, interrupts are requested using request\_irq(): 
\medskip
\begin{lstlisting}{}
/* claim our irq */
rc = -ENODEV;
if (request_irq(card->irq, &trident_interrupt, 
		SA_SHIRQ, card_names[pci_id->driver_data], 
		card)) {
        printk(KERN_ERR 
        "trident: unable to allocate irq %d\n", card->irq);
        goto out_proc_fs;
}
\end{lstlisting}
\end{slide} 

\begin{slide}{klife Interrupt Handler}
\begin{itemize}
\item It is impossible to request the timer interrupt.
\item  Instead, we will
directly modify the kernel code to call our interrupt handler, 
if it's
registered.
\item We can do this, because the code is open\dots
\end{itemize}
\end{slide} 


\begin{slide}{Aren't Timers Good Enough For You?} 

\begin{itemize} 
\item ``Does every driver which wishes to get periodic notifications
need to hook the timer interrupt?''  - {\red Nope}. 
\item Linux provides an excellent timer mechanism which can be used
for periodic notifications. 
\item The reason for hooking into the timer interrupt in klife is
because we wish to be called from {\blue hard interrupt context}, also
known as {\blue top half context} \dots 
\item \dots whereas timer functions are called in softirq 
{\blue bottom half context}. 
\item Why insist on getting called from hard interrupt context? 
\begin{itemize}
\item So we
can demonstrate {\blue deferring work}.
\end{itemize}
\end{itemize} 

\end{slide} 


\begin{slide}{The Timer Interrupt Hook Patch} 

\begin{itemize}
\item The patch adds a hook which a driver can register for, to be called
directly from the timer interrupt handler. It also creates two
functions:
\begin{itemize}
\item register\_timer\_interrupt
\item unregister\_timer\_interrupt
\end{itemize}
\end{itemize}

\end{slide}



\begin{slide}{Hook Into The Timer Interrupt Routine 1} 
'+' marks the lines added to the kernel.
\begin{lstlisting}{}
+struct timer_interrupt_hook* timer_interrupt_hook; 
+
+static void call_timer_hook(struct pt_regs *regs)
+{
+	struct timer_interrupt_hook* hook = timer_interrupt_hook; 
+
+	if (hook && hook->func) 
+		hook->func(hook->data); 
+}
@@ -851,6 +862,8 @@ void do_timer(struct pt_regs *regs)
        update_process_times(user_mode(regs));
 #endif
        update_times();
+
+       call_timer_hook(regs); 
 }
\end{lstlisting}
\end{slide} 

\begin{slide}{Hook Into The Timer Interrupt Routine 2} 
\begin{lstlisting}{}
+int register_timer_interrupt(struct timer_interrupt_hook* hook)
+{
+       printk(KERN_INFO "registering a timer interrupt hook %p "
+              "(func %p, data %p)\n", hook, hook->func, 
+              hook->data); 
+
+       xchg(&timer_hook, hook); 
+       return 0; 
+}
+
+void unregister_timer_interrupt(struct timer_interrupt_hook* hook)
+{
+       printk(KERN_INFO "unregistering a timer interrupt hook\n"); 
+
+       xchg(&timer_hook, NULL); 
+}
\end{lstlisting}

\end{slide} 

\begin{slide}{Commentary - The Timer Interrupt Hook} 
\begin{itemize}
\item Note that the register and unregister calls use xchg(), to
ensure atomic replacement of the pointer to the handler. Why use
xchg() rather than a lock? 
\item What context (hard interrupt, bottom half,
process context) will we be called in? 
\item Which CPU's timer interrupts
would we be called in? 
\item What happens on an SMP system? 
\end{itemize} 
\end{slide}

\begin{slide}{Deferring Work} 
\begin{itemize} 
\item You were supposed to learn in class about bottom halves,
softirqs, tasklets and other such curse words.
\item The timer interrupt (and every other interrupt) has to happen
very quickly. Why?
\item The interrupt handler (top half, hard irq) usually just sets
a flag which says ``there is work to be done''.
\item The work is then deferred to a bottom half context, where it is
done by an (old style) bottom half, softirq, or tasklet.
\item For klife, we defer the work we wish to do (updating the grid)
to a bottom half context by scheduling a {\blue tasklet}.
\end{itemize} 
\end{slide} 

\begin{slide}{Preparing The Tasklet} 

\begin{lstlisting}{}
DECLARE_TASKLET_DISABLED(klife_tasklet, klife_tasklet_func, 0); 

static void klife_timer_register(struct klife* klife)
{
        unsigned long flags; 
        int ret; 
        spin_lock_irqsave(&klife->lock, flags); 
        /* prime the tasklet with the correct data - ours */ 
        tasklet_init(&klife_tasklet, klife_tasklet_func, 
                (unsigned long)klife); 
        ret = register_timer_interrupt(&klife->timer_hook); 
        if (!ret)
                klife->timer = 1; 
        spin_unlock_irqrestore(&klife->lock, flags); 
        pr_debug("register_timer_interrupt returned %d\n", ret); 
}
\end{lstlisting}

\end{slide} 


\begin{slide}{The klife Tasklet} 
Here's what our klife tasklet does:
\begin{itemize}
\item First, it derives the klife structure from the parameter it gets.
\item Then, it locks it, to prevent concurrent access on another CPU.
What are we protecting against?
\item Then, it generates the new generation.
\begin{itemize}
\item What must we never do here?
\item Hint: can tasklets block?
\end{itemize}
\item Last, it releases the lock.
\end{itemize}

\end{slide} 


\begin{slide}{Deferring Work - The klife Tasklet} 
\begin{lstlisting}{}
static void klife_timer_irq_handler(void* data)
{
        struct klife* klife = data; 

        /* 2 times a second */ 
        if (klife->timer_invocation++ % (HZ / 2) == 0)
                tasklet_schedule(&klife_tasklet); 
}

static void klife_tasklet_func(unsigned long data)
{
        struct klife* klife = (void*)data; 
        spin_lock(&klife->lock); 
        klife_next_generation(klife); 
        spin_unlock(&klife->lock); 
}
\end{lstlisting}

\end{slide} 

\begin{slide}{Adding klife To The Build System}
Building the module in kernel 2.6 is a breeze. All that's required to
add klife to the kernel's build system are these tiny patches: 

\begin{itemize}
\item In drivers/char/Kconfig:
\begin{lstlisting}{}
+config GAME_OF_LIFE
+	tristate "kernel game of life"
+	help
+	  Kernel implementation of the Game of Life. 
\end{lstlisting}
\item in drivers/char/Makefile
\begin{lstlisting}{}
+obj-$(CONFIG_GAME_OF_LIFE) += klife.o
\end{lstlisting}
 
\end{itemize} 

\end{slide} 

\begin{slide}{Summary} 

\begin{itemize} 
\item Writing Linux drivers is easy \ldots
\item \dots and fun!
\item Most drivers do fairly simple things, which Linux provides APIs
for. 
\item The real fun is when dealing with the hardware's quirks. 
\item It gets easier with practice \ldots
\item \dots but it never gets boring.
\end{itemize} 

\begin{huge}{\blue Questions?}\end{huge}

\end{slide} 
\begin{slide}{Where To Get Help} % 

\begin{itemize} 

\item {\blue google}  
\item Community resources: {\blue web sites} and {\blue mailing lists}.
\item Distributed {\blue documentation} (books, articles, magazines)
\item Use {\red The Source}, Luke!
\item Your {\red fellow kernel hackers}.
\end{itemize}

\end{slide} 

\begin{slide}{Bibliography} % 

\begin{itemize} 
\item kernelnewbies - http://www.kernelnewbies.org
\item linux-kernel mailing list archives - 
\begin{lstlisting}{}
http://marc.theaimsgroup.com/?l=linux-kernel&w=2
\end{lstlisting}
\item Understanding the Linux Kernel, by Bovet and Cesati
\item Linux Device Drivers, 3rd edition, by Rubini et. al. 
\item Linux Kernel Development, 2nd edition, by Robert Love 
\item /usr/src/linux-xxx/
\end{itemize}

\end{slide} 

\end{document}