What happens when kernel and userland don't talk?
Why are Linux kernel-userland Application Programming Interfaces (system calls, /proc files, and so on) sometimes not as well designed as they should be? And why do they so often still have bugs upon release in the stable kernel? In this presentation I will look at some unfortunate examples of both problems, and suggest a few ways in which both kernel developers and userland programmers can reduce these problems..
In part, these problems originate from the disconnect between kernel developers, the creators of APIs, and userland programmers, the users of those APIs. On the one side, many kernel developers spend (or, have spent) little time writing userland programs, and often have only limited knowledge and experience of API design and of testing methodologies. Conversely, many userland programmers find the code of the Linux kernel difficult to read and understand. Add to this the lack of forums in which kernel developers and userland programmers can interact, and the significant delay between the implementation of a new API and its real-world use. The result is that kernel-userland APIs typically see far too little design review and testing before their (final) implementation.
The solution to these problems should be the traditional open-source one: involve more people in the design and testing phases, and involve them earlier. But how? One part of the solution is to improve the structures and processes employed in API design and testing. Another important part is documentation -- specifically the API documentation maintained within the Linux man-pages project. A manual page is not only a description of a finished implementation: during API development, it can act as the bridge between kernel developers and userland developers, by providing a specification for design review and implementation testing. Used in this manner, documentation benefits both kernel developers (who get more review and testing of their work) and userland developers (who don't end up living with the consequences of bad design decisions by kernel developers). Using concrete examples taken from recently developed APIs, I will show how writing man pages has helped to improve the design of kernel-userland interfaces, and to improve the testing process. The aim is to demonstrate the short- and long-term benefits of early documentation and testing for both kernel developers and userland programmers.
Michael Kerrisk first started programming on Unix systems in 1987. He has been involved with the Linux man-pages project, which documents the Linux kernel-userspace and the glibc APIs, since 2000, and has been the project maintainer since 2004. He is the sole author of over 130, and the co-author of another 120, of the around 900 pages in the man-pages package (and he has made changes to nearly all of the other pages). (See http://www.kernel.org/doc/man-pages/ and http://linux-man-pages.blogspot.com/.) In May 2008, he commenced a fellowship with the Linux Foundation to work full time on man-pages and related work. As well as maintaining kernel-userspace API documentation, he is also involved in testing and design review of new APIs, and has found many bugs in the APIs. He is currently nearing completion of a book providing a detailed description of the Linux kernel-userspace API (including the Linux IPC facilities).