---------------------------------------
Linux Kernel Project Management System
by Jordan Mendelson (jordy@wserv.com)
Initial Revision 19990927
Second Revision 19990927
About this Document
This is a proposal for a source management system and an overview of the
features required of such a system. It is not the purpose of this document to
advocate one solution over another, but instead provide requirements for a
complete system and suggestions for software that would meet these
requirements.
If you have any questions or comments, please email me.
Current Situation
The Linux kernel is a large open source project consisting of several hundred
developers and over 60 megabytes of source code. The source code is managed in
pieces with section managers heading up individual sections of the kernel
source code. Linus Torvalds has the final word on integrating changes from the
section managers into two branches of the kernel tree; the unstable branch and
the stable branch.
In addition there are a few additional people such as Alan Cox, Dave Miller,
and Andrea Arcangeli that submit multiple bundled patches to Linus directly.
Some of these patches come from maintainers while others come directly from
individual developers.
Currently there appear to be over 100 maintainers listed as managing these
individual sections and several hundred more submitting patches to these
maintainers to be integrated into the kernel tree. Some sections are as small
as an individual network card driver while others are as large as the
networking subsystem.
Bug reports and tracking is currently done by hand. Kernel crashes/deadlocks,
driver problems, and other kernel related issues are supposed to be submitted
to the maintainer, but more often than not they are sent to the Linux-kernel
mailing list or to a support service at one of the large commercial Linux
distributions such as Redhat. These reports are all too often missing critical
system information required for diagnosing the problem. Because kernel-related
bug reports are submitted to different people, it is difficult to track
similarities in the bug reports or provide any type of feedback to the
original bug poster about the status of their bug report.
Bug or feature patches are also managed by hand. Patches are submitted to the
section maintainer for review and are then forwarded to Linus Torvalds for
integration into the official tree. Unless these patches are posted to a
mailing list, no public review is done until the time an official patch for
the kernel source tree is released. Often, patches will be submitted to the
Linux-kernel mailing list for peer review and comment before being submitted
to the section maintainer or directly to Linus.
The official current kernel source tree is held privately by Linus Torvalds.
All patches made to the kernel source tree are made to the last release
version of a particular branch; stable or unstable. There are several CVS
repositories of the Linux-kernel tree, however they tend to differ from the
official versions. Because Linus does not have the resources to verify every
release will compile correctly on every type of machine, there have been
several instances where release kernels would not compile correctly.
There is currently no unified documentation of the Linux kernel. Information
about subsystems is available in the Documentation/ section, but information
relating to the internal API and in-depth architecture of the kernel
subsystems is largely missing.
Requirements for a Solution
If a solution to kernel source management is to be found, it must incorporate
all the same features that the current system allows. There are four distinct
pieces to the kernel development process:
§ Source Repository
§ Bug Reporting & Tracking System
§ Patch Submission & Peer Review System
§ API Documentation Submission
Each of these pieces should be tightly integrated with each other and provide
a consistent interface between them.
Source Repository Requirements
The source repository must provide a basic set of permissions. Everyone should
have the ability to read the current source repository. Only registered users
should have the ability to submit a patch for inclusion into the official
tree. Linus Torvalds and anyone he feels can handle the responsibility (Alan
Cox) should be the only ones which can accept a patch for inclusion.
The source repository should also provide a method for distributed
repositories whereby multiple repositories contain the current revision of the
kernel source. This would allow load distribution between multiple machines
similar to the way the kernel.org FTP/HTTP distribution system works.
The source repository must support the idea of a change set. A change set is a
group of changes integrated into a single patch. The ability to add or remove
these patches as a whole or partially any time during the development process
is a necessary feature.
The source repository should also allow both disconnected operation and fully
connected operation using push & pull methods of synchronization. Pull
synchronization benefits users that are not online all the time and is
adequate for most developers. Push synchronization benefits users that must
maintain a current copy of the kernel source at all times.
The source repository or associated tools should have an automated process for
testing submitted patches. These tests should ensure that the patch works
correctly against the current source tree and will build correctly as well.
Bug Reporting & Tracking System Requirements
The bug reporting system should use the same username & password system as the
source repository allowing a single unified login.
The bug reporting system should allow both web-based and email-based
submission. It is not necessary for a specific console-based submission as one
could just send email or post it to a web server.
The bug reporting system must distinguish between the various pieces of the
Linux kernel providing an easy method for section maintainers to look over
bugs that are pertinent to them. Email notices of changes to bug reports
should be sent via email to the section maintainer or section mailing list if
requested.
The bug reporting system should require certain information about the system
be submitted with the bug report. This information includes system hardware
configuration, Linux distribution, kernel version, decoded oops, etc.
A summary should be available to view any trends in regard to bug reports. If
there have been 50 bug reports in the last 2 weeks from people with VIA
chipsets on their motherboards, this fact should be noted somewhere.
A comment system that would allow comments to be added to bug reports should
be available. Email notices when a comment is posted should be sent to the
section maintainer and the original bug poster if requested. The comment
system should also allow attachments for binary files and hyperlinks to any
patches.
The bug reports should all have various states, open, closed, assigned, etc.
The Patch Submission & Peer Review System
The patch submission & peer review system should use the same username &
password system as the source repository and bug reporting system.
The patch submission & peer review system should accept patches through a web
based interface and through an email based interface. However, the web-based
interface should be the preferred method.
Developers should be able to submit new patches for peer review. These patches
must be tested to make sure they work correctly against the current tree. If
at any time these patches fail to work against the current tree and it has not
been accepted into the official source tree, an email should be generated to
the developer.
Developers should also be able to update & remove existing patches.
A comment system similar to the bug reporting comment system should be in
place. This will allow peer review of the patch in question as well as a place
to show the history of the patch in question.
The peer review system should color-code the patches for display for easy
viewing.
On submission of a patch, you should also have to submit a bit of information
about the patch including what it is for, what changes it makes, what new
features it adds, etc. If the patch is specific to a section of the source
code, an email should be generated to the section maintainer notifying him of
the new patch.
The ability to accept/reject a patch for the official kernel should be given
to Linus and friends. This would physically modify the official source tree.
The ability to combine multiple submitted patches into a larger meta-patch
would be nice as well. This ability would be useful for section maintainers
who want to limit the number of patches Linus needs to accept/reject.
The API Documentation System
The documentation system should use the same username & password system as
everything else.
Similar to the patch submission system, the API documentation system should
allow developers to document various functions. When these functions are
modified due to a newly integrated patch, a note should be made in the
function API documentation.
Because the author of a particular function may not maintain documentation on
it, it should be possible to assign yourself to the notification list when the
status of a particular function changes. An email notification should be
dispatched to the people on the notification list if required.
Each function in the Linux kernel tree should automatically be assigned a
function description template. If a patch adds a new function and is
integrated into the official tree, an email notification should be sent to the
patch author to make changes to the documentation for a particular function or
set of functions.
Recommendations for Implementation
The bug tracking, peer review systems and API documentation system are very
similar and it is my opinion that an existing bug tracking & reporting
software could be adapted to handle patch submission & peer review as well as
documentation system with minimal modifications.
A software package that implements all the features for a bug tracking system
above with the exception of an email-based submission facility is Bugzilla
(http://bugzilla.mozilla.org/). This software is open source and is would be
easy to modify to support the additional features required.
CVS has been extremely popular for source repositories, however it lacks the
ability to handle change sets and distributed repositories. I am aware of two
alternatives that do meet the above requirements, Aegis
(http://www.canb.auug.org.au/~millerp/aegis/aegis.html) by Peter Miller and
BitKeeper (http://www.bitkeeper.com) by BitMover Inc. Both appear to provide
comparable functionality, however Aegis provides a more liberal licensing
agreement while BitKeeper has some features specifically requested by Linus. A
more in-depth look at both of these products should be done to determine which
is better for this particular application or if other solutions can be found.
Implementation Notes
The idea is to create a centralized development system for the Linux kernel,
so all information should be available via a single website. This web page
should provide the following:
§ A link for information on accessing the repository directly via the source
repository client software.
§ A link for browsing the current source trees using LXR.
§ A link for accessing the bug reporting & tracking system.
§ A link for accessing the patch submission & peer review system.
§ A link for accessing the documentation submission system.
§ A link for adding, removing & updating user account information
The bug reporting & tracking system should provide the following:
§ A link for information about the bug tracking system.
§ A link for viewing current bugs.
§ A link for adding a new bug.
§ A link to go directly to a specific bug #.
§ A link for adding, removing & updating user account information
The patch submission & peer review system should provide the following:
§ A link for information about the patch submission & peer review system.
§ A link for viewing current patches.
§ A link for adding a new patch.
§ A link to go directly to a specific patch #.
§ A link for adding, removing & updating user account information
Bug reporting submissions should require the following information:
§ Section of the kernel which bug pertains to.
§ One line summary of the bug.
§ Detailed information about the problem.
§ Kernel version.
§ Output of the oops message (if applicable).
§ Steps required to trigger the problem.
§ The current system hardware.
§ The current system software.
§ A patch # fix the problem (if available).
§ Any additional comments.
Patch submission should require the following information:
§ Section of the kernel which patch pertains to.
§ Bug fix or new feature.
§ Bug # this patch pertains to.
§ Any additional comments.
Documentation submission should require the following information:
§ Function to be documented.
§ Summary of documentation change for history log.
§ The documentation change itself.
Final Notes
Because end-users typically will not know the difference between a kernel bug
and a distribution bug, we should encourage distribution support to file bug
reports on behalf of their users. A system utility that probes a system for
all information required for a bug report would benefit distribution support
as well and they may be better equipped to deal with the more technical
comments that may come from a bug report.
The SGI Linux Kernel Crashdump patches (http://oss.sgi.com/projects/lkcd/) may
prove to be an invaluable feature for reporting kernel related problems. It is
difficult at best for even a developer to correctly copy down information from
an Oops from the screen and although crashdumps are not a perfect solution,
they would allow for easy recording of a lot of crashes for end users and
allow for dumping far more information about the crash than an Oops can.
The web-based interface should be friendly to our GUI-impaired friends, but
the alternative email-based interface may prove to be a better solution for
these rare few. It is my opinion that email-based solution for reporting a bug
or submitting a patch would be very useful since a large number of servers
don't have X installed, but for tracking and peer review the web-based
solution should be emphasized.
Jordan
-- Jordan Mendelson : http://jordy.wserv.com Web Services, Inc. : http://www.wserv.com- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/