3When contemplating a Linux kernel development project, it can be tempting
   4to jump right in and start coding.  As with any significant project,
   5though, much of the groundwork for success is best laid before the first
   6line of code is written.  Some time spent in early planning and
   7communication can save far more time later on.
  12Like any engineering project, a successful kernel enhancement starts with a
  13clear description of the problem to be solved.  In some cases, this step is
  14easy: when a driver is needed for a specific piece of hardware, for
  15example.  In others, though, it is tempting to confuse the real problem
  16with the proposed solution, and that can lead to difficulties.
  18Consider an example: some years ago, developers working with Linux audio
  19sought a way to run applications without dropouts or other artifacts caused
  20by excessive latency in the system.  The solution they arrived at was a
  21kernel module intended to hook into the Linux Security Module (LSM)
  22framework; this module could be configured to give specific applications
  23access to the realtime scheduler.  This module was implemented and sent to
  24the linux-kernel mailing list, where it immediately ran into problems.
  26To the audio developers, this security module was sufficient to solve their
  27immediate problem.  To the wider kernel community, though, it was seen as a
  28misuse of the LSM framework (which is not intended to confer privileges
  29onto processes which they would not otherwise have) and a risk to system
  30stability.  Their preferred solutions involved realtime scheduling access
  31via the rlimit mechanism for the short term, and ongoing latency reduction
  32work in the long term.
  34The audio community, however, could not see past the particular solution
  35they had implemented; they were unwilling to accept alternatives.  The
  36resulting disagreement left those developers feeling disillusioned with the
  37entire kernel development process; one of them went back to an audio list
  38and posted this:
  40        There are a number of very good Linux kernel developers, but they
  41        tend to get outshouted by a large crowd of arrogant fools. Trying
  42        to communicate user requirements to these people is a waste of
  43        time. They are much too "intelligent" to listen to lesser mortals.
  47The reality of the situation was different; the kernel developers were far
  48more concerned about system stability, long-term maintenance, and finding
  49the right solution to the problem than they were with a specific module.
  50The moral of the story is to focus on the problem - not a specific solution
  51- and to discuss it with the development community before investing in the
  52creation of a body of code.
  54So, when contemplating a kernel development project, one should obtain
  55answers to a short set of questions:
  57 - What, exactly, is the problem which needs to be solved?
  59 - Who are the users affected by this problem?  Which use cases should the
  60   solution address?
  62 - How does the kernel fall short in addressing that problem now?
  64Only then does it make sense to start considering possible solutions.
  69When planning a kernel development project, it makes great sense to hold
  70discussions with the community before launching into implementation.  Early
  71communication can save time and trouble in a number of ways:
  73 - It may well be that the problem is addressed by the kernel in ways which
  74   you have not understood.  The Linux kernel is large and has a number of
  75   features and capabilities which are not immediately obvious.  Not all
  76   kernel capabilities are documented as well as one might like, and it is
  77   easy to miss things.  Your author has seen the posting of a complete
  78   driver which duplicated an existing driver that the new author had been
  79   unaware of.  Code which reinvents existing wheels is not only wasteful;
  80   it will also not be accepted into the mainline kernel.
  82 - There may be elements of the proposed solution which will not be
  83   acceptable for mainline merging.  It is better to find out about
  84   problems like this before writing the code.
  86 - It's entirely possible that other developers have thought about the
  87   problem; they may have ideas for a better solution, and may be willing
  88   to help in the creation of that solution.
  90Years of experience with the kernel development community have taught a
  91clear lesson: kernel code which is designed and developed behind closed
  92doors invariably has problems which are only revealed when the code is
  93released into the community.  Sometimes these problems are severe,
  94requiring months or years of effort before the code can be brought up to
  95the kernel community's standards.  Some examples include:
  97 - The Devicescape network stack was designed and implemented for
  98   single-processor systems.  It could not be merged into the mainline
  99   until it was made suitable for multiprocessor systems.  Retrofitting
 100   locking and such into code is a difficult task; as a result, the merging
 101   of this code (now called mac80211) was delayed for over a year.
 103 - The Reiser4 filesystem included a number of capabilities which, in the
 104   core kernel developers' opinion, should have been implemented in the
 105   virtual filesystem layer instead.  It also included features which could
 106   not easily be implemented without exposing the system to user-caused
 107   deadlocks.  The late revelation of these problems - and refusal to
 108   address some of them - has caused Reiser4 to stay out of the mainline
 109   kernel.
 111 - The AppArmor security module made use of internal virtual filesystem
 112   data structures in ways which were considered to be unsafe and
 113   unreliable.  This code has since been significantly reworked, but
 114   remains outside of the mainline.
 116In each of these cases, a great deal of pain and extra work could have been
 117avoided with some early discussion with the kernel developers.
 1203.3: WHO DO YOU TALK TO?
 122When developers decide to take their plans public, the next question will
 123be: where do we start?  The answer is to find the right mailing list(s) and
 124the right maintainer.  For mailing lists, the best approach is to look in
 125the MAINTAINERS file for a relevant place to post.  If there is a suitable
 126subsystem list, posting there is often preferable to posting on
 127linux-kernel; you are more likely to reach developers with expertise in the
 128relevant subsystem and the environment may be more supportive.
 130Finding maintainers can be a bit harder.  Again, the MAINTAINERS file is
 131the place to start.  That file tends to not always be up to date, though,
 132and not all subsystems are represented there.  The person listed in the
 133MAINTAINERS file may, in fact, not be the person who is actually acting in
 134that role currently.  So, when there is doubt about who to contact, a
 135useful trick is to use git (and "git log" in particular) to see who is
 136currently active within the subsystem of interest.  Look at who is writing
 137patches, and who, if anybody, is attaching Signed-off-by lines to those
 138patches.  Those are the people who will be best placed to help with a new
 139development project.
 141If all else fails, talking to Andrew Morton can be an effective way to
 142track down a maintainer for a specific piece of code.
 1453.4: WHEN TO POST?
 147If possible, posting your plans during the early stages can only be
 148helpful.  Describe the problem being solved and any plans that have been
 149made on how the implementation will be done.  Any information you can
 150provide can help the development community provide useful input on the
 153One discouraging thing which can happen at this stage is not a hostile
 154reaction, but, instead, little or no reaction at all.  The sad truth of the
 155matter is (1) kernel developers tend to be busy, (2) there is no shortage
 156of people with grand plans and little code (or even prospect of code) to
 157back them up, and (3) nobody is obligated to review or comment on ideas
 158posted by others.  If a request-for-comments posting yields little in the
 159way of comments, do not assume that it means there is no interest in the
 160project.  Unfortunately, you also cannot assume that there are no problems
 161with your idea.  The best thing to do in this situation is to proceed,
 162keeping the community informed as you go.
 167If your work is being done in a corporate environment - as most Linux
 168kernel work is - you must, obviously, have permission from suitably
 169empowered managers before you can post your company's plans or code to a
 170public mailing list.  The posting of code which has not been cleared for
 171release under a GPL-compatible license can be especially problematic; the
 172sooner that a company's management and legal staff can agree on the posting
 173of a kernel development project, the better off everybody involved will be.
 175Some readers may be thinking at this point that their kernel work is
 176intended to support a product which does not yet have an officially
 177acknowledged existence.  Revealing their employer's plans on a public
 178mailing list may not be a viable option.  In cases like this, it is worth
 179considering whether the secrecy is really necessary; there is often no real
 180need to keep development plans behind closed doors.
 182That said, there are also cases where a company legitimately cannot
 183disclose its plans early in the development process.  Companies with
 184experienced kernel developers may choose to proceed in an open-loop manner
 185on the assumption that they will be able to avoid serious integration
 186problems later.  For companies without that sort of in-house expertise, the
 187best option is often to hire an outside developer to review the plans under
 188a non-disclosure agreement.  The Linux Foundation operates an NDA program
 189designed to help with this sort of situation; more information can be found
 194This kind of review is often enough to avoid serious problems later on
 195without requiring public disclosure of the project.
 196 kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.