Folks, since bug tracking is the next thing we are attacking here at
BitMover, I have a great deal of interest in the bug tracking discussion
which happened last night at the summit. We already have a prototyped bug
tracking system which we are integrating into BitKeeper, but as usual,
it isn't good enough for the kernel team who have an interesting set
of requirements.
We want to make sure that the BK bug tracking system _could_ be used for
the kernel effort. Whether it is or not will be decided after years of
licensing flamewars, etc., etc. We're not going to go there so please
don't try.
Where I want to go is a discussion of requirements, then we converge on
a strawman proposal, and then all of the people who care enough about
this can go implement an answer and Linus and/or Alan and/or whoever can
choose one.
Let's stay focussed on arriving at a good set of agreed on requirements.
I've done a brain dump below. I'll track this mail thread and update this
doc as the discussion unfolds. I'll post updates as it is needed. If this
discussion takes on a life of its own, I may try and grab all the mail and
archive it so that people can browse; we'll see, it may be that everyone
is so busy that this will be the first and last message on the topic.
--lm
Brain dump on the bug tracking problem from the Kernel Summit discussions
[SCCS/s.BUGS vers 1.2 2001/04/01 10:46:55]
Outline
Problems
Problem details
Past experiences
Requirements
Problems
- getting quality bug reports
- not losing any bugs
- sorting low signal vs high signal into a smaller high signal pile
- simplified, preferably NNTP, access to the bug database (Linus
would use this; he's unlikely to use anything else)
Problem details
Bug report quality
There was lots of discussion on this. The main agreement was that we
wanted the bug reporting system to dig out as much info as possible
and prefill that. There was a lot of discussion about possible tools
that would dig out the /proc/pci info; there was discussion about
Andre's tools which can tell you if you can write your disk; someone
else had something similar.
But the main thing was to extract all the info we could
automatically. One thing was the machine config (hardware and
at least kernel version). The other thing was extract any oops
messages and get a stack traceback.
The other main thing was to define some sort of structure to the
bug report and try and get the use to categorize if they could.
In an ideal world, we would use the maintainers file and the
stack traceback to cc the bug to the maintainer. I think we want
to explore this a bit. I'm not sure that the maintainer file is
the way to go, what if we divided it up into much broader chunks
like "fs", "vm", "network drivers", and had a mail forwarder
for each area. That could fan out to the maintainers.
Not losing bugs
While there was much discussion about how to get rid of bad,
incorrect, and/or duplicate bug reports, several people - Alan
in particular - made the point that having a complete collection
of all bug reports was important. You can do data mining across
all/part of them and look for patterns. The point was that there
is some useful signal amongst all the noise so we do not want to
lose that signal.
Signal/noise
We had a lot of discussion about how to deal with signal/noise.
The bugzilla proponents thought we could do this with some additional
hacking to bugzilla. I, given the BitKeeper background, thought
that we could do this by having two databases, one with all the
crud in it and another with just the screened bugs in it. No matter
how it is done, there needs to be some way to both keep a full list,
which will likely be used only for data mining, and another, much
smaller list of screened bugs. Jens wants there to be a queue of
new bugs and a mechanism where people can come in the morning, pull
a pile of bugs off of the queue, sort them, sending some to the real
database. This idea has a lot of merit, it needs some pondering as
DaveM would say, to get to the point that we have a workable mechanism
which works in a distributed fashion.
The other key point seemed to be that if nobody picked up a bug and
nobody said that this bug should be picked up, then the bug expires
out of the pending queue. It gets stashed in the bug archive for
mining purposes and it can be resurrected if it later becomes a real
bug, but the key point seems to be that it _automatically_ disappears
out of the pending queue. I personally am very supportive of this
model. We need some way to just let junk stay junk. If junk has to
be pruned out of the system by humans, the system sucks. The system,
not humans, needs to autoprune.
Simplified access: browsing and updating
Linus made the point that mailing lists suck. He isn't on any and
refuses to join any. He reads lists with a news reader. I think
people should sit up and listen to that - it's a key point. If your
mailing list isn't gatewayed to a newsgroup, he isn't reading it and
a lot of other people aren't either.
There was a fair bit of discussion about how to get the bug database
connected to news. There doesn't seem to be any reason that the
bug system couldn't be a news server/gateway. You should be able to
browse
bitbucket.kernel.bugs - all the unscreened crud
screened.kernel.bugs - all bugs which have been screened
fs.kernel.bugs - screened bugs in the "fs" category
ext2.kernel.bugs - screened bugs in the "ext2" category
eepro.kernel.bugs - screened bugs in the "eepro" category
etc.
Furthermore, the bugs should be structured once they are screened,
i.e., they have a set of fields like (this is a strawman):
Synopsis - one line man-page like summary of the bug
Severity - how critical is this bug?
Priority - how soon does it need to be fixed?
Category - subsystem in which the bug occurs
Description - details on the bug, oops, stack trace, etc.
Hardware - hardware info
Software - kernel version, glibc version, etc.
Suggested fix - any suggestion on how to fix it
Interest list - set of email addresses and/or newsgroups for updates
It ought to work that if someone posts a followup to the bug then if
the followup changes any of the fields that gets propagated to the
underlying bug database. If this is done properly the news reader will
be the only interface that most people use.
Past experiences
This is a catch all for sound bytes that we don't want to forget...
- Sorting bugs by hand is a pain in the ass (Ted burned out on it and
Alan refuses to say that it is the joy of his life to do it)
- bug systems tend to "get in the way". Unless they are really trivial
to submit, search, update then people get tired of using them and go
back to the old way
- one key observation: let bugs "expire" much like news expires. If
nobody has been whining enough that it gets into the high signal
bug db then it probably isn't real. We really want a way where no
activity means let it expire.
- Alan pointed out that having all of the bugs someplace is useful,
you can search through the 200 similar bugs and notice that SMP
is the common feature.
Requirements
This section is mostly empty, it's here as a catch all for people's
bullet items.
- it would be very nice to be able to cross reference bugs to bug fixes
in the source management system, as well as the other way around.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Sat Apr 07 2001 - 21:00:08 EST