Stable maintainer tools

[Sasha Levin]

Hi all,

As I've started working on maintaining stable kernels I found that it's nearly
impossible to assure the correctness of the resulting kernel without a fair
amount of tools to help with the job.

Every missing commit means that there's an unfix bug, which may be critical.
Likewise, every commit that wasn't intended to that kernel version, or is
incorrectly backported means we might be introducing new bugs - some might
be non obvious to detect.

There is also very little coordination and cooperation between maintainers when
there should be much more. It's important to audit your tree against other
maintainers and both fix your tree if necessary and report mistakes to other
maintainers.

Therefore, to try and create a common toolset for stable tree maintainers
I've cleaned up and published my scripts, and below I'll describe some of
my workflows with them. The scripts are built around git, and I'd be happy
to take in a quilt version in as well if someone wants to donate his scripts.

The code is available at
git://git.kernel.org/pub/scm/linux/kernel/git/sashal/stable-tools.git . Ideas,
suggestions, bug reports and patches are more than welcome!


Some usecases:

* Initial selection: 'stable audit <range>' to generate a report of commits in
the provided range. An example with some explanation of the result:

Example 1:

	$ stable audit-range v4.3-rc5..v4.3-rc6
	[E+R] 874bbfe workqueue: make sure delayed work run in local cpu
		v3.10..stable/linux-3.10.y
		v3.14..stable/linux-3.14.y
		v4.1..stable/linux-4.1.y
		v4.2..stable/linux-4.2.y

Let's break the first part down:

	* E: Means the commit is marked for stable and exists in other stable
branches. Other options are "M" for a commit that is marked for stable but
doesn't exist in any other stable branch, and "N" for a commit not marked for
stable but exists in other stable branches.

	* "+": Means that the commit applies cleanly on the current branch. If
it wouldn't we'd see a "-" there.

	* "R": Means that the commit looks relevant to us. Either it "Fixes:"
a commit in our current branch, or the version tagging says it applies to us.

This is followed by a list of the branches where that commit exists.

Example 2:

	[N+] 9911a2d pinctrl: imx25: ensure that a pin with id i is at position i in the info array:
		v4.1..stable/linux-4.1.y
		v4.2..stable/linux-4.2.y

Pretty straightforward, while the commit isn't marked for stable, it exists in
other stable branches, so I should probably look at it as well.

Example 3:

	[M-] 2b02ec7 drm/radeon: add quirk for ASUS R7 370
	Possible dependency chain:
		2b02ec79004388a8c65e227bc289ed891b5ac8c6
		e78654799135a788a941bacad3452fbd7083e518

This is an example of a commit marked for stable, but it doesn't apply cleanly
on our branch (creating conflicts). In that scenario, we also get a list of
possible dependencies.

A list of dependencies may help us decide if the commit is relevant for us or
not, and if so - it can show how should it be backported. It may also point
out missing commits.

We can see that cherry-picking the commit as is doesn't work:

	$ git cherry-pick 2b02ec7
	error: could not apply 2b02ec7... drm/radeon: add quirk for ASUS R7 370

But if we follow the dependency list, we can see what commit was missing and
add it in as well:

	$ git cherry-pick e78654799135a788a941bacad3452fbd7083e518 2b02ec79004388a8c65e227bc289ed891b5ac8c6
	[detached HEAD 1f05b91] drm/radeon: add quirk for MSI R7 370
	 Author: Maxim Sheviakov <mrader3940@xxxxxxxxx>
	 Date: Wed Sep 23 17:10:51 2015 -0400
	 1 file changed, 1 insertion(+)
	[detached HEAD e31318c] drm/radeon: add quirk for ASUS R7 370
	 Author: Alex Deucher <alexander.deucher@xxxxxxx>
	 Date: Fri Oct 2 16:12:07 2015 -0400
	 1 file changed, 1 insertion(+)


Another option for patch selection is 'stable steal-commits'. This script will
go through a different stable tree and will attempt to grab all commits from
the other stable tree onto this stable tree.

The usecase here is that if there's another stable tree which is close in
version to your stable tree, you may want to start by grabbing commits out of
it, and then audit the result for correctness. It can save lots of time
automagically grabbing all the trivial commits leaving you to deal with the
more complicated and interesting ones.


* Backporting: When backporting a patch, it's important to learn from and
compare to the work of other maintainers. 'stable find-alts' can be used to
find the commit in other stable trees, where it would would possible to review
the work done by different maintainers to get insight on backporting.

Also, 'stable deps' can be used to provide a commit dependency chain between
the current branch and the provided commit. If the script is successful,
applying all commits one after the other would allow you to apply the given
commit.


* Validation/Bookkeeping: After the tree is complete, we want to audit it's
contents and make sure we haven't missed any commits. It's also important to
keep track of which commits shouldn't go into the tree so we won't end up
picking them later by mistake.

For this, 'stable show-missing' is useful to generate a list of all commits
that exist in the provided range but don't exist in the current branch. This
list should be audited and missing commits should be added to the branch, and
commits that shouldn't go in should be added to an ignore list.


* Build/sanity/ltp/etc testing: Run a test, and either 'stable yank' to remove
commits or 'stable insert' to insert commits into the tree based on the result
of the test.


Thanks,
Sasha
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/