Re: seccomp ptrace selftest failures with 4.4-stable [Was: Re: LTS testing with latest kselftests - some failures]

[Luis R. Rodriguez]

On Thu, Jun 22, 2017 at 07:40:49PM -0700, Andy Lutomirski wrote:
> On Thu, Jun 22, 2017 at 6:52 PM, Greg Kroah-Hartman
> <gregkh@xxxxxxxxxxxxxxxxxxx> wrote:
> > On Thu, Jun 22, 2017 at 10:50:43AM -0700, Kees Cook wrote:
> >> On Thu, Jun 22, 2017 at 10:49 AM, Andy Lutomirski <luto@xxxxxxxxxx> wrote:
> >> > On Thu, Jun 22, 2017 at 10:09 AM, Shuah Khan <shuah@xxxxxxxxxx> wrote:
> >> >> On 06/22/2017 10:53 AM, Kees Cook wrote:
> >> Right. I really think stable kernels should be tested with their own
> >> selftests. If some test is needed in a stable kernel it should be
> >> backported to that stable kernel.
> >
> > Well, ideally all new features added to the kernel should be able to be
> > detected by userspace somehow if they are present or not.
> >
> > How do you expect a program to know if a feature has "failed" or is just
> > "not enabled/present in this kernel"?  Normally with syscalls this is
> > easy, same for sysfs changes.  Is seccomp in the bad state where there
> > is no way to detect the two different states here?  How is userspace
> > supposed to deal with that?
> >
> > We make fun of glibc having a zillion crazy tests to determine kernel
> > features, and recently, just not wrapping new syscalls at all because
> > they are just frustrated at the compatibility issues over time.  Let's
> > not make their life any harder than it has to be please.
> >
> > I don't see how any of the kselftest programs are any different than any
> > other userspace program that wants to use our kernel api, and as such,
> > any version of kselftest should be able to successfully run on any
> > kernel release.  If not, then we messed up in how we either wrote the
> > test, or how we added a new kernel api.  Neither is acceptable.
> 
> That's a fair point.

I agreed with it as well just a few threads ago due to similar issues, however,
thinking this over I'm afraid this has some interesting side consequences for
fixes and what code goes upstream into kselftest.

<-- snip -->

> The problem is that the fix is moderately intrusive and doesn't seem
> like a great candidate for backporting, although we could plausibly do
> it.

Such is the case often actually.

So taking the position that any kselftest script on linux-next or a future
kernel should never break stable implicate that *any* fix going upstream for
which there is a respective ksefltest test *must* have a stable upstream fix.

Its not obvious to me that everyone is aware of this. What do we do about
those cases where we *don't* want a stable fix due to the complexity?

  Luis