Re: seccomp ptrace selftest failures with 4.4-stable [Was: Re: LTS testing with latest kselftests - some failures]

[Kees Cook]

On Thu, Jun 22, 2017 at 10:49 AM, Andy Lutomirski <luto@xxxxxxxxxx> wrote:
> On Thu, Jun 22, 2017 at 10:09 AM, Shuah Khan <shuah@xxxxxxxxxx> wrote:
>> On 06/22/2017 10:53 AM, Kees Cook wrote:
>>> On Thu, Jun 22, 2017 at 9:18 AM, Sumit Semwal <sumit.semwal@xxxxxxxxxx> wrote:
>>>> Hi Kees, Andy,
>>>>
>>>> On 15 June 2017 at 23:26, Sumit Semwal <sumit.semwal@xxxxxxxxxx> wrote:
>>>>> 3. 'seccomp ptrace hole closure' patches got added in 4.7 [3] -
>>>>> feature and test together.
>>>>> - This one also seems like a security hole being closed, and the
>>>>> 'feature' could be a candidate for stable backports, but Arnd tried
>>>>> that, and it was quite non-trivial. So perhaps  we'll need some help
>>>>> from the subsystem developers here.
>>>>
>>>> Could you please help us sort this out? Our goal is to help Greg with
>>>> testing stable kernels, and currently the seccomp tests fail due to
>>>> missing feature (seccomp ptrace hole closure) getting tested via
>>>> latest kselftest.
>>>>
>>>> If you feel the feature isn't a stable candidate, then could you
>>>> please help make the test degrade gracefully in its absence?
>>>
>>> I don't really want to have that change be a backport -- it's quite
>>> invasive across multiple architectures.
>>>
>>> I would say just add a kernel version check to the test. This is
>>> probably not the only selftest that will need such things. :)
>>
>> Adding release checks to selftests is going to problematic for maintenance.
>> Tests should fail gracefully if feature isn't supported in older kernels.
>>
>> Several tests do that now and please find a way to check for dependencies
>> and feature availability and fail the test gracefully. If there is a test
>> that can't do that for some reason, we can discuss it, but as a general
>> rule, I don't want to see kselftest patches that check release.
>
> If a future kernel inadvertently loses the new feature and degrades to
> the behavior of old kernels, that would be a serious bug and should be
> caught.

Right. I really think stable kernels should be tested with their own
selftests. If some test is needed in a stable kernel it should be
backported to that stable kernel.

-Kees

-- 
Kees Cook
Pixel Security