Re: [PATCH v2] Add kernel config option for fuzz testing.

[Jiri Slaby]

On 08. 03. 20, 7:52, Greg Kroah-Hartman wrote:
> On Sat, Mar 07, 2020 at 05:28:22PM +0100, Jiri Slaby wrote:
>> On 07. 03. 20, 14:58, Tetsuo Handa wrote:
>>> While syzkaller is finding many bugs, sometimes syzkaller examines
>>> stupid operations. Currently we prevent syzkaller from examining
>>> stupid operations by blacklisting syscall arguments and/or disabling
>>> whole functionality using existing kernel config options, but it is
>>> a whack-a-mole approach. We need cooperation from kernel side [1].
>>>
>>> This patch introduces a kernel config option which allows disabling
>>> only specific operations. This kernel config option should be enabled
>>> only when building kernels for fuzz testing.
>>>
>>> We discussed possibility of disabling specific operations at run-time
>>> using some lockdown mechanism [2], but conclusion seems that build-time
>>> control (i.e. kernel config option) fits better for this purpose.
>>> Since patches for users of this kernel config option will want a lot of
>>> explanation [3], this patch provides only kernel config option for them.
>>>
>>> [1] https://github.com/google/syzkaller/issues/1622
>>> [2] https://lkml.kernel.org/r/CACdnJutc7OQeoor6WLTh8as10da_CN=crs79v3Fp0mJTaO=+yw@xxxxxxxxxxxxxx
>>> [3] https://lkml.kernel.org/r/20191216163155.GB2258618@xxxxxxxxx
>>>
>>> Signed-off-by: Tetsuo Handa <penguin-kernel@xxxxxxxxxxxxxxxxxxx>
>>> Cc: Dmitry Vyukov <dvyukov@xxxxxxxxxx>
>>> ---
>>>  lib/Kconfig.debug | 10 ++++++++++
>>>  1 file changed, 10 insertions(+)
>>>
>>> Changes since v1:
>>>   Drop users of this kernel config option.
>>>   Update patch description.
>>>
>>> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
>>> index 53e786e0a604..e360090e24c5 100644
>>> --- a/lib/Kconfig.debug
>>> +++ b/lib/Kconfig.debug
>>> @@ -2208,4 +2208,14 @@ config HYPERV_TESTING
>>>  
>>>  endmenu # "Kernel Testing and Coverage"
>>>  
>>> +config KERNEL_BUILT_FOR_FUZZ_TESTING
>>> +       bool "Build kernel for fuzz testing"
>>
>> If we really want to go this way, I wouldn't limit it for fuzz testing
>> only. Static analyzers, symbolic executors, formal verifiers, etc. --
>> all of them should avoid the paths.
> 
> No, anything that just evaluates the code should be fine, we want static
> analyzers to be processing those code paths.  Just not to run them as
> root on a live system.

Even static analyzers generate real-world counter-examples in .c. They
are ran dynamically to check if the issue is real or if it's only a
shortcoming of static analysis. Klee is one of those and I used to run
it on the kernel some time ago. Throwing such generated input results in
the same weird behavior as using fuzzy testing, while it's still not
fuzzy testing, but accurate testing.

thanks,
-- 
js
suse labs