Re: [RFC 00/10] kmod: stress test driver, few fixes and enhancements

[Luis R. Rodriguez]

On Thu, Dec 08, 2016 at 10:47:51AM -0800, Luis R. Rodriguez wrote:
> Upon running into an old kmod v19 issue with mount (get_fs_type()) a few of us
> hunted for the cause of the issue. Although the issue ended up being a
> userspace issue, a stress test driver was written to help reproduce the issue,
> and along the way a few other fixes and sanity checks were implemented.
> 
> I've taken the time to generalize the stress test driver as a kselftest driver
> with a 9 test cases. The last two test cases reveal an existing issue which
> is not yet addressed upstream, even if you have kmod v19 present. A fix is
> proposed in the last patch. Orignally we had discarded this patch as too
> complex due to the alias handling, but upon further analysis of test cases
> and memory pressure issues, it seems worth considering. Other than the
> last patch I don't think much of the other patches are controversial, but
> sending as RFC first just in case.
> 
> If its not clear, an end goal here is to make module loading a bit more
> deterministic with stronger sanity checks and stress tests. Please note,
> the stress test diver requires 4 GiB of RAM to run all tests without running
> out of memory. A lot of this has to do with the memory requirements needed
> for a dynamic test for multiple threads, but note that the final memory
> pressure and OOMs actually don't come from this allocation, but instead
> from many finit_module() calls, this consumes quite a bit of memory, specially
> if you have a lot of dependencies which also need to be loaded prior to
> your needed module -- as is the case for filesystem drivers.
> 
> These patches are available on my linux-next git-tree on my branch
> 20161208-kmod-test-driver-try2 [0], which is based on linux-next tag
> next-20161208. Patches are also available based on v4.9-rc8 [1] for
> those looking for a bit more stable tree given x86_64 on linux-next is
> hosed at the moment.
> 
> Since kmod.c doesn't seem to get much love, and since I've been digging
> quite a bit into it for other users (firmware) I suppose I could volunteer
> myself to maintain this code as well, unless there are oppositions to this.
> 
> [0] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20161208-kmod-test-driver-try2
> [1] https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux.git/log/?h=20161208-kmod-test-driver
> 
> Luis R. Rodriguez (10):
>   kmod: add test driver to stress test the module loader
>   module: fix memory leak on early load_module() failures
>   kmod: add dynamic max concurrent thread count
>   kmod: provide wrappers for kmod_concurrent inc/dec
>   kmod: return -EBUSY if modprobe limit is reached
>   kmod: provide sanity check on kmod_concurrent access
>   kmod: use simplified rate limit printk
>   sysctl: add support for unsigned int properly
>   kmod: add helpers for getting kmod count and limit
>   kmod: add a sanity check on module loading
> 

A lot of good discussions have come up form this, and so also
a few more patches. I'm going to split up the work into changes
which make sense now and leave debug work for a follow up later.

  Luis