Re: [RFC PATCH 0/6] module, kbuild: Faster boot with custom kernel.

[Kay Sievers]

On Wed, Feb 18, 2009 at 12:57, Rusty Russell <rusty@xxxxxxxxxxxxxxx> wrote:
> On Wednesday 18 February 2009 19:45:21 Kay Sievers wrote:

>> The current load_module() wraps ~400 lines of pretty heavy code in
>> stop_machine(), if we could possibly make that window smaller, so that
>> multiple instances could prepare some of the work in parallel, and
>> only a fraction of the current work would need to be serialized?
>
> Module load doesn't use stop_machine (on non-error).

I see, right. It's only the mutex serializing all module loading work.

> I'm not sure where the code is spending time.  We could shuffle things around
> so most of load_module doesn't run under the lock, but before that can you tell
> me if we still lose 2-3 seconds on a UP kernel?

I tried only my dual core laptop with my "small" kernel and 50
modules, not the crazy distro setup.

I've added stuff like:
   ktime_t in, delta;
   unsigned long long duration;

   in = ktime_get();

   ...

   delta = ktime_sub(ktime_get(), in);
   duration = (unsigned long long) delta.tv64 >> 10;
   printk("XXX: loading %p (%Ld usecs)\n", umod, duration);

across init_module() and some modules wait for 200-500 milliseconds to
get the lock released, some larger modules spend 50 milliseconds in
load_module(), many of them around 20 milliseconds.

> If so, it's not locking, I'd guess it's probably modprobe userspace and reading in
> the modules from disk, not sys_init_module at all.

Bootcharts suggest that it's not I/O bound. We also see this on fast
SSDs and with pre-loading files into RAM. Older modprobe versions did
some heavy fnmatch() over an insanely large distro module.alias file,
which was very CPU expensive, but that got changed recently.

Thanks,
Kay
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/