RE: [PATCH 1/2] Refine mutex and rcu method in module.c,kernel<3.2.9>

[Chen, Dennis (SRDC SW)]

From: Chen, Dennis (SRDC SW) 
Sent: Thursday, March 08, 2012 8:18 PM
To: Rusty Russell; linux-kernel@xxxxxxxxxxxxxxx
Cc: Chen, Dennis (SRDC SW)
Subject: RE: [PATCH 1/2] Refine mutex and rcu method in module.c, kernel<3.2.9>

From: Rusty Russell [mailto:rusty@xxxxxxxxxx] 
Sent: Thursday, March 08, 2012 5:18 PM
To: Chen, Dennis (SRDC SW); linux-kernel@xxxxxxxxxxxxxxx
Cc: Chen, Dennis (SRDC SW)
Subject: Re: [PATCH 1/2] Refine mutex and rcu method in module.c, kernel<3.2.9>

On Wed, 7 Mar 2012 14:51:06 +0000, "Chen, Dennis (SRDC SW)" <Dennis1.Chen@xxxxxxx> wrote:
> 1. Narrow down the granularity of mutex_lock/ mutex_unlock
> 2. Replace some unnecessary mutex_lock/mutex_unlock pairs with RCU
> 3. Refine the consistent calling style of RCU functioan 

Hi Dennis,

        This follows a logical evolution, where we wean off the mutex,
but AFAICT this is lost in the noise.  Taking the mutex might be naive,
but adding or removing a module is the slow path.  Or am I missing
something?

> -
> -       /* Now sew it into the lists so we can get lockdep and oops
> -        * info during argument parsing.  No one should access us, since
> -        * strong_try_module_get() will fail.
> -        * lockdep/oops can run asynchronous, so use the RCU list insertion
> -        * function to insert in a way safe to concurrent readers.
> -        * The mutex protects against concurrent writers.
> -        */
> -       mutex_lock(&module_mutex);
> +        
> +        /* Concurrent writers for the global modules list are protected by RCU*/
>         if (find_module(mod->name)) {
>                 err = -EEXIST;
>                 goto unlock;
>         }

RCU does not protect concurrent writers:

> -
> +        
>         /* This has to be done once we're sure module name is unique. */
>         dynamic_debug_setup(info.debug, info.num_debug);

Now this is racy...

======================================================================================

Hi Rusty,

I known RCU doesn't protect concurrent writers, so all the writers for the global modules list has been 
protected by the original module_mutex in the 2-patch, I just make the scope of module_mutex become smaller 
as it can, for example, dynamic_debug_setup doesn't touch modules, so it should not be in the protection of
module_mutex.

I am a person like to see a perfect world, especially in the kernel space:) Maybe I can write a test case
to trigger something you don't expect to see while the original codes will...let's think about it

BRs,
Dennis 

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Hi Rusty,

Pls notice the following change in the patch (in set_all_modules_text_ro function):

/* Iterate through all modules and set each module's text as RO */
@@ -1693,7 +1699,7 @@
 {
        struct module *mod;
 
-       mutex_lock(&module_mutex);
+       rcu_read_lock();
        list_for_each_entry_rcu(mod, &modules, list) {
                if ((mod->module_core) && (mod->core_text_size)) {
                        set_page_attributes(mod->module_core,
@@ -1706,7 +1712,7 @@
                                                set_memory_ro);
                }
        }
-       mutex_unlock(&module_mutex);
+       rcu_read_unlock();
 }

This function just needs to iterate the modules list, but now it holds a unnecessary lock when it does that, 
The other module can't be inserted during this operation, also can you make sure the set_page_attributes will
run smoothly all the time, if not it's a risk action to hold a lock.
So summary--
I think the idea for kernel module protection is simple:
Writers to modules, use mutex_lock
Readers, use rcu. __ALL__ codes here should be with a unified style! This will make our kernel gracefully.

PS: my comments in the patch " /* Concurrent writers for the global modules list are protected by RCU*/" is not right, RCU
Should be mutex lock.

BRs,
Dennis 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/