Re: [PATCH] epoll: remove the on_list check for 'struct epitem'

[Andrew Morton]

On Fri, 1 Nov 2013 10:09:12 +1100 Stephen Rothwell <sfr@xxxxxxxxxxxxxxxx> wrote:

> Hi Andrew,
> 
> On Wed, 30 Oct 2013 18:32:41 +0000 (GMT) Jason Baron <jbaron@xxxxxxxxxx> wrote:
> >
> > By removing the 'int on_list' field from 'struct epitem', we avoid hitting the
> > BUILD_BUG_ON() for 'struct epitem' being larger than 128 bytes.
> > 
> > In file included from include/linux/init.h:4:0,
> >                  from fs/eventpoll.c:14:
> > fs/eventpoll.c: In function 'eventpoll_init':
> > include/linux/compiler.h:321:20: error: call to '__compiletime_assert_2137' declared with attribute error: BUILD_BUG_ON failed: sizeof(void *) <= 8 && sizeof(struct epitem) > 128
> >     prefix ## suffix();    \
> > 
> > The check to make sure that the 'struct epitem' was actually linked via
> > epi->fllink was added to avoid having the list removal primitives called twice
> > for the same 'struct epitem'. However, the double call possibility was removed
> > by 'Subject: epoll: optimize EPOLL_CTL_DEL using rcu'. There, the call to
> > 'list_del_init()' in eventpoll_release_file() was removed (we now rely on the
> > list delete happening entirely in 'ep_remove()', which is called from
> > eventpoll_release_file()).
> > 
> > There is also the question as to whether multiple ep_remove() calls could
> > happen concurrently. This can not happen since EPOLL_CTL_DEL can't
> > race with eventpoll_release_file() or ep_free() - it has to do an fget()
> > to proceed. Further, eventpoll_release_file() can not race with ep_free(),
> > since they both acquire the 'epmutex'.
> > 
> > Signed-off-by: Jason Baron <jbaron@xxxxxxxxxx>
> 
> Do you want me to put this in my copy of the mmotm instead of reverting
> these three?
> 
> epoll-do-not-take-global-epmutex-for-simple-topologies-fix
> epoll: do not take global 'epmutex' for simple topologies
> epoll: optimize EPOLL_CTL_DEL using rcu

Sure.  Here's my epoll-optimize-epoll_ctl_del-using-rcu-fix.patch:

From: Jason Baron <jbaron@xxxxxxxxxx>
Subject: epoll: remove the on_list check for 'struct epitem'

By removing the 'int on_list' field from 'struct epitem', we avoid hitting
the BUILD_BUG_ON() for 'struct epitem' being larger than 128 bytes.

In file included from include/linux/init.h:4:0,
                 from fs/eventpoll.c:14:
fs/eventpoll.c: In function 'eventpoll_init':
include/linux/compiler.h:321:20: error: call to '__compiletime_assert_2137' declared with attribute error: BUILD_BUG_ON failed: sizeof(void *) <= 8 && sizeof(struct epitem) > 128
    prefix ## suffix();    \

The check to make sure that the 'struct epitem' was actually linked via
epi->fllink was added to avoid having the list removal primitives called
twice for the same 'struct epitem'.  However, the double call possibility
was removed by 'Subject: epoll: optimize EPOLL_CTL_DEL using rcu'.  There,
the call to 'list_del_init()' in eventpoll_release_file() was removed (we
now rely on the list delete happening entirely in 'ep_remove()', which is
called from eventpoll_release_file()).

There is also the question as to whether multiple ep_remove() calls could
happen concurrently.  This can not happen since EPOLL_CTL_DEL can't race
with eventpoll_release_file() or ep_free() - it has to do an fget() to
proceed.  Further, eventpoll_release_file() can not race with ep_free(),
since they both acquire the 'epmutex'.

Signed-off-by: Jason Baron <jbaron@xxxxxxxxxx>
Reported-by: Wu Fengguang <fengguang.wu@xxxxxxxxx>
Cc: Nathan Zimmer <nzimmer@xxxxxxx>
Cc: Eric Wong <normalperson@xxxxxxxx>
Cc: Nelson Elhage <nelhage@xxxxxxxxxxx>
Cc: Al Viro <viro@xxxxxxxxxxxxxxxxxx>
Cc: Davide Libenzi <davidel@xxxxxxxxxxxxxxx>
Cc: "Paul E. McKenney" <paulmck@xxxxxxxxxx>
Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
---

 fs/eventpoll.c |   13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

diff -puN fs/eventpoll.c~epoll-optimize-epoll_ctl_del-using-rcu-fix fs/eventpoll.c
--- a/fs/eventpoll.c~epoll-optimize-epoll_ctl_del-using-rcu-fix
+++ a/fs/eventpoll.c
@@ -171,9 +171,6 @@ struct epitem {
 
 	/* The structure that describe the interested events and the source fd */
 	struct epoll_event event;
-
-	/* The fllink is in use. Since rcu can't do 'list_del_init()' */
-	int on_list;
 };
 
 /*
@@ -707,10 +704,7 @@ static int ep_remove(struct eventpoll *e
 
 	/* Remove the current item from the list of epoll hooks */
 	spin_lock(&file->f_lock);
-	if (epi->on_list) {
-		list_del_rcu(&epi->fllink);
-		epi->on_list = 0;
-	}
+	list_del_rcu(&epi->fllink);
 	spin_unlock(&file->f_lock);
 
 	rb_erase(&epi->rbn, &ep->rbr);
@@ -1273,7 +1267,6 @@ static int ep_insert(struct eventpoll *e
 	epi->event = *event;
 	epi->nwait = 0;
 	epi->next = EP_UNACTIVE_PTR;
-	epi->on_list = 0;
 	if (epi->event.events & EPOLLWAKEUP) {
 		error = ep_create_wakeup_source(epi);
 		if (error)
@@ -1307,7 +1300,6 @@ static int ep_insert(struct eventpoll *e
 	/* Add the current item to the list of active epoll hook for this file */
 	spin_lock(&tfile->f_lock);
 	list_add_tail_rcu(&epi->fllink, &tfile->f_ep_links);
-	epi->on_list = 1;
 	spin_unlock(&tfile->f_lock);
 
 	/*
@@ -1348,8 +1340,7 @@ static int ep_insert(struct eventpoll *e
 
 error_remove_epi:
 	spin_lock(&tfile->f_lock);
-	if (epi->on_list)
-		list_del_rcu(&epi->fllink);
+	list_del_rcu(&epi->fllink);
 	spin_unlock(&tfile->f_lock);
 
 	rb_erase(&epi->rbn, &ep->rbr);
_

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/