[PATCH v7 00/10] kthread: Kthread worker API improvements

From: Petr Mladek
Date: Mon May 30 2016 - 11:00:01 EST

I send the kthread worker API improvements separately as discussed
in v6, see
They seem to be ready for inclusion in 4.8.

I will send the conversion of the particular kthreads once
the API changes are in some maintainers three (-mm?) and
visible in linux-next. If nobody suggests some other approach.

Also I plan to continue with conversion of more kthreads.

Just to remember. The intention of this patchset is to make
it easier to manipulate and maintain kthreads. Especially, I want
to replace all the custom main cycles with a generic one.
Also I want to make the kthreads sleep in a consistent
state in a common place when there is no work.

Changes against v6:

+ no changes.

Changes against v5:

+ removed spin_trylock() from delayed_kthread_work_timer_fn();
instead temporary released worked->lock() when calling
del_timer_sync(); made sure that any queueing was blocked
by work->canceling in the meatime

+ used 0th byte for KTW_FREEZABLE to reduce confusion

+ fixed warnings in comments reported by make htmldocs

+ sigh, there was no easy way to create an empty va_list
that would work on all architectures; decided to make
@namefmt generic in create_kthread_worker_on_cpu()

+ converted khungtaskd a better way; it was inspired by
the recent changes that appeared in 4.6-rc1

Changes against v4:

+ added worker->delayed_work_list; it simplified the check
for pending work; we do not longer need the new timer_active()
function; also we do not need the link work->timer. On the
other hand we need to distinguish between the normal and
the delayed work by a boolean parameter passed to
the common functions, e.g. __cancel_kthread_work_sync()

+ replaced most try_lock repeat cycles with a WARN_ON();
the API does not allow to use the work with more workers;
so such a situation would be a bug; it removed the
complex try_lock_kthread_work() function that supported
more modes;

+ renamed kthread_work_pending() to queuing_blocked();
added this function later when really needed

+ renamed try_to_cancel_kthread_work() to __cancel_kthread_work();
in fact, this a common implementation for the async cancel()

+ removed a dull check for invalid cpu number in
create_kthread_worker_on_cpu(); removed some other unnecessary
code structures as suggested by Tejun

+ consistently used bool return value in all new __cancel functions

+ fixed ordering of cpu and flags parameters in
create_kthread_worker_on_cpu() vs. create_kthread_worker()

+ used memset in the init_kthread_worker()

+ updated many comments as suggested by Tejun and as
required the above changes

+ removed obsolete patch adding timer_active()

+ removed obsolete patch for using try_lock in flush_kthread_worker()

+ double checked all existing users of kthread worker API
that they reinitialized the work when the worker was started
and would not print false warnings; all looked fine

+ added taken acks for the Intel Powerclamp conversion

Changes against v3:

+ allow to free struct kthread_work from its callback; do not touch
the struct from the worker post-mortem; as a side effect, the structure
must be reinitialized when the worker gets restarted; updated
khugepaged, and kmemleak accordingly

+ call del_timer_sync() with worker->lock; instead, detect canceling
in the timer callback and give up an attempt to get the lock there;
do busy loop with spin_is_locked() to reduce cache bouncing

+ renamed ipmi+func() -> ipmi_kthread_worker_func() as suggested
by Corey

+ added some collected Reviewed-by

Changes against v2:

+ used worker->lock to synchronize the operations with the work
instead of the PENDING bit as suggested by Tejun Heo; it simplified
the implementation in several ways

+ added timer_active(); used it together with del_timer_sync()
to cancel the work a less tricky way

+ removed the controversial conversion of the RCU kthreads

+ added several other examples: hung_task, kmemleak, ipmi,
IB/fmr_pool, memstick/r592, intel_powerclamp

+ the helper fixes for the ring buffer benchmark has been improved
as suggested by Steven; they already are in the Linus tree now

+ fixed a possible race between the check for existing khugepaged
worker and queuing the work

Changes against v1:

+ remove wrappers to manipulate the scheduling policy and priority

+ remove questionable wakeup_and_destroy_kthread_worker() variant

+ do not check for chained work when draining the queue

+ allocate struct kthread worker in create_kthread_work() and
use more simple checks for running worker

+ add support for delayed kthread works and use them instead
of waiting inside the works

+ rework the "unrelated" fixes for the ring buffer benchmark
as discussed in the 1st RFC; also sent separately

+ convert also the consumer in the ring buffer benchmark

I have tested this patch set against the stable Linus tree
for 4.7-rc1.

Comments against v6 can be found at

Petr Mladek (10):
kthread/smpboot: Do not park in kthread_create_on_cpu()
kthread: Allow to call __kthread_create_on_node() with va_list args
kthread: Add create_kthread_worker*()
kthread: Add drain_kthread_worker()
kthread: Add destroy_kthread_worker()
kthread: Detect when a kthread work is used by more workers
kthread: Initial support for delayed kthread work
kthread: Allow to cancel kthread work
kthread: Allow to modify delayed kthread work
kthread: Better support freezable kthread workers

include/linux/kthread.h | 57 +++++
kernel/kthread.c | 571 +++++++++++++++++++++++++++++++++++++++++++-----
kernel/smpboot.c | 5 +
3 files changed, 581 insertions(+), 52 deletions(-)

CC: Catalin Marinas <catalin.marinas@xxxxxxx>
CC: linux-watchdog@xxxxxxxxxxxxxxx
CC: Corey Minyard <minyard@xxxxxxx>
CC: openipmi-developer@xxxxxxxxxxxxxxxxxxxxx
CC: Doug Ledford <dledford@xxxxxxxxxx>
CC: Sean Hefty <sean.hefty@xxxxxxxxx>
CC: Hal Rosenstock <hal.rosenstock@xxxxxxxxx>
CC: linux-rdma@xxxxxxxxxxxxxxx
CC: Maxim Levitsky <maximlevitsky@xxxxxxxxx>
CC: Zhang Rui <rui.zhang@xxxxxxxxx>
CC: Eduardo Valentin <edubezval@xxxxxxxxx>
CC: Jacob Pan <jacob.jun.pan@xxxxxxxxxxxxxxx>
CC: linux-pm@xxxxxxxxxxxxxxx
CC: Sebastian Andrzej Siewior <bigeasy@xxxxxxxxxxxxx>