Re: [RFC PATCH 00/86] Make the kernel preemptible

[Mark Rutland]

On Wed, Nov 08, 2023 at 04:34:41PM -0800, Ankur Arora wrote:
> Mark Rutland <mark.rutland@xxxxxxx> writes:
> 
> > On Tue, Nov 07, 2023 at 01:56:46PM -0800, Ankur Arora wrote:
> >> What's broken:
> >>  - ARCH_NO_PREEMPT (See patch-45 "preempt: ARCH_NO_PREEMPT only preempts
> >>    lazily")
> >>  - Non-x86 architectures. It's trivial to support other archs (only need
> >>    to add TIF_NEED_RESCHED_LAZY) but wanted to hold off until I got some
> >>    comments on the series.
> >>    (From some testing on arm64, didn't find any surprises.)
> >
> > When you say "testing on arm64, didn't find any surprises", I assume you mean
> > with an additional patch adding TIF_NEED_RESCHED_LAZY?
> 
> Yeah. And, handling that in the user exit path.
> 
> > Note that since arm64 doesn't use the generic entry code, that also requires
> > changes to arm64_preempt_schedule_irq() in arch/arm64/kernel/entry-common.c, to
> > handle TIF_NEED_RESCHED_LAZY.
> 
> So, the intent (which got muddied due to this overly large series)
> was to delay handling TIF_NEED_RESCHED_LAZY until we are about to
> return to user.

Ah, I missed that detail -- thanks for clarifying!

> I think arm64_preempt_schedule_irq() should only handle TIF_NEED_RESCHED
> and the _TIF_NEED_RESCHED_LAZY should be handled via _TIF_WORK_MASK
> and do_notify_resume().

Digging a bit more, I think that should still work.

One slight clarification: arm64_preempt_schedule_irq() doesn't look at
TIF_NEED_RESCHED today, as it relies on the scheduler IPI calling
preempt_fold_need_resched() to propogate TIF_NEED_RESCHED into
PREEMPT_NEED_RESCHED. That should still work since this series makes
preempt_fold_need_resched() check tif_need_resched(RESCHED_eager).

I was a bit cnofused because in the generic entry code,
irqentry_exit_cond_resched() explicitly checks for TIF_NEED_RESCHED, and I'm
not sure why it does that rather than relying on the scheduler IPI as above.

> (The design is much clearer in Thomas' PoC:
> https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/)
> 
> >>  - ftrace support for need-resched-lazy is incomplete
> >
> > What exactly do we need for ftrace here?
> 
> Only support for TIF_NEED_RESCHED_LAZY which should be complete.
> That comment was based on a misreading of the code.

Cool; thanks!

Mark.