Re: Kmemleak infrastructure improvement for task_struct leaks and call_rcu()

[Catalin Marinas]

On Sun, May 10, 2020 at 05:27:41PM -0400, Qian Cai wrote:
> On May 9, 2020, at 5:44 AM, Catalin Marinas <catalin.marinas@xxxxxxx> wrote:
> > On Thu, May 07, 2020 at 01:29:04PM -0400, Qian Cai wrote:
> >> On May 7, 2020, at 1:16 PM, Catalin Marinas <catalin.marinas@xxxxxxx> wrote:
> >>> I don't mind adding additional tracking info if it helps with debugging.
> >>> But if it's for improving false positives, I'd prefer to look deeper
> >>> into figure out why the pointer reference graph tracking failed.
> >> 
> >> No, the task struct leaks are real leaks. It is just painful to figure
> >> out the missing or misplaced put_task_struct() from the kmemleak
> >> reports at the moment.
> > 
> > We could log the callers to get_task_struct() and put_task_struct(),
> > something like __builtin_return_address(0) (how does this work if the
> > function is inlined?). If it's not the full backtrace, it shouldn't slow
> > down kmemleak considerably. I don't think it's worth logging only the
> > first/last calls to get/put. You'd hope that put is called in reverse
> > order to get.
> > 
> > I think it may be better if this is added as a new allocation pointed to
> > from kmemleak_object rather than increasing this structure since it will
> > be added on a case by case basis. When dumping the leak information, it
> > would also dump the get/put calls, in the order they were called. We
> > could add some simple refcount tracking (++ for get, -- for put) to
> > easily notice any imbalance.
> > 
> > I'm pretty busy next week but happy to review if you have a patch ;).
> 
> I am still thinking about a more generic way for all those
> refcount-based leaks without needing of manual annotation of all those
> places. Today, I had another one,
> 
> unreferenced object 0xe6ff008924f28500 (size 128):
>   comm "qemu-kvm", pid 4835, jiffies 4295141828 (age 6944.120s)
>   hex dump (first 32 bytes):
>     01 00 00 00 6b 6b 6b 6b 00 00 00 00 ad 4e ad de  ....kkkk.....N..
>     ff ff ff ff 6b 6b 6b 6b ff ff ff ff ff ff ff ff  ....kkkk........
>   backtrace:
>     [<000000005ed1a868>] slab_post_alloc_hook+0x74/0x9c
>     [<00000000c65ee7dc>] kmem_cache_alloc_trace+0x2b4/0x3d4
>     [<000000009efa9e6e>] do_eventfd+0x54/0x1ac
>     [<000000001146e724>] __arm64_sys_eventfd2+0x34/0x44
>     [<0000000096fc3a61>] do_el0_svc+0x128/0x1dc
>     [<000000005ae8f980>] el0_sync_handler+0xd0/0x268
>     [<0000000043f2c790>] el0_sync+0x164/0x180
> 
> That is eventfd_ctx_fileget() / eventfd_ctx_put() pairs.

In this case it uses kref_get() to increment the refcount. We could add
a kmemleak_add_trace() which allocates a new array and stores the stack
trace, linked to the original object. Similarly for kref_put().

If we do this for each inc/dec call, I'd leave it off as default and
only enable it explicitly by cmdline argument or
/sys/kerne/debug/kmemleak when needed. In most cases you'd hope there is
no leak, so no point in tracking additional metadata. But if you do hit
a problem, just enable the additional tracking to help with the
debugging.

-- 
Catalin