Re: [regression] oops on heavy compilations ("kernel BUG at mm/zswap.c:1005!" and "Oops: invalid opcode: 0000")

[Nhat Pham]

On Tue, Sep 3, 2024 at 10:49 AM Yosry Ahmed <yosryahmed@xxxxxxxxxx> wrote:
>
> On Mon, Sep 2, 2024 at 1:58 AM Piotr Oniszczuk
> <piotr.oniszczuk@xxxxxxxxx> wrote:
> >
> >
> >
> > > Wiadomość napisana przez Yosry Ahmed <yosryahmed@xxxxxxxxxx> w dniu 31.08.2024, o godz. 19:23:
> > >
> > > On Sat, Aug 31, 2024 at 2:41 AM Piotr Oniszczuk
> > > <piotr.oniszczuk@xxxxxxxxx> wrote:
> > >>
> > >>
> > >>
> > >>> Wiadomość napisana przez Yosry Ahmed <yosryahmed@xxxxxxxxxx> w dniu 29.08.2024, o godz. 23:54:
> > >>>
> > >>> I also noticed that you are using z3fold as the zpool. Is the problem
> > >>> reproducible with zsmalloc? I wouldn't be surprised if there's a
> > >>> z3fold bug somewhere.
> > >>>
> > >>
> > >> Hmm - yesterday i recompiled 6.9.12 with zsmalloc and …. after 16h of continuous tests I can’t reproduce issue.
> > >> With zsmalloc 6.9.12 looks to me like stable.
> > >
> > > Interesting, and a little bit what I hoped for tbh.
> >
> > :-)
> >
> > I tested mainline 6.10.7 with 26h test and also it is stable with zsmalloc
> >
> > >
> > >>
> > >> With this - what will be your advice to move forward?
> > >
> > > Well, it's possible that some zswap change was not fully compatible
> > > with z3fold, or surfaced a dormant bug in z3fold. Either way, my
> > > recommendation is to use zsmalloc.
> > > I have been trying to deprecate
> >
> > IMHO - isn’t bug in this report + difficulties to reproduce->fix enough to depreciate z3fold?
>
> I would say this bug report is yet another reason why we should deprecate it.

+100000.

This is precisely why I was asking which allocator was being used
here. We have also accidentally selected z3fold internally a couple
times in the past, which had bitten us as well.

>
> >
> > > z3fold, and honestly you are the only person I have seen use z3fold in
> > > a while -- which is probably why no one else reported such a problem.
> >
> > Well - in fact this is ArchLinux - not me.
> > I’m using Arch and kernel in builder machine with ArchLinux config + packaging
>
> According to [1], zsmalloc should be the default allocator for zswap
> on ArchLinux. Anyway, I initially thought that no one was using z3fold
> and it was bitrot, but apparently some people are using it and it's
> actively harming them.
>
> [1]https://wiki.archlinux.org/title/Zswap
>
> >
> > >
> >
> > I see benefits already: on very memory demanding qtwebkit compile:
> > z3fold: swap frequently gets 6..8G from 16G available
> > zsmalloc: can’t see more than 1..2G

Exactly :) zsmalloc is better than z3fold in a lot of workloads that I
have observed.

> >
> > > doubt that you (or anyone) wants to spend time debugging a z3fold
> > > problem :)
> >
> > lets depreciate it!
>
> I tried deprecating it before [2] and performed some analysis [3], but
> there was some.. resistance. Maybe I will try again and use this bug
> report as yet another argument for deprecating z3fold :)
>
> [2] https://lore.kernel.org/linux-mm/20240112193103.3798287-1-yosryahmed@xxxxxxxxxx/
> [3] https://lore.kernel.org/lkml/CAJD7tkbRF6od-2x_L8-A1QL3=2Ww13sCj4S3i4bNndqF+3+_Vg@xxxxxxxxxxxxxx/

I don't wanna sound like a broken record. But this has been the nth
time we need to spend extra engineering time and effort unnecessarily
because we have not deprecated z3fold.

If you need more datapoint - here's our last conversation where z3fold
was a problem:

https://lore.kernel.org/lkml/CAKEwX=Mo+EaaxBYcLMTHYADB4WhqC3QmWV3WQ0h2KM491FRuQA@xxxxxxxxxxxxxx/