Re: [PATCH] checkpatch: improve handling of email comments

[Dwaipayan Ray]

On Sat, Oct 31, 2020 at 9:44 AM Joe Perches <joe@xxxxxxxxxxx> wrote:
>
> On Fri, 2020-10-30 at 12:58 +0100, Lukas Bulwahn wrote:
> >
> > On Fri, 30 Oct 2020, Joe Perches wrote:
> >
> > > On Fri, 2020-10-30 at 14:37 +0530, Dwaipayan Ray wrote:
> > > > checkpatch has limited support for parsing email comments. It only
> > > > support single name comments or single after address comments.
> > > > Whereas, RFC 5322 specifies that comments can be inserted in
> > > > between any tokens of the email fields.
> > > >
> > > > Improve comment parsing mechanism in checkpatch.
> > > >
> > > > What is handled now:
> > > >
> > > > - Multiple name/address comments
> > > > - Comments anywhere in between name/address
> > > > - Nested comments like (John (Doe))
> > > >
> > > > A brief analysis of checkpatch output on v5.0..v5.7 showed that
> > > > after these modifications, the number of BAD_SIGN_OFF warnings
> > > > came down from 2944 to 1424, and FROM_SIGN_OFF_MISMATCH came
> > > > down from 2366 to 2330.
> > > >
> > > > So, a total of 1556 false positives were resolved in total.
> > >
> > > A mere reduction in messages emitted isn't necessarily good.
> > >
> >
> > Agree. That is why I also went through the list of those warnings.
> >
> > I could not spot any obvious true positive among the reduced ones.
> >
> >
> > > Please send me privately a complete list of these nominally
> > > false positive messages that are no longer emitted.
> > >
> > > I believe one of the relatively common incorrect messages is
> > > for the cc: <stable@xxxxxxxxxxxxxxx> where a version number is
> > > continued on the same line after a #.
> > >
> > > CC: stable@xxxxxxxxxxxxxxx # for versions x.y.z and above
> > >
> >
> > That was one,
>
> It's not just one, it's ~90% of the list that Dwaipayan sent me.
>
> $ wc -l mismatches
> 831 mismatches
>
> $ grep -v -i stable mismatches | wc -l
> 98
>
> > another common pattern was just quotes put inconsistently at
> > different places.
>
> Yes, there are some defects there.
> But there are also now false negatives.
>
> For instance, this is not appropriate to ignore:
>
> WARNING:BAD_SIGN_OFF: email address 'jacek.anaszewski@xxxxxxxxx, linux-leds@xxxxxxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, dmurphy@xxxxxx' might be better as 'jacek.anaszewski@xxxxxxxxx,linux-leds@xxxxxxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, dmurphy@xxxxxx'
>
> From the file that Dwaipayan sent me, all the rest not including the
> stable variants, which IMO should be handled separately, are below.
>
> Of these 98 in total, 60+% are unicode which IMO should always be quoted
> and most are doubled with BAD_SIGN_OFF doubling FROM_SIGN_OFF_MISMATCH
> (and I don't quite understand why it's "From:/" then "Signed-off-by:"
>
> $ grep -v -i stable dwai | sort | uniq -c | sort -rn
>      31 WARNING:BAD_SIGN_OFF: email address '周琰杰 (Zhou Yanjie) <zhouyanjie@xxxxxxxxxxxxxx>' might be better as '"周琰杰"(Zhou Yanjie) <zhouyanjie@xxxxxxxxxxxxxx>'
>      30 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "周琰杰"(Zhou Yanjie) <zhouyanjie@xxxxxxxxxxxxxx>' != 'Signed-off-by: 周琰杰 (Zhou Yanjie) <zhouyanjie@xxxxxxxxxxxxxx>'
>
> These 29 in total would be better stripping any bits in parentheses from
> the name portion only when _not_ inside quotes.
>
>      20 WARNING:BAD_SIGN_OFF: email address 'Thomas Hellström (VMware) <thomas_os@xxxxxxxxxxxx>' might be better as '"Thomas Hellström"(VMware) <thomas_os@xxxxxxxxxxxx>'
>       5 WARNING:BAD_SIGN_OFF: email address 'H. Peter Anvin (Intel) <hpa@xxxxxxxxx>' might be better as '"H. Peter Anvin"(Intel) <hpa@xxxxxxxxx>'
>
>
>       1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Thomas Hellström"(VMware) <thomas_os@xxxxxxxxxxxx>' != 'Signed-off-by: Thomas Hellström (VMware) <thomas_os@xxxxxxxxxxxx>'
>
>       1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Srivatsa S. Bhat"(VMware) <srivatsa@xxxxxxxxxxxxx>' != 'Signed-off-by: Srivatsa S. Bhat (VMware) <srivatsa@xxxxxxxxxxxxx>'
>       1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: JanNieuwenhuizen(janneke) <janneke@xxxxxxx>' != 'Signed-off-by: Jan Nieuwenhuizen <janneke@xxxxxxx>'
>       1 WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email name mismatch: 'From: "Frédéric Pierret"(fepitre) <frederic.pierret@xxxxxxxxxxxx>' != 'Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@xxxxxxxxxxxx>'
>
> So these 8 others are ones where quotes are either oddly placed
> or perhaps should always exist and the comment in parentheses
> is suggested poorly.  7 of these should be fixed and one should
> still be reported.
>
>       1 WARNING:BAD_SIGN_OFF: email address '"Thomas Hellström (VMware)" <thomas_os@xxxxxxxxxxxx>' might be better as '"Thomas Hellström"(VMware) <thomas_os@xxxxxxxxxxxx>'
>       1 WARNING:BAD_SIGN_OFF: email address 'Srivatsa S. Bhat (VMware) <srivatsa@xxxxxxxxxxxxx>' might be better as '"Srivatsa S. Bhat"(VMware) <srivatsa@xxxxxxxxxxxxx>'
>       1 WARNING:BAD_SIGN_OFF: email address '"Rantala, Tommi T. (Nokia - FI/Espoo)" <tommi.t.rantala@xxxxxxxxx>' might be better as '"Rantala, Tommi T."(Nokia - FI/Espoo) <tommi.t.rantala@xxxxxxxxx>'
>       1 WARNING:BAD_SIGN_OFF: email address '"Kai Mäkisara (Kolumbus)" <kai.makisara@xxxxxxxxxxx>' might be better as '"Kai Mäkisara"(Kolumbus) <kai.makisara@xxxxxxxxxxx>'
>       1 WARNING:BAD_SIGN_OFF: email address 'jacek.anaszewski@xxxxxxxxx, linux-leds@xxxxxxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, dmurphy@xxxxxx' might be better as 'jacek.anaszewski@xxxxxxxxx,linux-leds@xxxxxxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, dmurphy@xxxxxx'
>       1 WARNING:BAD_SIGN_OFF: email address 'Frédéric Pierret (fepitre) <frederic.pierret@xxxxxxxxxxxx>' might be better as '"Frédéric Pierret"(fepitre) <frederic.pierret@xxxxxxxxxxxx>'
>       1 WARNING:BAD_SIGN_OFF: email address 'David.Laight@xxxxxxxxxx (big endian system concerns)' might be better as 'David.Laight@xxxxxxxxxx(big endian system concerns)'
>       1 WARNING:BAD_SIGN_OFF: email address 'apenwarr@xxxxxxxxxx (Avery Pennarun)' might be better as 'apenwarr@xxxxxxxxxx(Avery Pennarun)'
>
>
Hi,
Thanks for the review.

So I get that the parentheses from within quotes should not
be extracted. I will do that.

But for the names which should be quoted, I think the errors appeared
because of a parsing bug. There is no separate mechanism
to distinguish quoted and unquoted names currently.

Names which have must quote characters without any comments are
not warned about right now:

D. Ray <dwaipayanray1@xxxxxxxxx> doesn't throw any warning, while
D. Ray (Dwai) <dwaipayanray1@xxxxxxxxx> does.

Do you think this should be dealt separately from this patch?
Perhaps as another warning?

Thanks,
Dwaipayan.