Re: [PATCH 01/12] extarray: define helpers for arrays defined in linker scripts

[Vegard Nossum]

On 10/17/2016 01:45 PM, Peter Zijlstra wrote:
> On Mon, Oct 17, 2016 at 01:27:08PM +0200, Vegard Nossum wrote:
> > On 10/17/2016 11:09 AM, Peter Zijlstra wrote:
> > > On Mon, Oct 17, 2016 at 11:01:13AM +0200, Jiri Slaby wrote:
> > > > On the top of that, it's incorrect C according to the standard.
> > >
> > > According to the standard non of the kernel has any chance in hell of
> > > working, so don't pretend you care about that :-)
> >
> > I think that's a bit of a false dilemma. It's obviously true that kernel
> > code does not conform to the standards, but that doesn't mean it's not
> > something we should strive towards or care about in general. It helps
> > static analysis tools, compiler diversity, etc.
>
> Sure, but this, two separately allocated objects their address should
> not be compared and therefore... stuff is explicitly relied upon by the
> kernel in many places.
>
> We have workarounds in various places, and this patch adds yet another
> instance of it.
>
> The workaround is simply confusing the compiler enough to have it not do
> the 'optimization'. But we very much still rely on this 'undefined'
> behaviour.
>
> I think it makes more sense to explicitly allow it than to obfuscate our
> code and run the risk a future compiler will see through our tricks.

Actually, I think we're all a bit wrong.

It's not comparing the pointers that's undefined behavior, that was my
bad trying to oversimplify the issue.

Of course, comparing arbitrary (valid) pointers with each other is not
undefined behavior. The undefined behavior is technically doing iter++
past the end of the array, that is "creating" a pointer that points
outside an array.

What gcc does wrong is that it sees us iterating over one array and the
optimizer concludes that the iterator can never point to the second
array. I'd argue there's no real undefined behaviour happening here.
Thus, the code _is_ correct and valid C as it stands, it just doesn't do
what you would expect intuitively.

However, from the linker script's point of view there is no difference
between one big array and two consecutive arrays of the same type, and
the fact that the compiler doesn't know the memory layout -- although
we do.

When we call OPTIMIZER_HIDE_VAR() we're not really "confusing" the
compiler, it's more like calling a function defined in a different file
(therefore the compiler can't see into it) that returns a pointer which
we _know_ is valid because it still points to (the end of) the array.

If we obtain a pointer somehow (be it from a hardware register or from
the stack of a userspace process), the compiler must assume that it's a
valid pointer, right? The only reason it didn't do that for the arrays
was because we had declared the two arrays as separate variables so it
"knew" it was the case that the iterator initialised to point to one
array could never point to the second array.

Anyway, IANALL.


Vegard