Re: [PATCH 0/3] Remove accidental VLA usage

[Kees Cook]

On Thu, Mar 8, 2018 at 2:12 PM, Rasmus Villemoes
<linux@xxxxxxxxxxxxxxxxxx> wrote:
> On 8 March 2018 at 21:39, Kees Cook <keescook@xxxxxxxxxxxx> wrote:
>> However, this works for me:
>>
>> #define __new_max(t1, t2, max1, max2, x, y)                    \
>>        __builtin_choose_expr(__builtin_constant_p(x) && \
>>                              __builtin_constant_p(y) && \
>>                              __builtin_types_compatible_p(t1, t2),     \
>>                              (t1)(x) > (t2)(y) ? (t1)(x) : (t2)(y),    \
>>                              __max(t1, t2, max1, max2, x, y))
>>
>> #define new_max(x, y) \
>>         __new_max(typeof(x), typeof(y),                 \
>>               __UNIQUE_ID(max1_), __UNIQUE_ID(max2_),   \
>>               x, y)
>
> Yes, that would seem to do the trick.
>
> Thinking out loud: do we really want or need the
> __builtin_types_compatible condition when x and y are compile-time
> constants? I think it would be nice to be able to use max(16,
> sizeof(bla)) without having to cast either the literal or the sizeof.
> Just omitting the type compatibility check might be dangerous, but
> perhaps it could be relaxed to a check that both values are
> representable in their common promoted type. Something like
>
> (type_signed(t1) == type_signed(t2)) || ((t1)x >= 0 && (t2)y >= 0)
>
> should be safe (if the types have same signedness, or the value of
> signed type is positive), though it doesn't allow a few corner cases
> (e.g. short vs. unsigned char is always ok due to promotion to int,
> and also if the signed type is strictly wider than the unsigned type).

I agree, it would be nice. However, I think it'd be better to continue
to depend on max_t() for these kinds of cases though. For example:

char foo[max_t(size_t, 6, sizeof(something))];

Works with the proposed patch.

Also, I think this mismatch would already be triggering warnings, so
we shouldn't have any currently.

-Kees

-- 
Kees Cook
Pixel Security