Re: [PATCH v8 09/14] iio: afe: rescale: fix accuracy for small fractional scales

[Peter Rosin]

On 2021-09-02 04:27, Liam Beguin wrote:
> On Mon, Aug 30, 2021 at 04:30:54PM +0200, Peter Rosin wrote:
>>
>> Having said that, if 'scale' and 'offset' indeed are static, then the gcd
>> cost can be mitigated by caching the result. Exact rescaling is always
>> nice...
>>
>> If IIO_VAL_INT overflows while rescaling, we are SOL whichever way we turn,
>> so ignore doing anything about that.
> 
> I was thinking of using check_mul_overflow() to do something about the
> overflow, but I'm happy to leave it out for now.

My mistake, you are right. A sufficiently large denominator can of course
be used to dodge overflow in the numerator by sacrificing some accuracy
even if our maximum scale is still limited to 32 bits.

I apparently didn't have my brains about when I wrote the above...

Cheers,
Peter