Re: [PATCH 2/3] lib/string_helpers.c: don't lose precision in string_get_size()

[Vitaly Kuznetsov]

Rasmus Villemoes <linux@xxxxxxxxxxxxxxxxxx> writes:

> So I also played with this over the weekend, and also threw together a
> stupid script to check the output. I see you have more or less the same
> idea I used, namely to combine size and blk_size earlier.
>
> I put some code on github, https://github.com/Villemoes/get_size. All
> versions still fail a very simply case, size=1594323, blk_size=1, for
> which the correct answer is "1.52 MiB", but we get "1.51 MiB".

Yes, the algorithm still has minor glitches (e.g. rounding issues) but
I'd say keeping it simple enough here is worth it.

> But both
> your version and my two attempts seem to have the property that they are
> always at most one ULP (unit in the last place) too low, and never too
> high. ATM, my version 2 fails 70 of the 13598 test cases, while yours
> fail 86 (the former being a strict subset of the latter), so they're
> very similar.
>
> Regardless of which algorithm we go with, I have some cleanups I'd like
> to do, but they're mostly independent and can wait.
>
> Rasmus

-- 
  Vitaly
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/