[PATCH 3.16 001/346] lib/mpi: mpi_read_raw_data(): fix nbits calculation

From: Ben Hutchings
Date: Sun Nov 13 2016 - 21:04:38 EST


3.16.39-rc1 review patch. If anyone has any objections, please let me know.

------------------

From: Nicolai Stange <nicstange@xxxxxxxxx>

commit eef0df6a59537032ab6b708f30b28d9530f8760e upstream.

The number of bits, nbits, is calculated in mpi_read_raw_data() as follows:

nbits = nbytes * 8;

Afterwards, the number of leading zero bits of the first byte get
subtracted:

nbits -= count_leading_zeros(buffer[0]);

However, count_leading_zeros() takes an unsigned long and thus,
the u8 gets promoted to an unsigned long.

Thus, the above doesn't subtract the number of leading zeros in the most
significant nonzero input byte from nbits, but the number of leading
zeros of the most significant nonzero input byte promoted to unsigned long,
i.e. BITS_PER_LONG - 8 too many.

Fix this by subtracting

count_leading_zeros(...) - (BITS_PER_LONG - 8)

from nbits only.

Fixes: e1045992949 ("MPILIB: Provide a function to read raw data into an
MPI")
Signed-off-by: Nicolai Stange <nicstange@xxxxxxxxx>
Signed-off-by: Herbert Xu <herbert@xxxxxxxxxxxxxxxxxxx>
[bwh: Backported to 3.16: adjust context]
Signed-off-by: Ben Hutchings <ben@xxxxxxxxxxxxxxx>
---
lib/mpi/mpicoder.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

--- a/lib/mpi/mpicoder.c
+++ b/lib/mpi/mpicoder.c
@@ -48,7 +48,7 @@ MPI mpi_read_raw_data(const void *xbuffe
return NULL;
}
if (nbytes > 0)
- nbits -= count_leading_zeros(buffer[0]);
+ nbits -= count_leading_zeros(buffer[0]) - (BITS_PER_LONG - 8);
else
nbits = 0;