Re: [PATCH v2 1/1] lib: Optimise hex_dump_to_buffer()

[Andy Shevchenko]

On Wed, Mar 12, 2025 at 07:18:16PM +0000, David Laight wrote:
> On Mon, 10 Mar 2025 11:05:13 +0200
> Andy Shevchenko <andriy.shevchenko@xxxxxxxxxxxxxxx> wrote:
> > On Sat, Mar 08, 2025 at 09:34:21AM +0000, David Laight wrote:

...

> > > -extern int hex_dump_to_buffer(const void *buf, size_t len, int rowsize,
> > > -			      int groupsize, char *linebuf, size_t linebuflen,
> > > -			      bool ascii);
> > > +extern size_t hex_dump_to_buffer(const void *buf, size_t len, size_t rowsize,  
> > > +				 size_t groupsize, char *linebuf,
> > > +				 size_t linebuflen, bool ascii);  
> > 
> > int - > size_t in the returned value is incorrect change.
> > This is explained in the comments to the test cases patch series.
> 
> I don't see you mentioning why.
> The return value is 'the number of bytes that would be output if the buffer
> were large enough' - it is never negative.

True...

> Although given 'a large enough buffer' length is trivially calculable
> it would have been safer to return the actual number of bytes added
> (excluding the '\0').

...but the functions keep the snprintf() semantics, which returns an int.
This makes it more-or-less 1:1 snprintf() substitute in cases where it can
be done in general.

> There were no tests for 'len == 0 && linebuflen == 0', with !ascii the
> existing hex_dump_to_buffer() even manages to return -1.
> (and the function than generates the 'test compare data' is also broken.)

Then you can start with fixes of those?

> Note that libc snprintf() has the same return type as fprintf() which can
> be -1, but any code the looks at is probably broken!
> 
> So an unsigned return type it better.

Maybe, but this will deviate from the prototype and use cases.

-- 
With Best Regards,
Andy Shevchenko