Re: [PATCH v3 4/5] console: Avoid positive return code from unregister_console()

[Andy Shevchenko]

On Tue, Jan 28, 2020 at 06:37:26PM +0900, Sergey Senozhatsky wrote:
> On (20/01/28 11:22), Andy Shevchenko wrote:
> > On Tue, Jan 28, 2020 at 01:43:32PM +0900, Sergey Senozhatsky wrote:
> > > On (20/01/27 13:47), Andy Shevchenko wrote:
> > > [..]
> > > >  	res = _braille_unregister_console(console);
> > > > -	if (res)
> > > > +	if (res < 0)
> > > >  		return res;
> > > > +	if (res > 0)
> > > > +		return 0;
> > > >  
> > > > -	res = 1;
> > > > +	res = -ENODEV;
> > > >  	console_lock();
> > > >  	if (console_drivers == console) {
> > > >  		console_drivers=console->next;
> > > > @@ -2838,6 +2840,9 @@ int unregister_console(struct console *console)
> > > >  	if (!res && (console->flags & CON_EXTENDED))
> > > >  		nr_ext_console_drivers--;
> > > >  
> > > > +	if (res && !(console->flags & CON_ENABLED))
> > > > +		res = 0;
> > > 
> > > Console is not on the console_drivers list. Why does !ENABLED case
> > > require extra handling?
> > 
> > It's mirroring (to some extend) the register_console() abort conditions.
> 
> Could you please explain?
> 
> I see the "newcon->flags & CON_ENABLED" error out path. I'm guessing,
> that the expectation is that this is how we filter out consoles which
> were not matched (there is that "newcon->flags |= CON_ENABLED" several
> lines earlier.) So this looks like the assumption is that consoles don't
> have CON_ENABLED bit set prior to register_console(), as far as I understand.

I put it to cover the case when register_console() fails (since it has no
return code caller is not able to say this anyhow) somebody may call
unregister_console() on it unconditionally (and I guess many do like this).
In such case we shouldn't return an error code.

> Well, look at these
> ...
> drivers/net/netconsole.c:       .flags  = CON_ENABLED,
> drivers/tty/ehv_bytechan.c:     .flags  = CON_PRINTBUFFER | CON_ENABLED,
> drivers/tty/serial/mux.c:	.flags = CON_ENABLED | CON_PRINTBUFFER,
> ...

The code there (I meant register_console() and unregister_console() and
their usage) is quite twisted and probably abused, so, I have definitely
miss something.

-- 
With Best Regards,
Andy Shevchenko