Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez <mcgrof@xxxxxxxx> wrote:
> On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
>> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel <arend@xxxxxxxxxxxx> wrote:
>> > On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
>> >>
>> >> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
>> >>>
>> >>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
>> >>>>
>> >>>> On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>> >>>>>
>> >>>>> On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
>> >>>>>>
>> >>>>>> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
>> >>>>>> <arend@xxxxxxxxxxxx> wrote:
>> >>>>>>>
>> >>>>>>> Does this mean a built-in driver can not get firmware from initramfs
>> >>>>>>> or
>> >>>>>>> built in the kernel early. Seems a bit too aggressive. The problem
>> >>>>>>> stated in
>> >>>>>>> this thread is when the firmware is not on initramfs but only on the
>> >>>>>>> rootfs.
>> >>>>>>
>> >>>>>>
>> >>>>>> Yes, strictly speaking, user mode request can't be handled with defer
>> >>>>>> probe
>> >>>>>> during booting because we don't know how the user helper handles the
>> >>>>>> request,
>> >>>>>
>> >>>>>
>> >>>>> FWIW I have a strategy in mind to help us compartamentalize the user
>> >>>>> mode
>> >>>>> helper only to the dell-rbu driver, and as such phase out that code
>> >>>>> eventually
>> >>>>> completely. Its part of the goals I have with the extensible firmware
>> >>>>> API I've
>> >>>>> been proposing.
>> >>>>>
>> >>>>>> that said even checking if the firmware exists in current path doesn't
>> >>>>>> make sense for user mode request.
>> >>>>>>
>> >>>>>> So the patch should have used defer proble for direct load only
>> >>>>>> during booting.
>> >>>>>
>> >>>>>
>> >>>>> What exact guarantees would we be giving to callers if they follow up
>> >>>>> on probe
>> >>>>> with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
>> >>>>> / probe
>> >>>>> (note that unless you're using async probe since we batch both so it
>> >>>>> doesn't really
>> >>>>> matter where you place your code) all together and then for the few
>> >>>>> remaining
>> >>>>> stragglers understand the requirements and provide an interface that
>> >>>>> lets them
>> >>>>> claim their requirements and try to meets them.
>> >>>>>
>> >>>>> A grammatical hunt for drivers who call fw API on init / probe can be
>> >>>>> completed, although I know the hunt needs a bit more fine tuning it
>> >>>>> surely can
>> >>>>> be completed. If we don't have many callers the compexity added for
>> >>>>> only a
>> >>>>> few callers with rather loose criteria seems rather unnecessary,
>> >>>>> specially if
>> >>>>> we can change the drivers and make these driver sthe exception rather
>> >>>>> than
>> >>>>> a norm.
>> >>>>>
>> >>>>> Then as for drivers *needing* the fw at probe why not have a proper
>> >>>>> interface
>> >>>>> that does guarantee they get the requirements they ask for first ? For
>> >>>>> instance
>> >>>>> a new probe type specified by the driver could enable the core to wait
>> >>>>> for say
>> >>>>> an event and then tirgger a probe, kind of how we ended up defining
>> >>>>> the async
>> >>>>> probe type preference:
>> >>>>>
>> >>>>> static struct some_bus_driver some_driver = {
>> >>>>>          .probe = some_probe,
>> >>>>>          .id_table = some_id,
>> >>>>>          .driver = {
>> >>>>>                  .name = DEVICE_NAME,
>> >>>>>                  .pm = &some_pm_ops,
>> >>>>>                  .probe_type = PROBE_PREFER_POST_FOO,
>> >>>>>          },
>> >>>>> };
>> >>>>>
>> >>>>> Then we just don't try just hoping for completion but rather can do
>> >>>>> something
>> >>>>> about the criteria passed.
>> >>>
>> >>>
>> >>> So should the probe type indicate some event or should it just
>> >>> indicate what the driver needs, ie. .probe_type =
>> >>> PROBE_TYPE_NEED_FW.
>> >>
>> >>
>> >> Right so this is an open question. I suggested something like the above
>> >> since the deferred probe documentation on drivers/base/dd.c states:
>> >>
>> >>   * Sometimes driver probe order matters, but the kernel doesn't always
>> >> have
>> >>   * dependency information
>> >>
>> >> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
>> >> kernel where some work can be done to not only list the dependency
>> >> the information from the driver but also we know we can get it from
>> >> the kernel. In this case I do believe we could not only express the
>> >> requirement but also wait for it in the kernel. Before we do that
>> >> though I think it'd be good to do a grammar hunt to determine exactly
>> >> how popular all this fw on probe needed really is.
>> >
>> >
>> > Ok. So some background why we need it in brcm80211 drivers. So as a wireless
>> > network device driver the answer we got when asking for an event to load
>> > firware is upon IF_UP for a registered net device. Because we try to do
>> > things smart we query the firmware running on the device for capabilities
>> > before we can register the net device hence we request the firmware during
>> > probe. This may be specific to wireless drivers (Intel has same approach if
>> > not mistaken) but I suspect there may be more.
>>
>> We have the same issue with input devices: before we can register one
>> we need to set their capabilities and to know their capabilities we
>> quite often need to load their firmware/config and query the device.
>
> Should Arend's driver use async probe then?

What has async probe have to do with anything? We will still be
waiting for async probes to finish before we mount rootfs so it will
not change absolutely anything.

>
> IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
> preemptively hacky. Sadly I can't think of clear and clever way for the kernel
> to know when firmware will be ready either...  Would userspace know? Should the
> kernel learn this from userspace ?

Yes. Given only userspace knows when firmware is available (I could
have it on a separate device and mount it at some time). So maybe
userpsace should simply try and scan busses for unbound devices and
tell them to re-probe when it decides that firmware is finally
available.

Thanks.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/