Re: [PATCH 02/16] fpga: add FPGA device framework

[Alan Tull]

On Wed, Jul 26, 2017 at 4:50 AM, Wu Hao <hao.wu@xxxxxxxxx> wrote:
> On Tue, Jul 25, 2017 at 04:32:10PM -0500, Alan Tull wrote:
>> On Sat, Apr 1, 2017 at 7:18 AM, Wu Hao <hao.wu@xxxxxxxxx> wrote:
>>
>> Hi Hao,
>>
>> > On Fri, Mar 31, 2017 at 12:01:13PM -0700, matthew.gerlach@xxxxxxxxxxxxxxx wrote:
>> >> On Fri, 31 Mar 2017, Wu Hao wrote:
>> >> >On Fri, Mar 31, 2017 at 08:09:09AM +0200, Greg KH wrote:
>> >> >>On Thu, Mar 30, 2017 at 08:08:02PM +0800, Wu Hao wrote:
>> >> >>>During FPGA device (e.g PCI-based) discovery, platform devices are
>> >> >>>registered for different FPGA function units. But the device node path
>> >> >>>isn't quite friendly to applications.
>> >> >>>
>> >> >>>Consider this case, applications want to access child device's sysfs file
>> >> >>>for some information.
>> >> >>>
>> >> >>>1) Access using bus-based path (e.g PCI)
>> >> >>>
>> >> >>>  /sys/bus/pci/devices/xxxxx/fpga_func_a.0/sysfs_file
>> >> >>>
>> >> >>>  From the path, it's clear which PCI device is the parent, but not perfect
>> >> >>>  solution for applications. PCI device BDF is not fixed, application may
>> >> >>>  need to search all PCI device to find the actual FPGA Device.
>> >> >>>
>> >> >>>2) Or access using platform device path
>> >> >>>
>> >> >>>  /sys/bus/platform/devices/fpga_func_a.0/sysfs_file
>> >> >>>
>> >> >>>  Applications find the actual function by name easily, but no information
>> >> >>>  about which fpga device it belongs to. It's quite confusing if multiple
>> >> >>>  FPGA devices are in one system.
>> >> >>>
>> >> >>>'FPGA Device' class is introduced to resolve this problem. Each node under
>> >> >>>this class represents a fpga device, which may have one or more child
>> >> >>>devices. Applications only need to search under this FPGA Device class
>> >> >>>folder to find the child device node it needs.
>> >> >>>
>> >> >>>For example, for the platform has 2 fpga devices, each fpga device has
>> >> >>>3 child devices, the hierarchy looks like this.
>> >> >>>
>> >> >>>Two nodes are under /sys/class/fpga/:
>> >> >>>/sys/class/fpga/fpga.0
>> >> >>>/sys/class/fpga/fpga.1
>> >> >>>
>> >> >>>Each node has 1 function A device and 2 function B devices:
>> >> >>>/sys/class/fpga/fpga.0/func_a.0
>> >> >>>/sys/class/fpga/fpga.0/func_b.0
>> >> >>>/sys/class/fpga/fpga.0/func_b.1
>> >> >>>
>> >> >>>/sys/class/fpga/fpga.1/func_a.1
>> >> >>>/sys/class/fpga/fpga.1/func_b.2
>> >> >>>/sys/class/fpga/fpga.1/func_b.3
>> >> >>>
>> >> >>>This following APIs are provided by FPGA device framework:
>> >> >>>* fpga_dev_create
>> >> >>>  Create fpga device under the given parent device.
>> >> >>>* fpga_dev_destroy
>> >> >>>  Destroy fpga device
>> >> >>>
>> >> >>>The following sysfs files are created:
>> >> >>>* /sys/class/fpga/<fpga.x>/name
>> >> >>>  Name of the fpga device.
>> >> >>
>> >> >>How does this interact with the existing "fpga class" that is in the
>> >> >>kernel already?
>> >> >
>> >> >The fpga-dev introduced by this patch, is only a container device, and
>> >>
>> >> I completely understand the need for a container device.  The fpga-region is
>> >> also primarily a container, and in some cases the fpga-region may represent
>> >> the entire fpga.  Over time this code may become redundant.
>> >
>> > Thanks a lot for your review and comments.
>> >
>> > I feel that the fpga-region implies that it supports reconfiguration,
>>
>> On Arria10, we create base fpga region which does not support full
>> reconfiguration.  It corresponds to the whole FPGA area, which was
>> loaded with a static FPGA image in the bootloader.  The partial
>> reconfiguration regions are children of the base FPGA region.  Any
>> devices in the FPGA are child devices of either the base region or a
>> region which is a child of it.
>>
>> > but
>> > in our cases, the Intel FPGA device, doesn't have base fpga-region for
>> > full reconfiguration, but many accelerators with partial reconfiguration
>> > support. A fpga-region brings together everything needed for the
>> > reconfiguration, and a fpga-dev is trying to brings everything on a FPGA
>> > device together, including fpga-region/bridge/manager, access different
>> > accelerators and other function units.
>> >
>> > I think it's not mandatory to use fpga-dev, as fpga-dev is just trying to
>> > provide one more option here for some complex hardware.
>>
>> Now that you've put out v2 which uses fpga-regions, do you still need
>> fpga-dev class?
>
> Hi Alan
>
> Thanks for the comments.
>
> In v2, I have updated the driver organization section in intel-fpga.txt[1].


I've read your v2 of this document.  It's changed as you've said, but
not that much.  I'm just continuing the previous conversation.  I'll
add further comments on the v2 version.

Alan

> The fpga-regions/bridges/manager are created as children of FME module, as
> the partial reconfiguration function is only a sub feature of FME module.
>
> If switch to fpga-region as container device, it may not be easy for user
> space applications to know which one represents a FPGA device and which one
> represents a reconfigurable region as all have the similar name 'regionx'
> in the same sysfs folder. Please consider this case, if we have 5 fpga
> devices on one system and each fpga device has multiple PR regions (e.g 20+).
> Then user space applications need to search all regions to locate the ones
> represent the FPGA device, even we add some attributes to it.
>
> [1]http://marc.info/?l=linux-fpga&m=149844234509825&w=2
>
> Thanks
> Hao
>
>>
>> Alan
>>
>> >
>> > Thanks
>> > Hao
>> >
>> >> >drivers could register different functions under it. Per my understanding,
>> >> >the existing "fpga class", including fpga-region, fpga-bridge and
>> >> >fpga-manager, is used to provide reconfiguration function for FPGA. So
>> >> >driver can create child node using this existing "fpga class" to provide
>> >> >FPGA reconfiguration function, and more nodes under this container for
>> >> >different functions for given FPGA device.
>> >> >
>> >> >For Intel FPGA device, partial reconfiguration is only one function of
>> >> >Intel FPGA Management Engine (FME). FME driver creates fpga_manager under
>> >> >below path for partial reconfiguration, and other interfaces for more
>> >> >functions, e.g power management, virtualization support and etc.
>> >> >
>> >> >/sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/fpga_manager
>> >> >
>> >> >Thanks
>> >> >Hao
>> >> >
>> >> >>
>> >> >>thanks,
>> >> >>
>> >> >>greg k-h
>> >> >--
>> >> >To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
>> >> >the body of a message to majordomo@xxxxxxxxxxxxxxx
>> >> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> >> >