Re: [PATCH 02/16] fpga: add FPGA device framework

[Wu Hao]

On Tue, Jul 25, 2017 at 04:32:10PM -0500, Alan Tull wrote:
> On Sat, Apr 1, 2017 at 7:18 AM, Wu Hao <hao.wu@xxxxxxxxx> wrote:
> 
> Hi Hao,
> 
> > On Fri, Mar 31, 2017 at 12:01:13PM -0700, matthew.gerlach@xxxxxxxxxxxxxxx wrote:
> >> On Fri, 31 Mar 2017, Wu Hao wrote:
> >> >On Fri, Mar 31, 2017 at 08:09:09AM +0200, Greg KH wrote:
> >> >>On Thu, Mar 30, 2017 at 08:08:02PM +0800, Wu Hao wrote:
> >> >>>During FPGA device (e.g PCI-based) discovery, platform devices are
> >> >>>registered for different FPGA function units. But the device node path
> >> >>>isn't quite friendly to applications.
> >> >>>
> >> >>>Consider this case, applications want to access child device's sysfs file
> >> >>>for some information.
> >> >>>
> >> >>>1) Access using bus-based path (e.g PCI)
> >> >>>
> >> >>>  /sys/bus/pci/devices/xxxxx/fpga_func_a.0/sysfs_file
> >> >>>
> >> >>>  From the path, it's clear which PCI device is the parent, but not perfect
> >> >>>  solution for applications. PCI device BDF is not fixed, application may
> >> >>>  need to search all PCI device to find the actual FPGA Device.
> >> >>>
> >> >>>2) Or access using platform device path
> >> >>>
> >> >>>  /sys/bus/platform/devices/fpga_func_a.0/sysfs_file
> >> >>>
> >> >>>  Applications find the actual function by name easily, but no information
> >> >>>  about which fpga device it belongs to. It's quite confusing if multiple
> >> >>>  FPGA devices are in one system.
> >> >>>
> >> >>>'FPGA Device' class is introduced to resolve this problem. Each node under
> >> >>>this class represents a fpga device, which may have one or more child
> >> >>>devices. Applications only need to search under this FPGA Device class
> >> >>>folder to find the child device node it needs.
> >> >>>
> >> >>>For example, for the platform has 2 fpga devices, each fpga device has
> >> >>>3 child devices, the hierarchy looks like this.
> >> >>>
> >> >>>Two nodes are under /sys/class/fpga/:
> >> >>>/sys/class/fpga/fpga.0
> >> >>>/sys/class/fpga/fpga.1
> >> >>>
> >> >>>Each node has 1 function A device and 2 function B devices:
> >> >>>/sys/class/fpga/fpga.0/func_a.0
> >> >>>/sys/class/fpga/fpga.0/func_b.0
> >> >>>/sys/class/fpga/fpga.0/func_b.1
> >> >>>
> >> >>>/sys/class/fpga/fpga.1/func_a.1
> >> >>>/sys/class/fpga/fpga.1/func_b.2
> >> >>>/sys/class/fpga/fpga.1/func_b.3
> >> >>>
> >> >>>This following APIs are provided by FPGA device framework:
> >> >>>* fpga_dev_create
> >> >>>  Create fpga device under the given parent device.
> >> >>>* fpga_dev_destroy
> >> >>>  Destroy fpga device
> >> >>>
> >> >>>The following sysfs files are created:
> >> >>>* /sys/class/fpga/<fpga.x>/name
> >> >>>  Name of the fpga device.
> >> >>
> >> >>How does this interact with the existing "fpga class" that is in the
> >> >>kernel already?
> >> >
> >> >The fpga-dev introduced by this patch, is only a container device, and
> >>
> >> I completely understand the need for a container device.  The fpga-region is
> >> also primarily a container, and in some cases the fpga-region may represent
> >> the entire fpga.  Over time this code may become redundant.
> >
> > Thanks a lot for your review and comments.
> >
> > I feel that the fpga-region implies that it supports reconfiguration,
> 
> On Arria10, we create base fpga region which does not support full
> reconfiguration.  It corresponds to the whole FPGA area, which was
> loaded with a static FPGA image in the bootloader.  The partial
> reconfiguration regions are children of the base FPGA region.  Any
> devices in the FPGA are child devices of either the base region or a
> region which is a child of it.
> 
> > but
> > in our cases, the Intel FPGA device, doesn't have base fpga-region for
> > full reconfiguration, but many accelerators with partial reconfiguration
> > support. A fpga-region brings together everything needed for the
> > reconfiguration, and a fpga-dev is trying to brings everything on a FPGA
> > device together, including fpga-region/bridge/manager, access different
> > accelerators and other function units.
> >
> > I think it's not mandatory to use fpga-dev, as fpga-dev is just trying to
> > provide one more option here for some complex hardware.
> 
> Now that you've put out v2 which uses fpga-regions, do you still need
> fpga-dev class?

Hi Alan

Thanks for the comments.

In v2, I have updated the driver organization section in intel-fpga.txt[1].
The fpga-regions/bridges/manager are created as children of FME module, as
the partial reconfiguration function is only a sub feature of FME module.

If switch to fpga-region as container device, it may not be easy for user
space applications to know which one represents a FPGA device and which one
represents a reconfigurable region as all have the similar name 'regionx'
in the same sysfs folder. Please consider this case, if we have 5 fpga
devices on one system and each fpga device has multiple PR regions (e.g 20+).
Then user space applications need to search all regions to locate the ones
represent the FPGA device, even we add some attributes to it.

[1]http://marc.info/?l=linux-fpga&m=149844234509825&w=2

Thanks
Hao

> 
> Alan
> 
> >
> > Thanks
> > Hao
> >
> >> >drivers could register different functions under it. Per my understanding,
> >> >the existing "fpga class", including fpga-region, fpga-bridge and
> >> >fpga-manager, is used to provide reconfiguration function for FPGA. So
> >> >driver can create child node using this existing "fpga class" to provide
> >> >FPGA reconfiguration function, and more nodes under this container for
> >> >different functions for given FPGA device.
> >> >
> >> >For Intel FPGA device, partial reconfiguration is only one function of
> >> >Intel FPGA Management Engine (FME). FME driver creates fpga_manager under
> >> >below path for partial reconfiguration, and other interfaces for more
> >> >functions, e.g power management, virtualization support and etc.
> >> >
> >> >/sys/class/fpga/<fpga.x>/<intel-fpga-fme.x>/fpga_manager
> >> >
> >> >Thanks
> >> >Hao
> >> >
> >> >>
> >> >>thanks,
> >> >>
> >> >>greg k-h
> >> >--
> >> >To unsubscribe from this list: send the line "unsubscribe linux-fpga" in
> >> >the body of a message to majordomo@xxxxxxxxxxxxxxx
> >> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >> >