Re: [PATCH v2 3/3] IB/core: Obtain subnet_prefix from cache in IB devices

[Leon Romanovsky]

On Thu, Jun 03, 2021 at 09:29:32AM +0000, Haakon Bugge wrote:
> 
> 
> > On 3 Jun 2021, at 11:07, Leon Romanovsky <leon@xxxxxxxxxx> wrote:
> > 
> > On Thu, Jun 03, 2021 at 12:20:24PM +0530, Anand Khoje wrote:
> >> ib_query_port() calls device->ops.query_port() to get the port
> >> attributes. The method of querying is device driver specific.
> >> The same function calls device->ops.query_gid() to get the GID and
> >> extract the subnet_prefix (gid_prefix).
> >> 
> >> The GID and subnet_prefix are stored in a cache. But they do not get
> >> read from the cache if the device is an Infiniband device. The
> >> following change takes advantage of the cached subnet_prefix.
> >> Testing with RDBMS has shown a significant improvement in performance
> >> with this change.
> >> 
> >> The function ib_cache_is_initialised() is introduced because
> >> ib_query_port() gets called early in the stage when the cache is not
> >> built while reading port immutable property.
> >> 
> >> In that case, the default GID still gets read from HCA for IB link-
> >> layer devices.
> >> 
> >> Fixes: fad61ad ("IB/core: Add subnet prefix to port info")
> >> Signed-off-by: Anand Khoje <anand.a.khoje@xxxxxxxxxx>
> >> Signed-off-by: Haakon Bugge <haakon.bugge@xxxxxxxxxx>
> >> ---
> >> drivers/infiniband/core/cache.c  | 7 ++++++-
> >> drivers/infiniband/core/device.c | 9 +++++++++
> >> include/rdma/ib_cache.h          | 6 ++++++
> >> include/rdma/ib_verbs.h          | 6 ++++++
> >> 4 files changed, 27 insertions(+), 1 deletion(-)
> > 
> > Can you please help me to understand how cache is updated?
> > 
> > There are a lot of calls to ib_query_port() and I wonder how callers can
> > get new GID after it was changed in already initialized cache.
> 
> The cache is initialized when it is created, just before the bit IB_PORT_CACHE_INITIALIZED is set in flags.
> 
> After commit d58c23c92548 ("IB/core: Only update PKEY and GID caches on respective events"), the GID portion of the cache is updated when a IB_EVENT_GID_CHANGE event is received.
> 
> Before said commit, it was updated on any event.

This part is clear to me, the missing piece is to understand what will
happen if cache and GID are not in sync because of asynchronous nature of
events.

Thanks

> 
> 
> Thxs, Håkon
>