Re: [RFC PATCH rdma-next 08/10] RDMA/rxe: Implement flush execution in responder side

[lizhijian@xxxxxxxxxxx]

On 12/01/2022 21:12, Jason Gunthorpe wrote:
> On Wed, Jan 12, 2022 at 09:50:38AM +0000, lizhijian@xxxxxxxxxxx wrote:
>>
>> On 12/01/2022 04:48, Jason Gunthorpe wrote:
>>> On Tue, Jan 11, 2022 at 05:34:36AM +0000, lizhijian@xxxxxxxxxxx wrote:
>>>
>>>> Yes, that's true. that's because only pmem has ability to persist data.
>>>> So do you mean we don't need to prevent user to create/register a persistent
>>>> access flag to a non-pmem MR? it would be a bit confusing if so.
>>> Since these extensions seem to have a mode that is unrelated to
>>> persistent memory,
>> I can only agree with part of them, since the extensions also say that:
>>
>> oA19-1: Responder shall successfully respond on FLUSH operation only
>> after providing the placement guarantees, as specified in the packet, of
>> preceding memory updates (for example: RDMA WRITE, Atomics and
>> ATOMIC WRITE) towards the memory region.
>>
>> it mentions *shall successfully respond on FLUSH operation only
>> after providing the placement guarantees*. If users request a
>> persistent placement to a non-pmem MR without errors,  from view
>> of the users, they will think of their request had been *successfully responded*
>> that doesn't reflect the true(data was persisted).
> The "placement guarentees" should obviously be variable depending on
> the type of memory being targeted.

> Since these extensions seem to have a mode that is unrelated to
> persistent memory,


Although the SPEC doesn't link extensions to persistent memory directly, but
the *Persistence* is linked to pmem like indicated memory region. See below:

A19.2 G LOSSARY

Global Visibility
   Ensuring the placement of the preceding data accesses in the indicated
   memory region is visible for reading by the responder platform.
Persistence
   Ensuring the placement of the preceding data accesses in the indicated
   memory region is persistent and the data will be preserved across power
   cycle or other failure of the responder platform.

Thanks
Zhijian


>
>> Further more, If we have a checking during the new MR creating/registering,
>> user who registers this MR can know if the target MR supports persistent access flag.
>> Then they can tell this information to the request side so that request side can
>> request a valid placement type later. that is similar behavior with current librpma.
> Then you can't use ATOMIC_WRITE with non-nvdimm memory, which is
> nonsense
>
> Jason