Re: [RFC] Adding A64FX hardware prefetch sysfs interface

[James Morse]

Hello!

(CC: +Catalin and Will)

On 07/06/2021 09:11, Borislav Petkov wrote:
> (not trimming the mail so that ARM folks can see the whole thing)
> 
> On Mon, Jun 07, 2021 at 01:39:21AM +0000, tarumizu.kohei@xxxxxxxxxxx wrote:
>> Hello
>>
>> I'm Kohei Tarumizu from Fujitsu Limited. 
>>
>> Fujitsu A64FX processor implements a vendor specific function, the HPC extensions[1].
>> A64FX has some registers for HPC extensions.
>> We would like to use the register IMP_PF_STREAM_DETECT_CTRL_EL0 for tuning the hardware prefetch, but it's not accessible from userspace.
>> We are considering to implement a kernel common interface via sysfs as a way to control IMP_PF_STREAM_DETECT_CTRL_EL0 from userspace.


>> FYI, A64FX also has registers (e.g. IMP_PF_INJECTION_*) to control the behavior of the hardware prefetch from the software using "HPC tag address override", but this time we don't considered.
>>
>> [1]https://github.com/fujitsu/A64FX/tree/master/doc/
>>    A64FX_Specification_HPC_Extension_v1_EN.pdf

While this is initially about sysfs, don't you need the 'HPC tag address override' to be
enabled for this to be useful? I don't think that feature can be managed by a driver:

'HPC tag address override' changes the top byte of all user-space pointers from being
ignored (as they have been since day-1 on arm64) to having implications for the hardware.
If I've read the document correctly this affects the prefetch mode and where in the L1/L2
such accesses will be allocated.

This would impact user-space that is using the top-byte for their own purposes.
For example hwasan uses this field as a tag it allocates itself:
https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html
Enabling 'HPC tag address override' for all user-space is going to have weird performance
effects.

To make this work, I think you'd need a per-process opt-in, and __switch_to() would need
to toggle your IMP_FJ_TAG_ADDRESS_CTRL_EL1.TBOx bits. Because its an
implementation-defined feature, but the controls can't be confined to a driver, I don't
think enabling 'HPC tag address override' is viable.

Is the sysfs information useful without it?


Thanks,

James