RE: [PATCH v5 0/6] Add hardware prefetch control driver for A64FX and x86

[tarumizu.kohei@xxxxxxxxxxx]

Hi Greg,

> That's not ok.  Linux is a "general purpose" operating system and needs to
> work well for all applications.  Doing application-specific-tuning based on the
> specific hardware like this is a nightmare for users,

Hardware prefetch behavior is enabled by default in x86 and A64FX.
Many applications can perform well without changing the register
setting. Use this feature for some applications that want to be
improved performance.

In particular, A64FX's hardware prefetch behavior is used for HPC
applications. The user running HPC applications needs to improve
performance as much as possible. This feature is useful for such
users. Therefore, some A64FX machines have their own drivers that
control hardware prefetch behavior. It is built into the software
products for A64FX and cannot be used without purchase.

I want to make this feature available to people who want to improve
performance without purchase the product. This is limited in use and
depends on the characteristics of the application. Isn't this match
with "general purpose"?

> and will be for you as you
> will now have to support this specific model to work correctly on all future
> kernel releases for the next 20+ years.
> Are you willing to do that?

Rather than relying on a specific model of this API, I want to make it
generally available. However, it may not be so now. I am willing to
support this if I could make it a community-approved interface.

> Then perhaps it isn't anything that they should try out :)
> 
> Shouldn't the kernel know how the application works (based on the resources
> it asks for) and tune itself based on that automatically?
> 
> If not, how is a user supposed to know how to do this?

It is useful for users if it can be done automatically by the kernel.
I will consider if there is anything I can do using statistical
information.

> What is "1024" here?  Where is any of this documented?

This parameter means the difference in bytes between the memory
address the program is currently accessing and the memory address
accessed by the hardware prefetch. My document in
sysfs-devices-system-cpu does not specify what the distance means, so
I will add it.

For reference, the hardware prefetch details are described below.

"https://github.com/fujitsu/A64FX/tree/master/doc";
    A64FX_Microarchitecture_Manual_en_1.7.pdf

> And why these
> specific sysfs files and not others?

I wanted to show an example of changing only the hardware prefetch
distance. There is no special reason not to specify with other sysfs
files.

> If you have no such user today, nor a library, how do you know any of this works
> well?

The prefetch control function included in the software product for
A64FX does the similar operation, and it works well.

> And again, how will you support this going forward?
> Or is this specific api only going to be for one specific piece of hardware and
> never any future ones?

In order to make the interface widely usable in the future, I will
consider a different specification from the current one. For example
control groups which Linus proposaled is one of them.

> So you haven't tested this on any real applications?  We need real users before
> being able to add new apis.  Otherwise we can just remove the apis :)

At least, some A64FX users use this behavior. However, currently, I
don't have which applications and how much performance will be
improved. I will try to get the application actually used and confirm
that it is effective.

> Kernel programming for a general purpose operating system is hard, but it is
> possible :)

I will try to do kernel programming for a general purpose operating
system.