Re: Allow data races on some read/write operations

From: Ralf Jung
Date: Wed Mar 05 2025 - 17:01:49 EST

Hi,

On 05.03.25 19:38, Andreas Hindborg wrote:
"Ralf Jung" <post@xxxxxxxx> writes:

Hi,

On 05.03.25 04:24, Boqun Feng wrote:
On Tue, Mar 04, 2025 at 12:18:28PM -0800, comex wrote:

On Mar 4, 2025, at 11:03 AM, Ralf Jung <post@xxxxxxxx> wrote:

Those already exist in Rust, albeit only unstably:
<https://doc.rust-lang.org/nightly/std/intrinsics/fn.volatile_copy_memory.html>.
However, I am not sure how you'd even generate such a call in C? The
standard memcpy function is not doing volatile accesses, to my
knowledge.

The actual memcpy symbol that exists at runtime is written in
assembly, and should be valid to treat as performing volatile
accesses.

memcpy is often written in C... and AFAIK compilers understand what that
function does and will, for instance, happily eliminate the call if they can
prove that the destination memory is not being read from again. So, it doesn't
behave like a volatile access at all.

But both GCC and Clang special-case the memcpy function. For example,
if you call memcpy with a small constant as the size, the optimizer
will transform the call into one or more regular loads/stores, which
can then be optimized mostly like any other loads/stores (except for
opting out of alignment and type-based aliasing assumptions). Even if
the call isn’t transformed, the optimizer will still make assumptions.
LLVM will automatically mark memcpy `nosync`, which makes it undefined
behavior if the function “communicate[s] (synchronize[s]) with another
thread”, including through “volatile accesses”. [1]

The question is more, what do clang and GCC document / guarantee in a stable
way regarding memcpy? I have not seen any indication so far that a memcpy call
would ever be considered volatile, so we have to treat it like a non-volatile
non-atomic operation.

However, these optimizations should rarely trigger misbehavior in
practice, so I wouldn’t be surprised if Linux had some code that
expected memcpy to act volatile…


Also in this particular case we are discussing [1], it's a memcpy (from
or to) a DMA buffer, which means the device can also read or write the
memory, therefore the content of the memory may be altered outside the
program (the kernel), so we cannot use copy_nonoverlapping() I believe.

[1]: https://lore.kernel.org/rust-for-linux/87bjuil15w.fsf@xxxxxxxxxx/

Is there actually a potential for races (with reads by hardware, not other
threads) on the memcpy'd memory?

There is another use case for this: copying data to/from a page that is
mapped into user space. In this case, a user space process can
potentially modify the data in the mapped page while we are
reading/writing that data. This would be a misbehaved user space
process, but it should not be able to cause UB in the kernel anyway.

Yeah that sounds like *the* prototypical case of sharing memory with an untrusted third party.


The C kernel just calls memcpy directly for this use case.

For this use case, we do not interpret or make control flow decisions
based on the data we read/write. And _if_ user space decides to do
concurrent writes to the page, we don't care if the data becomes
garbage. We just need the UB to be confined to the data moved from that
page, and not leak into the rest of the kernel.

There is no such thing as "confined UB". Well, there is "poison data", which can act a bit like that, but sadly the C standard is extremely ambiguous on that subject and has been for decades despite repeated requests for clarifications, so it is entirely unclear whether and how "poison data" could exist in C. clang, for once, has decided that "poison data" is UB in most situations (including just copying it to / returning it from another function), and this is consistent with some of the messaging of the standards committee. I don't know enough about the internals of gcc to comment on what they do.

Personally, I think that's a mistake; there needs to be some clear way to deal with uninitialized memory (which is the typical example of "poison data").

In Rust we have a fairly clear idea of what our rules should be here, and you can have "poison data" inside the `MaybeUninit` type. However, neither Rust nor C have a way to do reads where data races cause "poison data" rather than UB. See my other email I just sent for the rest of this line of discussion.
(I'm not used to the sprawling tree of a discussion that is this mailing list, so not sure how to best deal with replies that want to "merge" things said in different emails.)

Kind regards,
Ralf