Re: [PATCH] arm64: mmu: no write cache for O_SYNC flag

[Wang, Li]

å 2020/3/27 0:55, Catalin Marinas åé:
> On Thu, Mar 26, 2020 at 09:36:25AM -0700, Li Wang wrote:
> > reproduce steps:
> > 1.
> > disable CONFIG_STRICT_DEVMEM in linux kernel
> > 2.
> > Process A gets a Physical Address of global variable by
> > "/proc/self/pagemap".
> > 3.
> > Process B writes a value to the same Physical Address by mmap():
> > fd=open("/dev/mem",O_SYNC);
> > Virtual Address=mmap(fd);
> >
> > problem symptom:
> > after Process B write a value to the Physical Address,
> > Process A of the value of global variable does not change.
> > They both W/R the same Physical Address.
> >
> > technical reason:
> > Process B writing the Physical Address is by the Virtual Address,
> > and the Virtual Address comes from "/dev/mem" and mmap().
> > In arm64 arch, the Virtual Address has write cache.
> > So, maybe the value is not written into Physical Address.
> >
> > fix reason:
> > giving write cache flag in arm64 is in phys_mem_access_prot():
> > =====
> > arch/arm64/mm/mmu.c
> > phys_mem_access_prot()
> > {
> >    if (!pfn_valid(pfn))
> >      return pgprot_noncached(vma_prot);
> >    else if (file->f_flags & O_SYNC)
> >      return pgprot_writecombine(vma_prot);
> >    return vma_prot;
> > }
> > ====
> > the other arch and the share function drivers/char/mem.c of phys_mem_access_prot()
> > does not add write cache flag.
> > So, removing the flag to fix the issue
> Other architectures may have transparent caches and don't require
> different attributes.
>
> > Signed-off-by: Li Wang <li.wang@xxxxxxxxxxxxx>
> > Cc: Catalin Marinas <catalin.marinas@xxxxxxx>
> > Cc: Will Deacon <will@xxxxxxxxxx>
> > Cc: linux-arm-kernel@xxxxxxxxxxxxxxxxxxx
> > Cc: linux-kernel@xxxxxxxxxxxxxxx
> > ---
> >   arch/arm64/mm/mmu.c | 2 --
> >   1 file changed, 2 deletions(-)
> >
> > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > index 128f70852bf3..d7083965ca17 100644
> > --- a/arch/arm64/mm/mmu.c
> > +++ b/arch/arm64/mm/mmu.c
> > @@ -81,8 +81,6 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn,
> >   {
> >   	if (!pfn_valid(pfn))
> >   		return pgprot_noncached(vma_prot);
> > -	else if (file->f_flags & O_SYNC)
> > -		return pgprot_writecombine(vma_prot);
> >   	return vma_prot;
> >   }
> >   EXPORT_SYMBOL(phys_mem_access_prot);
> A better solution is for user space not to pass O_SYNC when opening
> /dev/mem. We've had this ABI for a long time (arch/arm/ and several
> other architectures do the same), why change it now?


1.

no pass O_SYNC in user space is not a good idea.

in fact, the codes come from 'devmem' command of busybox:

=====

busybox-1.24.1/miscutils$ vim devmem.c

fd = xopen("/dev/mem", O_SYNC);

=====

the codes are used for a long time.


2.

according to info of open man about "O_SYNC":

=====

http://man7.org/linux/man-pages/man2/open.2.html

the output data and associated file metadata have been transferred to 
the underlying hardware

=====

I think "O_SYNC" means no cache.


3.

/dev/mem of driver offers 2 ways to operate physical memory.

one is mmap, the other is read/write.

when use read/write way, it operates uncached memory:

=====

kernel-source/drivers/char/mem.c

write_mem(){

/* it must also be accessed uncached */

}

=====


4.

arm64 arch is different with other arch about phys_mem_access_prot().

you can see no any other arch add cache flag in the function.

only arm and arm64 add write cache for O_SYNC flag.


x86/mm/pat.c

phys_mem_access_prot(){

return vma_prot;

}


powerpc/mm/mem.c

phys_mem_access_prot(){
ÂÂÂÂÂÂÂ if (ppc_md.phys_mem_access_prot)
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ return ppc_md.phys_mem_access_prot(file, pfn, size, 
vma_prot);
ÂÂÂÂÂÂÂ if (!page_is_ram(pfn))
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ vma_prot = pgprot_noncached(vma_prot);
ÂÂÂÂÂÂÂ return vma_prot;
}


drivers/char/mem.c

phys_mem_access_prot()
{
#ifdef pgprot_noncached
ÂÂÂÂÂÂÂ phys_addr_t offset = pfn << PAGE_SHIFT;

ÂÂÂÂÂÂÂ if (uncached_access(file, offset))
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ return pgprot_noncached(vma_prot);
#endif
ÂÂ return vma_prot;
}


Thanks,

LiWang.