Re: WT memory type on x86_64?

[Dave Airlie]

On Sat, Apr 27, 2013 at 10:37 AM, Andy Lutomirski <luto@xxxxxxxxxxxxxx> wrote:
> On Wed, Apr 24, 2013 at 12:33 PM, Andy Lutomirski <luto@xxxxxxxxxxxxxx> wrote:
>> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
>> write-through memory.  I'd like to gauge how unpleasant this will be.
>>
>> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
>> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
>> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
>> crossed and hope that there are enough free MTRRs, or I could try to
>> free up a PAT entry.
>>
>> How nasty will the latter be?  I just looked at two rather different
>> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
>> MTRRs in the WC or WP states.  As long as those MTRR types aren't
>> used, I think the UC- PAT entry is useless -- it behaves identically
>> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
>> video memory.  Is there any need for this on modern machines?  That
>> is, are there any drivers that actually need the mtrr_add call to
>> succeed on a machine that has a working PAT?
>>
>
> FWIW, I've done a bit of a survey.  Things that use UC or UC- include:
>
>  - ioremap_nocache: ISTM that any correct caller wants genuine UC memory.
>
>  - plain ioremap: Are there architectures where it's not
> ioremap_nocache?  (Tn any case, this is irrelevant.)
>
>  - pci_iomap: This is used all over the framebuffer code.  It seems to
> be equivalent to ioremap or ioremap_nocache, which are the same thing
> on x86.
>
>  - AGP: The AGP code seems inconsistent.  alloc_page gets a cacheable
> page of RAM.  alloc_pages gets uncached pages of RAM.  In there's a WC
> MTRR on RAM, then everything is screwed up anyway.
>
>  - ttm: This code is newish.  I imagine that everything using ttm that
> wants WC memory asks TTM for WC, which will work just fine.  In any
> case, the allocations are AFAICS backed by RAM, so there should be no
> conflicts.
>
>  - radeon's gart: Ditto
>
>  - efi: presumably !WB means UC is fine.  (Why would EFI need WC?)
>
>  - uvesafb: The MTRR code is terrifying.  It looks nearly useless (it
> has alignment issues) and it's unnecessary on a system with PAT.  In
> any case, this code certainly isn't expecting a WC MTRR with any kind
> of mapping other than ioremap_wc.
>
>
> mtrr_add users include:
>
>  - tdfxfb, vt8623fb, sgivwfb, s3fb, etc. should be converted to use ioremap_wc
>  - myri10ge tries to use an MTRR.  This is, IMO, strange.
>  - Infiniband.  I think it's okay if the MTRR doesn't work.
>
>
> The only problematic (and not trivially fixable) thing I found is
> pci_mmap_page_range, which uses UC- and is part of the ABI -- old X
> drivers may care.
>
> I wonder if X (using UMS) will slow down if WC MTRRs become illegal or
> stop being added by old framebuffer drivers.  (If so, they can be
> randomly slow anyway -- lots of machines have no free MTRRs).

Don't forget you can add mtrrs from userspace via /proc/mtrr. I'm not sure
what sort ABI guarantees are on this.

TTM allocations are not necessarily backed by RAM, they can also from
device memory.

Also i915 has mtrr code, but we avoid touching mtrrs if we are on a PAT cpu.

Dave.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/