RE: [PATCH v3 0/2] Move x86_cpu_to_apicid to the __read_mostlysection

[Shai Fultheim (Shai@xxxxxxxxxxx)]

Ingo,

The reason for this, as you pointed out, is the 'cache line' size (4096 bytes).  We see significant false sharing is we do not move this next to each other.  

We observed this for x86_cpu_to_apicid only, but per earlier request on this thread we implemented for the other (RO) vars as well.

Regards,
Shai.


--Shai 


> -----Original Message-----
> From: Ingo Molnar [mailto:mingo.kernel.org@xxxxxxxxx] On Behalf Of Ingo
> Molnar
> Sent: Monday, May 21, 2012 12:07
> To: Vlad Zolotarov
> Cc: Thomas Gleixner; linux-kernel; Ingo Molnar; H. Peter Anvin; Shai Fultheim
> (Shai@xxxxxxxxxxx); Ido Yariv
> Subject: Re: [PATCH v3 0/2] Move x86_cpu_to_apicid to the __read_mostly
> section
> 
> 
> * Vlad Zolotarov <vlad@xxxxxxxxxxx> wrote:
> 
> > Pls., consider applying this patch series.
> > It contains the following changes:
> >  - Adds two new macros DEFINE_EARLY_PER_CPU_READ_MOSTLY() and
> >    DECLARE_EARLY_PER_CPU_READ_MOSTLY().
> >  - Adds "read-mostly" qualifier to the following variables in smp.h:
> >   - cpu_sibling_map
> >   - cpu_core_map
> >   - cpu_llc_shared_map
> >   - cpu_llc_id
> >   - cpu_number
> >   - x86_cpu_to_apicid
> >   - x86_bios_cpu_apicid
> >   - x86_cpu_to_logical_apicid
> >
> > As long as all the variables above are only written during the
> > initialization, this change is meant to prevent the false
> > sharing and improve the performance on large multiprocessor
> > systems.
> 
> Mind explaining why it helps performance? Are these percpu
> variables also accessed from other CPUs, increasing the cost of
> false sharing of cache-lines?
> 
> Or is the problem that the 'cache line' size is 4096 bytes on
> vSMP, making even singular cache misses expensive and you'd like
> to move these next to each other?
> 
> So it would be nice to see some before/after PMU stats
> demonstrating the improvement. (perf works on vSMP, right?)
> 
> Thanks,
> 
> 	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/