Re: [PATCH v10 3/6] mm: Introduce Reported pages

[Michael S. Tsirkin]

On Mon, Sep 23, 2019 at 07:50:15AM -0700, Alexander Duyck wrote:
> > > +static inline void
> > > +page_reporting_reset_boundary(struct zone *zone, unsigned int order, int mt)
> > > +{
> > > +     int index;
> > > +
> > > +     if (order < PAGE_REPORTING_MIN_ORDER)
> > > +             return;
> > > +     if (!test_bit(ZONE_PAGE_REPORTING_ACTIVE, &zone->flags))
> > > +             return;
> > > +
> > > +     index = get_reporting_index(order, mt);
> > > +     reported_boundary[index] = &zone->free_area[order].free_list[mt];
> > > +}
> >
> > So this seems to be costly.
> > I'm guessing it's the access to flags:
> >
> >
> >         /* zone flags, see below */
> >         unsigned long           flags;
> >
> >         /* Primarily protects free_area */
> >         spinlock_t              lock;
> >
> >
> >
> > which is in the same cache line as the lock.
> 
> I'm not sure what you mean by this being costly?

I've just been wondering why does will it scale report a 1.5% regression
with this patch.

> Also, at least on my system, pahole seems to indicate they are in
> different cache lines.
> 
> /* --- cacheline 3 boundary (192 bytes) --- */
> struct zone_padding        _pad1_;               /*   192     0 */
> struct free_area           free_area[11];        /*   192  1144 */
> /* --- cacheline 20 boundary (1280 bytes) was 56 bytes ago --- */
> long unsigned int          flags;                /*  1336     8 */
> /* --- cacheline 21 boundary (1344 bytes) --- */
> spinlock_t                 lock;                 /*  1344     4 */
> 
> Basically these flags aren't supposed to be touched unless we are
> holding the lock anyway so I am not sure it would be all that costly
> for this setup. Basically we are holding the lock when the flag is set
> or cleared, and we only set it if it is not already set. If needed
> though I suppose I could look at moving the flags if you think that is
> an issue. However I would probably need to add some additional padding
> to prevent the lock from getting into the same cache line as the
> free_area values.

-- 
MST