RE: Cache Allocation Technology Design

[Auld, Will]

I maybe repeating what Peter has just said but for elements in the hierarchy where the mask is the same as its parents mask there is no need for a separate CLOS even in the case where there are tasks in the group. So we can inherit the CLOS of the parent until which time both the mask is different than the parent and there are tasks in the group. 

Thanks,

Will

> -----Original Message-----
> From: Peter Zijlstra [mailto:peterz@xxxxxxxxxxxxx]
> Sent: Wednesday, October 29, 2014 6:45 AM
> To: Matt Fleming
> Cc: Vikas Shivappa; linux-kernel@xxxxxxxxxxxxxxx; Fleming, Matt; Auld,
> Will; Tejun Heo; Shivappa, Vikas
> Subject: Re: Cache Allocation Technology Design
> 
> On Wed, Oct 29, 2014 at 12:48:34PM +0000, Matt Fleming wrote:
> > On Wed, 29 Oct, at 09:16:40AM, Peter Zijlstra wrote:
> > >
> > > Ah, so one way around that is to only assign a (whats the CQE
> > > equivalent of RMIDs again?) once you stick a task in.
> >
> > I think you're after "Class of Service" (CLOS) ID.
> >
> > Yeah we can do the CLOS ID assignment on-demand but what we can't do
> > on-demand is the cache bitmask assignment, i.e. how we carve up the
> LLC.
> > These need to persist irrespective of which task is running. And it's
> > the cache bitmask that I'm specifically talking about not allowing
> > arbitrarly deep nesting.
> >
> > So if I create a cgroup directory with a mask of 0x3 in the root
> > cgroup directory for CAT (meow).
> 
> All we now need is a DOG to go woof :-) and they can have a party.
> 
> > Then, create two sub-directories, and split my
> > 0x3 bitmask into 0x2 and 0x1, it's impossible to nest any further,
> i.e.
> >
> >                     /sys/fs/cgroup/cacheqe           0xffffffff
> >                               |
> >                               |
> >                             meow                     0x3
> >                            /    \
> >                           /      \
> >                         sub1    sub2                 0x1, 0x2
> >
> > Of course the pathological case is creating a cgroup directory with
> > bitmask 0x1, so you can't have sub-directories because you can't
> split
> > the cache allocation at all.
> >
> > Does this fly in the face of "full hierarchies"? Or is this a
> > reasonable limitation?
> 
> I don't see a reason why we should not allow further children of sub1,
> they'll all have to have 0x1, but that should be fine, pointless
> perhaps, but perfectly consistent.
> 
> > > But basically it means you need to allow things like:
> > >
> > > root/virt/more/crap/hostA
> > >                    /hostB
> > >     /sanityA
> > >     /random/other/yunk
> > >
> > > Now, the root will have the entire bitmask set, any child, say
> > > virt/more/crap can also have them all set, and you can maybe only
> > > start differentiating in the /host[AB] bits.
> > >
> > > Whether or not it makes sense, libvirt likes to create these
> > > pointless deep hierarchies, as do a lot of other people for that
> matter.
> >
> > OK, this is something I hadn't considered; that you may *not* want to
> > split the cache bitmask as you move down the hierarchy.
> >
> > I think that's something we could do without too much pain, though
> > actually programming that from a user perspective makes my head hurt.
> 
> Right, also note that in the libvirt case, most of the intermediate
> groups are empty (of tasks) and would thus not actually instantiate a
> CLOS thingy.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/