RE: Cache coherency... and locking

From: Linda Walsh (law@sgi.com)
Date: Fri Jul 21 2000 - 12:54:45 EST


> And: do you need a synchroneous update?
>
> I mean: cpu1 is near the audit point, and cpu2 changes the audit mask.
> Do you have to guarantee that if cpu1 is before the audit point, and
> cpu2 has returned from the "change_audit" function, that then cpu1 will
> see the new audit mask?
>
> All fast cpu queue memory writes, so the window is at least several
> dozend cpu ticks even on i386, and on NUMA it could be huge.

---
	It is the 'huge' case I'm worried about.  I'm an operator.  I see a 
process that is suspicious.  I set an auditmask on the process to audit more
things.  How long from a user perspective would it be before the auditmask
took?  1-2 seconds might be in the noise level, 1-2 minutes would not.

> If you want to align a structure, then use > > struct my_struct { > int my_val; > } __attribute__ ((__aligned__(SMP_CACHE_BYTES))); --- Very cool.

> > > From another mail: > > So it sounds like the safest way to do things is to use a spinlock on a per-process > > basis -- which should almost never fail resulting in mininal overhead. If I want > > to globally turn audit on/off, I need to lock the task struct and walk through > > it setting audit on/off flags to allowed values. > > You could also use a big reader spinlock, see > linux/include/linux/brlock.h. > > > The thing that is a pain is > > that the auditmask is 256 bytes long or a full cache line. > > 256 bits? --- well it's 2 masks of 128, 1 set for event successes, 1 set for event failures. > > > I suppose we could use 4x32 bit words for an audit mask rather than 2x64 -- speeding > > things up for the 32 bit case. > > 4*32 == 256? --- New math? :-) My brain was befuddled with mask vs. struct size...

thanks....I'll look into bigreader. I'm still getting some emails from folks who think that the case w/o locking should work fine.

I think the bottom line may be the upper bound on processor inconsistency in NUMA. Is it in user-noise level or is it a noticable user-time delay. If it can be the 2nd then a lock (perhaps we need a new lock that is a dummy lock for SMP, but a real lock for NUMA?) of some sort is needed for the NUMA case. Looking over the source, I don't see any references to a 'NUMA' variable to check for. It it is going to require different semantics to guarantee reasonable update time should there be one?

-l

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Sun Jul 23 2000 - 21:00:16 EST