Re: RCU issue with SELinux (Re: SELINUX performance issues)

From: Kaigai Kohei
Date: Fri Aug 20 2004 - 08:40:34 EST


Hello, everyone.

Tuesday, August 17, 2004 12:19 AM
James Morris wrote:
> > Is removing direct reference to AVC-Entry approach acceptable?
> >
> > I'll try to consider this issue further.
>
> Sure, if you can make it work without problems.

The attached patches against to 2.6.8.1 kernel improve the performance
and the scalability of SELinux by RCU-approach.

The evaluation results are as follows:
<Environment>
CPU: Itanium2(1GHz) x 4/8/16/32
Memory: enough (no swap)
OS: 2.6.8.1 (SELinux disabled by 'selinux=0' boot option)
2.6.8.1 (SELinux enabled)
2.6.8.1 + rwlock patch by KaiGai
2.6.8.1 + RCU patch by KaiGai

The test program iterates write() to files on tmpfs 500,000 times in parallel.
It is attached as 'rcutest.c'.

We observed performance improvement and perfect scalability.

[rcutest Loop=500000 Parallel=<Num of CPU>]
- upper column is the RealTime [second].
- lefthand of lower column is the UserTime, righthand is the SysTime.
--- 4CPU --- --- 8CPU --- --- 16CPU --- --- 32CPU ---
2.6.8.1(disable) 8.0158[s] 8.0183[s] 8.0146[s] 8.0037[s]
(2.08/ 6.07) (1.86/6.31) (0.78/ 7.33) (2.03/5.94)
2.6.8.1(enable) 78.0957[s] 319.0451[s] 1322.0313[s] too long
(2.47/76.48) (2.47/316.96) (2.43/1319.8) (---/---)
2.6.8.1.rwlock 20.0100[s] 49.0390[s] 90.0200[s] 177.0533[s]
(2.57/17.52) (2.45/46.93) (2.37/87.78) (2.41/175.1)
2.6.8.1.rcu 11.0464[s] 11.0205[s] 11.0372[s] 11.0496[s]
(2.64/8.82) (2.21/8.98) (2.67/ 8.68) (2.51/8.95)
-------------------------------------------------------------------------


These patches achieve lock-free read access to AVC cache.

Significant changes are as follows:
- Direct references by avc_entry_ref were removed for RCU.
- Some structures(avc_entry/avc_node/avc_cache) were changed for RCU.
- avc_node type objects are allocated dynamically, not statically.
avc_reclaim_node() reclaims some avc_node objects when the number of
objects exceeds AVC_CACHE_THRESHOLD.
- For updating av_decision structure, I came up with a new RCU primitive,
list_replace_rcu() in include/linux/list.h .
It is needed to switch the linked pointer atomically, and replace
an old object with a new one. list_replace_rcu() is used in avc_update_node()
to update av_decision structure, and an original object is duplicated and updated.
We now pay more cost for updating av_decision structure as a compensation
for lock-free read access. But, in my opinion, updating av_decision is so rare.

NOTE: The fifth arguments of avc_has_perm() and avc_has_perm_noaudit()
are not necessary. But the prototype declarations and all the callers
remain, since I want to avoid messing up the patches.

Any comments?
----------
KaiGai <kaigai@xxxxxxxxxxxxx>

Attachment: selinux.rcu-2.6.8.1.patch
Description: Binary data

Attachment: list_replace_rcu-2.6.8.1.patch
Description: Binary data

Attachment: rcutest.c
Description: Binary data