BKL removal from VFS

From: Nakayama Shintaro (nakayama@tritech.co.jp)
Date: Tue Feb 19 2002 - 02:17:52 EST

Hi,

I've found great BKL contention when running multiple postmark
benchmarks. Here is the postmark results with lock contention
sampled by lockmeter.

        # of trans/sec spinlock contention
1CPU 1197 --
4CPU 1549 62.2% (BKL=61.9%)
8CPU 1607 77.2% (BKL=77.0%)

        - kernel is 2.4.17
        - postmark params(number=5000 transactions=500000)
        - Xeon 550Mhz, 1G memory
        - virtually no disk I/O wait
        - concurrelty running 8 postmark processes

To improve postmark results, I removed BKL from contenting points
with modifications as follows.

- removed BKL from following operations
        i_op->create
        i_op->lookup
        i_op->link
        i_op->mknod
        i_op->mkdir
        i_op->unlink
        i_op->rmdir
        i_op->truncate
        i_op->setattr
        i_op->permission
        i_op->symlink
        f_op->readdir

        removed lock_kernel/unlock_kernel
                -> LOCK_KERNEL/UNLOCK_KERNEL

- fine-grained ext2 locks

        global resource protecting lock
        -----------------------------------------
        dir operation i_sem, i_zombie
        inode, block bitmap per-group lock
        inode meta data per-group lock
        ext2 super block super block lock

- changed VFS locking strategy
        fs_lock protects fs global operations such as sync,
        mount. fs_lock is read/write lock. Each entry into
        filesystem read-locks fs_lock, while global operations
        do write-locking.

- internaling BKL into some filesystems
        rootfs
        proc
        driverfs
        tmpfs (not shmfs)
        binfmt_misc
        ext3
        nfs

        inserted lock_kernel/unlock_kernel
                -> fs_lock_kernel/fs_unlock_kernel

        other filesystems are not modified, thus dangerous to
        use currently (they have no protecting lock!)

By this modification, BKL contention was reduced greatly.

        # of trans/sec spinlock contention
1CPU 2430 --
2CPU 4098 1.3%
4CPU 7529 3.2%
8CPU 11394 10.7%

        - 2.5.1 with BKL removal patch
        - postmark params(number=1000 transactions=200000)
        - others same as above

I've attached patch against 2.5.1 and m-postmark script. This
patch is tested on 8-way SMP machine with postmark and dbench.
You can get patch and data from

        http://www.tritech.co.jp/members/nakayama/linux/vfs-bkl-2.5.1.diff.gz
        http://www.tritech.co.jp/members/nakayama/linux/results.tar.bz2

I'm not in the list, please cc any reply to me. Hope this helps
to improve Linux.

(This work is paid by Fujitsu)


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

This archive was generated by hypermail 2b29 : Sat Feb 23 2002 - 21:00:18 EST