Re: [PATCH 0/2] tmpfs: Improve tmpfs scalability

[Hugh Dickins]

On Tue, 18 May 2010, tim wrote:
> 
> We created a token jar library implementing
> per cpu cache of tokens to avoid lock contentions whenever
> we retrieve or return a token to a token jar.  Using this library
> with tmpfs, we find Aim7 fserver throughput improved 270% 
> on a 4 socket, 32 cores NHM-EX system.
> 
> In current implementation of tmpfs, whenever we
> get a new page, stat_lock in shmem_sb_info needs to be acquired.  
> This causes a lot of lock contentions when multiple
> threads are using tmpfs simultaneously, which makes
> system with large number of cpus scale poorly.
> Almost 75% of cpu time was spent contending on
> stat_lock when we ran Aim7 fserver load with 128 threads
> on a 4 socket, 32 cores NHM-EX system.
> 
> The first patch in the series implements the quick token jar. 
> The second patch update the shmem code of tmpfs to use this
> library to improve tmpfs performance.

Interesting, thank you - I'll take a look, but not this week.

I do hope you're using Aim7 just as an example: you know that mounting
tmpfs with nr_blocks=0,nr_inodes=0 skips those shmem_sb_info updates
altogether?  Mounting in such a way should be fine for getting better
numbers out of Aim7; but yes, there are reallife uses for tmpfs which
are safer with the nr_blocks,nr_inodes limits.

Hugh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/