Peter Zijlstra wrote:On Mon, 2008-08-25 at 13:22 +0300, TÃrÃk Edwin wrote:
Well, the real program (clamd) that this testprogram tries to simulate does an mmap for almost every file, and I have lots of small files.
6.5G, 114122 files, average size 57k.
I'll run latencytop again, last time it has showed 100ms - 500ms latency
Latencytop output attached.
There is 4 - 60 ms latency for mmap/munmap, and the more threads there are the total latency gets higher (latencytop says sum was ~480ms).
Running with MaxThreads 4 gets me 300-400% CPU usage, but with MaxThreads 8 CPU usage drops to around 120-250%.
Now, maxthreads 4 looks like a good choice from a CPU usage point of view, but is actually bad because it means that threads gets stuck in iowait, and the CPU won't have anything to do. MaxThreads 8 looked like a good alternative to fill the iowait gaps, but we run into the mmap_sem issue.
In a real world environment MaxThreads influences how many mails you can process in parallel with your MTA, so generally it should be as high as possible.
On 2.6.27-rc4:
MaxThreads 4 time, empty database (all cached, almost no I/O):
1m9s
MaxThreads 4 time, after echo 3>/proc/sys/vm/drop_caches:
1m29s
MaxThreads 8 time, empty database (all cached, almost no I/O):
2m16s
MaxThreads 8 time, after echo 3>/proc/sys/vm/drop_caches:
2m15s
Of course running with a full database will give different results, so I'll do some timing with that too (will take a little longer though).
for clamd, and it was about mmap, I'll provide you with the exact output.
Right - does it make sense to teach clamav about pread() ?
If it is preferred over mmap, then maybe yes.
Peter Zijlstra wrote:
Best regards,
--Edwin
OK, I'll poke a little more at is later today to see if I can spot
something
Thanks!