Slow copy_to_user performance after kernel upgrade

[Thad Phetteplace]

I am debugging an issue with copy_to_user performance on recent linux
kernels.  I've noticed a 4X slowdown in copy speed when I move from
kernel 2.6.18 to more recent kernels on the same hardware.  I've
tested so far on 2.6.32 and on various 3.x kernels.  I noticed that
assembly code implementation of copy_to_user has changed, so as a
sanity check I ported the 2.6.18 version over to 2.6.32 but saw the
same slowdown when using it.  This is the hardware I am using:

Dell PowerEdge R710
BIOS version 6.3.0
Intel(R) Xeon(R) CPU x5650 @ 2.67GHz
72GB RAM (ECC DDR3 800MHz)

I am running the x86_64 version of course.  I see the issue when I do
a copy_to_user from a dedicated block of memory located above the 4GB
boundary into normal user space mem.  The boot parameter mem=65G is
passed to the kernel to carve off the dedicated block of memory that
is used as a DMA accessible buffer for a specialized network card.
That is the buffer we do the copy_to_user from.  I should also mention
we have been testing this on CentOS 4.8, 6.2, and 6.3, though I've
replaced their stock kernels with ones straight from kernel.org, so I
don't know that the linux distro is really relevant.

I've browsed the kernel mailing list looking for clues and have even
been looking at CPU/hardware specific errata, but so far I can see no
logical reason why the 2.6.32+ kernels should be performing so much
worse than the 2.6.18.  Does anyone have any suggestions on areas to
dig into besides the copy_to_user implementation itself?  Maybe
somewhere specific in the MMU code?  Any suggestions would be
appreciated.  Please copy me directly on replies if you don't mind.

Thanks,

Thad Phetteplace
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/