Re: Page alloc failures under network/disk IO load

From: Peter Zijlstra
Date: Thu Dec 04 2008 - 03:23:16 EST


On Wed, 2008-12-03 at 22:27 -0500, Dan Noà wrote:
> This is on Linux 2.6.28-rc7, on a Core 2 Duo. The system has plenty of
> memory:
>
> total used free shared buffers
> cached
> Mem: 1893 1822 70 0 0

filled to the brim with data

> 1573
> -/+ buffers/cache: 249 1644
> Swap: 1906 37 1868
>
> I am using rsync to transfer data onto this system. The filesystem is
> XFS, and the target drive is a 1TB Western Digital on ata_piix. The
> system files are on a RAID 1 (Linux md, also on ata_piix).
>
> Periodically I get page allocation failures, from __netdev_alloc_skb.
> I suppose this causes the driver to drop packets and thus hurts
> performance.

There isn't much we can do about that, memory is filled and your network
card tries to allocate memory in a mode that doesn't allow freeing some.

Looking at the timestamps its not very frequent, so it doesn't hurt
performance much if anything. If you're really bothered with this, you
could quiet it by sticking in a __GFP_NOWARN in __netdev_alloc_skb() or
something..



> Below are the logs from the entire event.
>
> Cheers,
> Dan
>
> [112854.339298] rsync: page allocation failure. order:1, mode:0x4020
> [112854.339338] Pid: 20229, comm: rsync Not tainted 2.6.28-rc7 #3
> [112854.339365] Call Trace:
> [112854.339387] <IRQ> [<ffffffff802c1cfe>]
> __alloc_pages_internal+0x40e/0x530
> [112854.339433] [<ffffffff802f14a4>] new_slab+0x274/0x350
> [112854.339461] [<ffffffff802f2670>] __slab_alloc+0x2c0/0x440
> [112854.339498] [<ffffffff8047dea8>] ? __netdev_alloc_skb+0x38/0x60
> [112854.339527] [<ffffffff8047dea8>] ? __netdev_alloc_skb+0x38/0x60
> [112854.339555] [<ffffffff802f3b14>]
> __kmalloc_node_track_caller+0x94/0x110
> [112854.339583] [<ffffffff8047dea8>] ? __netdev_alloc_skb+0x38/0x60
> [112854.339610] [<ffffffff8047dafc>] __alloc_skb+0x7c/0x160
> [112854.339637] [<ffffffff8047dea8>] __netdev_alloc_skb+0x38/0x60
> [112854.339684] [<ffffffffa00d975a>] rtl8169_rx_fill+0xca/0x1f0 [r8169]
> [112854.339716] [<ffffffffa00d9ad1>] rtl8169_rx_interrupt+0x251/0x540
> [r8169]
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/