Re: [Xen-devel] Regression in netfilter code under Xen

From: Ian Campbell
Date: Wed Nov 26 2014 - 04:45:33 EST

On Tue, 2014-11-25 at 15:23 -0500, Boris Ostrovsky wrote:
> We have a regression due to (5195c14c8: netfilter: conntrack: fix race
> in __nf_conntrack_confirm against get_next_corpse). I have not been able
> to reproduce this on baremetal but dom0 crashes reliably after a few
> seconds of idle time.

Are guests running when this happens? (IOW is netback possibly
involved).

> This doesn't appear to be dependent on Xen version
> --- I saw it on at least a 4.2 and unstable).
>
> I don't know much about networking (and will be out for most of the rest
> of the week) so I wonder whether anyone has any ideas. The stack is below.
>
>
> Thanks.
> -boris
>
> # 
> [
> 54.901368] general protection fault: 0000 [#1] SMP
> [ 54.919324] Modules linked in: dm_multipath dm_mod xen_evtchn
> iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport
> libcrc32c crc32c_generic crc32c_intel sg sr_mod cdrom sd_mod i915 fbcon
> tileblit font bitblit softcursor tpm_tis ahci libahci libata
> drm_kms_helper video scsi_mod e1000e wmi xen_blkfront xen_netfront
> fb_sys_fops sysimgblt sysfillrect syscopyarea xenfs xen_privcmd
> [ 54.996767] CPU: 1 PID: 0 Comm: swapper/1 Not tainted
> 3.18.0-rc5upstream-00179-g8a84e01 #1
> [ 55.016730] Hardware name: LENOVO ThinkServer TS130/ , BIOS
> 9HKT47AUS 01/10/2012
> [ 55.036873] task: ffff880037e48a10 ti: ffff880037e54000 task.ti:
> ffff880037e54000
> [ 55.057338] RIP: e030:[<ffffffff815ea29e>] [<ffffffff815ea29e>]
> nf_ct_del_from_dying_or_unconfirmed_list+0x3e/0x70
> [ 55.[ 55.099069] RAX: dead000000200200 RBX: ffffe8ffffc89108 RCX:
> ffffffff81cc5820
> [ 55.119918] RDX: 0000000080000001 RSI: 0000000000000011 RDI:
> ffffe8ffffc89108
> [ 55.140768] RBP: ffff88003e283908 R08: 00000000c5f889ff R09:
> 00000000a1a10f28
> [ 55.161785] R10: ffff880037e5f1b8 R11: 0000000000000001 R12:
> ffff880037e5f148
> [ 55.183005] R13: ffffffff815e6a6b R14: ffff88002bed8400 R15:
> ffff88002bb30000
> [ 55.204214] FS: 00007f668f694700(0000) GS[ 55.225733] CS: e033
> DS: 002b ES: 002b CR0: 0000000080050033
> [ 55.247220] CR2: 00007f66c0a598eb CR3: 000000002bc39000 CR4:
> 0000000000042660
> [ 55.268863] Stack:
> [ 55.290200] ffff880037e5f148 ffffffff81c98080 ffff88003e283928
> ffffffff815eb59d
> [ 55.312068] ffff88002bed8400 ffff88002bed8400 ffff88003e283938
> ffffffff815e689a
> [ 55.333849] ffff88003e283958 ffffffff815a37a5 ffff88003e283968
> ffff88002bed8400
> [ 55.355449] Call Trace:
> [ 55.376552] <IRQ>
> [ 55.376728] [<ffffffff815eb59d>] destroy_conntrack+0x5d/0xc0
> [ 55.418187] [<ffffffff815e689a>] nf_conntrack_destroy+0x1a/0x40
> [ 55.439349] [<ffffffff815a37a5>] skb_release_head_state+0x85/0xd0
> [ 55.460468] [<ffffffff815a5d61>] skb_release_all+0x11/0x30
> [ 55.481440] [<ffffffff815a5dd1>] __kfree_skb+0x11/0xc0
> [ 55.502239] [<ffffffff815a5f78>] kfree_skb+0x38/0xa0
> [ 55.522661] [<ffffffff816044b0>] ? ip_rcv+0x3a0/0x3a0
> [ 55.542811] [<ffffffff816044b0>] ? ip_rcv+0x3a0/0x3a0
> [ 55.562543] [<ffffffff815e6a6b>] nf_hook_slow+0x10b/0x120
> [ 55.582100] [<ffffffff816044b0>] ? ip_rcv+0x3a0/0x3a0
> [ 55.601470] [<ffffffff8170a02b>] ? _raw_spin_unlock_irqrestore+0[
> 55.620900] [<ffffffff81604713>] ip_local_deliver+0x73/0x80
> [ 55.640154] [<ffffffff81603dc9>] ip_rcv_finish+0x109/0x310
> [ 55.659244] [<ffffffff8160439c>] ip_rcv+0x28c/0x3a0
> [ 55.678287] [<ffffffff815b69ae>]
> __netif_receive_skb_core+0x58e/0x[ 55.697287] [<ffffffff815b6b4d>]
> __netif_receive_skb+0x1d/0x70
> [ 55.715820] [<ffffffff815b6d9e>] netif_receive_skb_internal+0x1e/0xb0
> [ 55.734442] [<ffffffff815b6e47>] netif_receive_skb+0x17/0x70
> [ 55.752898] [<ffffffff816c17e2>] br_handle_frame_finish+0x192/0x360
> [ 55.771709] [<ffffffff816c156d>] br_handle_frame+0
> [ 55.790274] [<ffffffff816c13f0>] ? br_handle_local_finish+0x40/0x40
> [ 55.808760] [<ffffffff815b65f7>] __netif_receive_skb_core+0x1d7/0x710
> [ 55.827302] [<ffffffff815b6b4d>] __netif_receive_skb+0x1d/0x70
> [ 55.845594] [<ffffffff815b6d9e>] netif_receive_skb_internal+0x1e/0xb0
> [ 55.863836] [<ffffffff815b7a38>] napi_gro_receive+0x118/0x1a0
> [ 55.881549] [<ffffffff814c430f>] tg3_poll_work+0xe1f/0x1[
> 55.898629] [<ffffffff814d75c9>] tg3_poll+0x69/0x3c0
> [ 55.915010] [<ffffffff815b7523>] net_rx_action+0x103/0x290
> [ 55.930840] [<ffffffff810a7713>] __do_softirq+0xf3/0x2e0
> [ 55.946310] [<ffffffff810a7a2d>] irq_exit+0xcd/0xe0
> [ 55.961399] [<ffffffff8140d4c7>] xen_evtchn_do_upcall+0x37/0x50
> 766] [<ffffffff8170c1fe>] xen_do_hypervisor_callback+0x1e/0x30
> [ 55.990802] <EOI>
> [ 55.990977] [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
> [ 56.018897] [<ffffffff810013aa>] ? xen_hypercall_sched_op+0xa/0x20
> [ 56.032815] [<ffffffff810454b0>] ? xen_safe_halt+0x10/0x20
> [ 56.046583] [<ffffffff81059e3a>] ? default_idle+0x1a/0xd0
> [ 56.060307] [<ffffffff8105950a>] ? arch_cpu_idle+0xa/0x10
> [ 56.073920] [<ffffffff810de483>] ? cpu_startup_entry+0x2d3/0[
> 56.100421] [<ffffffff8104c355>] ? cpu_bringup_and_idle+0x25/0x40
> [ 56.113427] Code: 98 08 08 00 00 0f b7 47 08 48 03 1c c5 a0 42 cc 81
> 48 89 df e8 94 fc 11 00 49 8b 44 24 18 48 85 c0 74 2d 49[
> 56.141507] RIP [<ffffffff815ea29e>]
> nf_ct_del_from_dying_or_unconfirmed_list+0x3e/0x70
> [ 56.155425] RSP <ffff88003e2838f8>
> [ 56.169234] ---[ end trace 4c64578e4c629cc4 ]---
> [ 56.182913] Kernel panic - not syncing: Fatal exception in interrupt
> [ 56.197026] Kernel Offset: 0x0 from 0xffffffff81000k text console
> (XEN) [2014-11-23 06:05:48] Domain 0 crashed: rebooting machine in 5
> seconds.
> (XEN) [2014-11-23 06:05:53] Resetting with ACPI MEMORY or I/O RESET_REG.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/