Re: questions on how x86 uses IPI for tlb invalidates

[Kumar Gala]

On Nov 18, 2008, at 10:03 AM, Nick Piggin wrote:

> On Wednesday 19 November 2008 02:51, Kumar Gala wrote:
> > On Nov 18, 2008, at 9:44 AM, Nick Piggin wrote:
> > > On Wednesday 19 November 2008 02:20, Kumar Gala wrote:
> > > > I'm looking at adding some code into powerpc for doing tlb  
> > > > invalidate
> > > > broadcasts via IPI and am looking at the x86 code as a model.  I  
> > > > was
> > > > wondering how the x86 code ensures we aren't interrupted in the
> > > > process of doing something like flush_tlb_page().
> > >
> > > What's the problem with being interrupted?
> >
> > I'm in the middle of doing an invalidate, I get interrupted, in
> > processing the interrupt I cause another tlb invalidate while the
> > first one is still in progress and hasn't finished.
> >
> > For example I'm seeing in my code hacking:
> >
> > Freeing unused kernel memory: 208k init
> > BUG: soft lockup - CPU#1 stuck for 61s! [init:1]
> > Modules linked in:
> > NIP: c00649c4 LR: c00649a8 CTR: 00000000
> > REGS: ef82b750 TRAP: 0901   Not tainted  (2.6.28-rc5-00019-ge14c8bf-
> > dirty)
> > MSR: 00029000 <EE,ME>  CR: 44000428  XER: 00000000
> > TASK = ef830000[1] 'init' THREAD: ef82a000 CPU: 1
> > GPR00: 00000001 ef82b800 ef830000 c1002320 00029000 ef82b8f8 00000001
> > 00000000
> > GPR08: 00000002 00000001 00000000 00029000 84000422
> > NIP [c00649c4] generic_exec_single+0x84/0xb0
> > LR [c00649a8] generic_exec_single+0x68/0xb0
> > Call Trace:
> > [ef82b800] [c02f4084] dev_hard_start_xmit+0x324/0x3c0 (unreliable)
> > [ef82b830] [c0064afc] smp_call_function_single+0xdc/0x110
> > [ef82b870] [c0064ce8] smp_call_function_mask+0x1b8/0x250
> > [ef82b8f0] [c0015468] flush_tlb_page+0xa8/0xc0
> > [ef82b920] [c03a599c] xdr_partial_copy_from_skb+0x20c/0x2f0
> > [ef82b970] [c03a5ae8] csum_partial_copy_to_xdr+0x68/0x170
> > [ef82b990] [c03a8d60] xs_udp_data_ready+0x100/0x250
> > [ef82b9d0] [c02e8308] sock_queue_rcv_skb+0xf8/0x120
> > [ef82b9f0] [c033766c] __udp_queue_rcv_skb+0x1c/0x110
> > [ef82ba00] [c033897c] udp_queue_rcv_skb+0x28c/0x2e0
> > [ef82ba20] [c0338bf0] __udp4_lib_rcv+0x220/0x890
> > [ef82ba80] [c03136b0] ip_local_deliver+0xa0/0x240
> > [ef82baa0] [c03133ec] ip_rcv+0x3bc/0x5e0
> > [ef82bae0] [c02f3754] netif_receive_skb+0x284/0x310
> > [ef82bb10] [c0219c98] gfar_clean_rx_ring+0xd8/0x430
> > [ef82bb50] [c021a080] gfar_poll+0x90/0x170
> > [ef82bb70] [c02f1374] net_rx_action+0x104/0x200
> > [ef82bbc0] [c004166c] __do_softirq+0xbc/0x180
> > [ef82bc00] [c0004dc4] do_softirq+0x64/0x70
> > [ef82bc10] [c00410f4] irq_exit+0x54/0x70
> > [ef82bc20] [c0004f1c] do_IRQ+0x8c/0x140
> > [ef82bc40] [c00109cc] ret_from_except+0x0/0x18
> > [ef82bd00] [c0510e78] per_cpu____irq_regs+0x0/0x4
> > [ef82bd30] [c0064afc] smp_call_function_single+0xdc/0x110
> > [ef82bd70] [c0064ce8] smp_call_function_mask+0x1b8/0x250
> > [ef82bdf0] [c0015468] flush_tlb_page+0xa8/0xc0
> > [ef82be20] [c008c60c] handle_mm_fault+0x70c/0xa80
> > [ef82be80] [c0013df4] do_page_fault+0x274/0x510
> > [ef82bf40] [c00107b4] handle_page_fault+0xc/0x80
> > Instruction dump:
> > 913f0004 7c641b78 7f43d378 93e90000 4836226d 7c0004ac 7f9cc800  
> > 419e0030
> > 73c00001 40820008 48000010 a01f0010 <70090001> 4082fff8 80010034
> > bb010010
>
> You can't do the broadcast TLB invalidates in kunmap_atomic.
> All that's needed in that case is to just invalidate the local
> CPU.

Agreed, I was planning on fixing that.  I guess the high level  
question is if a broadcast invalidate can occur in soft or hard irq  
context.

- k
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/