Re: [PATCH] powerpc/32s: Fix napping restore in data storage interrupt (DSI)

[Nicholas Piggin]

Excerpts from Christophe Leroy's message of August 4, 2021 11:28 pm:
> 
> 
> Le 04/08/2021 à 13:36, Nicholas Piggin a écrit :
>> Excerpts from Christophe Leroy's message of August 4, 2021 4:21 pm:
>>> Hi Nic,
>>>
>>> I think I'll need your help on that one.
>>>
>>> Le 04/08/2021 à 08:07, Christophe Leroy a écrit :
>>>>
>>>>
>>>> Le 04/08/2021 à 06:04, Finn Thain a écrit :
>> 
>> Hi Finn!
>> 
>>>>> On Tue, 3 Aug 2021, Christophe Leroy wrote:
>>>>>
>>> ...
>>>>>
>>>>> ------------[ cut here ]------------
>>>>> kernel BUG at arch/powerpc/kernel/interrupt.c:49!
>>>>> Oops: Exception in kernel mode, sig: 5 [#1]
>>>>> BE PAGE_SIZE=4K MMU=Hash SMP NR_CPUS=2 PowerMac
>>>>> Modules linked in:
>>>>> CPU: 0 PID: 1859 Comm: xfce4-session Not tainted 5.13.0-pmac-VMAP #10
>>>>> NIP:  c0011474 LR: c0011464 CTR: 00000000
>>>>> REGS: e2f75e40 TRAP: 0700   Not tainted  (5.13.0-pmac-VMAP)
>>>>> MSR:  00021032 <ME,IR,DR,RI>  CR: 2400446c  XER: 20000000
>>>>>
>>>>> GPR00: c001604c e2f75f00 ca284a60 00000000 00000000 a5205eb0 00000008 00000020
>>>>> GPR08: ffffffc0 00000001 501200d9 ce030005 ca285010 00c1f778 00000000 00000000
>>>>> GPR16: 00945b20 009402f8 00000001 a6b87550 a51fd000 afb73220 a6b22c78 a6a6aecc
>>>>> GPR24: 00000000 ffffffc0 00000020 00000008 a5205eb0 00000000 e2f75f40 000000ae
>>>>> NIP [c0011474] system_call_exception+0x60/0x164
>>>>> LR [c0011464] system_call_exception+0x50/0x164
>>>>> Call Trace:
>>>>> [e2f75f00] [00009000] 0x9000 (unreliable)
>>>>> [e2f75f30] [c001604c] ret_from_syscall+0x0/0x28
>>>>> --- interrupt: c00 at 0xa69d6cb0
>>>>> NIP:  a69d6cb0 LR: a69d6c3c CTR: 00000000
>>>>> REGS: e2f75f40 TRAP: 0c00   Not tainted  (5.13.0-pmac-VMAP)
>>>>> MSR:  0000d032 <EE,PR,ME,IR,DR,RI>  CR: 2400446c  XER: 20000000
>>>>>
>>>>> GPR00: 000000ae a5205de0 a5687ca0 00000000 00000000 a5205eb0 00000008 00000020
>>>>> GPR08: ffffffc0 401201ea 401200d9 ffffffff c158f230 00c1f778 00000000 00000000
>>>>> GPR16: 00945b20 009402f8 00000001 a6b87550 a51fd000 afb73220 a6b22c78 a6a6aecc
>>>>> GPR24: afb72fc8 00000000 00000001 a5205f30 afb733dc 00000000 a6b85ff4 a5205eb0
>>>>> NIP [a69d6cb0] 0xa69d6cb0
>>>>> LR [a69d6c3c] 0xa69d6c3c
>>>>> --- interrupt: c00
>>>>> Instruction dump:
>>>>> 7cdb3378 93810020 7cbc2b78 93a10024 7c9d2378 93e1002c 7d3f4b78 4800d629
>>>>> 817e0084 931e0088 69690002 5529fffe <0f090000> 69694000 552997fe 0f090000
>>>>> ---[ end trace c66c6c3c44806276 ]---
>>>>>
>>>
>>> Getting a BUG at arch/powerpc/kernel/interrupt.c:49 meaning MSR_RI is not set, but the c00 interrupt
>>> frame shows MSR_RI properly set, so what ?
>> 
>> Could the stack be correct but regs pointer incorrect?
>> 
>> Instruction dump is
>> 
>>     0:   78 33 db 7c     mr      r27,r6
>>     4:   20 00 81 93     stw     r28,32(r1)
>>     8:   78 2b bc 7c     mr      r28,r5
>>     c:   24 00 a1 93     stw     r29,36(r1)
>>    10:   78 23 9d 7c     mr      r29,r4
>>    14:   2c 00 e1 93     stw     r31,44(r1)
>>    18:   78 4b 3f 7d     mr      r31,r9
>>    1c:   29 d6 00 48     bl      0xd644
>>    20:   84 00 7e 81     lwz     r11,132(r30)
>>    24:   88 00 1e 93     stw     r24,136(r30)
>>    28:   02 00 69 69     xori    r9,r11,2
>>    2c:   fe ff 29 55     rlwinm  r9,r9,31,31,31
>>    30:   00 00 09 0f     twnei   r9,0
>>    34:   00 40 69 69     xori    r9,r11,16384
>>    38:   fe 97 29 55     rlwinm  r9,r9,18,31,31
>>    3c:   00 00 09 0f     twnei   r9,0
>> 
>> regs->msr is in r11 == 0xce030005 so some kernel address?
>> 
>> r1  == 0xe2f75f00
>> r30 == 0xe2f75f40
>> 
>> I think that matches if the function allocates 48 bytes of stack.
>> STACK_FRAME_OVERHEAD is 16, so the difference would be 0x40 in that
>> case. Seems okay.
>> 
>> I'm not sure then. Can you get a hash fault interrupt come in here
>> because of the vmap stack access and clobber r11? Hmm...
>> 
>> fast_hash_page_return:
>>          andis.  r10, r9, SRR1_ISI_NOPT@h        /* Set on ISI, cleared on DSI */
>> 
>> Is that really right? DSI can set this bit for NOHPTE as well no?
> 
> On DSI, the error bits are in DSISR while they are in SRR1 on ISI.
> 
> r9 is supposed to contain SRR1 in both cases. Powerpc 32 bits programming manual explicitely says 
> that bits 1-4 and 10-15 of SRR1 are cleared on DSI.

Ah right, I had in mind it was DSISR on DSI and SRR1 on ISI because
we put them together early on 64s. Can't think of anything else at the 
moment.

Thanks,
Nick