Re: [PATCH v6 15/24] arm64: Switch to PMR masking when starting CPUs

[Julien Thierry]

On 04/12/18 17:51, Catalin Marinas wrote:
> On Mon, Nov 12, 2018 at 11:57:06AM +0000, Julien Thierry wrote:
>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>> index 8dc9dde..e495360 100644
>> --- a/arch/arm64/kernel/smp.c
>> +++ b/arch/arm64/kernel/smp.c
>> @@ -35,6 +35,7 @@
>>  #include <linux/smp.h>
>>  #include <linux/seq_file.h>
>>  #include <linux/irq.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>>  #include <linux/percpu.h>
>>  #include <linux/clockchips.h>
>>  #include <linux/completion.h>
>> @@ -175,6 +176,25 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle)
>>  	return ret;
>>  }
>>  
>> +static void init_gic_priority_masking(void)
>> +{
>> +	u32 gic_sre = gic_read_sre();
>> +	u32 cpuflags;
>> +
>> +	if (WARN_ON(!(gic_sre & ICC_SRE_EL1_SRE)))
>> +		return;
>> +
>> +	WARN_ON(!irqs_disabled());
>> +
>> +	gic_write_pmr(GIC_PRIO_IRQOFF);
>> +
>> +	cpuflags = read_sysreg(daif);
>> +
>> +	/* We can only unmask PSR.I if we can take aborts */
>> +	if (!(cpuflags & PSR_A_BIT))
>> +		write_sysreg(cpuflags & ~PSR_I_BIT, daif);
> 
> I don't understand this. If you don't switch off PSR_I_BIT here, where
> does it happen? In which scenario do we actually have the A bit still
> set? At a quick look, smp_prepare_boot_cpu() would have the A bit
> cleared previously by setup_arch(). We have secondary_start_kernel()
> where you call init_gic_priority_masking() before local_daif_restore().
> 

So this is for secondary CPUs where PSR.A can be still set.

The thing is that the daifflags.h establishes the order for disabling
types of exceptions:
Debug > Abort > IRQ

The idea is that when introducing pseudo-NMIs this becomes:
Debug > Abort > pseudo-NMI > IRQ

Whenever aborts are disabled (maybe because we just took an abort) we
don't want to take an NMI.

> So what happens if you always turn off PSR_I_BIT here?
> 

So semantically it would be saying "we can take a pseudo-NMI here".
Realistically, I think it depends on the state of the GIC redistributor
for this CPU:
- If the re-distributor was initialized, nothing bad could happen as no
NMI could have been configured for this CPU yet.
- If the re-distributor initialization is done between the call to
init_gic_priority_mask() and the local_daif_restore() then probably bad
things could happen

I can try to figure out if it is safe to just clear PSR.I always, but I
also find it easier to always play by the rule "if PSR.A is set, PSR.I
is set".

Thanks,

-- 
Julien Thierry