Re: [PATCH v2 6/6] selftests/resctrl: Reduce failures due to outliers in MBA/MBM tests

[Reinette Chatre]

Hi Ilpo,

On 9/15/2023 8:44 AM, Ilpo Järvinen wrote:
> The initial value of 5% chosen for the maximum allowed percentage
> difference between resctrl mbm value and IMC mbm value in commit
> 06bd03a57f8c ("selftests/resctrl: Fix MBA/MBM results reporting
> format") was "randomly chosen value" (as admitted by the changelog).
> 
> When running tests in our lab across a large number platforms, 5%
> difference upper bound for success seems a bit on the low side for the
> MBA and MBM tests. Some platforms produce outliers that are slightly
> above that, typically 6-7%, which leads MBA/MBM test frequently
> failing.
> 
> Replace the "randomly chosen value" with a success bound that is based
> on those measurements across large number of platforms by relaxing the
> MBA/MBM success bound to 8%. The relaxed bound removes the failures due
> the frequent outliers.
> 
> Fixes: 06bd03a57f8c ("selftests/resctrl: Fix MBA/MBM results reporting format")
> Cc: <stable@xxxxxxxxxxxxxxx>
> Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@xxxxxxxxxxxxxxx>

Thank you.

Reviewed-by: Reinette Chatre <reinette.chatre@xxxxxxxxx>

Reinette