Re: Reg the next LTS kernel (6.1?)

From: Michal Simek
Date: Mon Jan 16 2023 - 07:03:21 EST



On 1/13/23 17:22, Greg KH wrote:
On Fri, Jan 13, 2023 at 01:54:29PM +0100, Michal Simek wrote:


On 1/13/23 12:27, Greg KH wrote:
On Thu, Jan 12, 2023 at 01:26:28PM +0100, Michal Simek wrote:
Hi Greg,

On 1/12/23 13:23, Kris Chaplin wrote:
Hello Greg,

You tell me please.  How has your testing gone for 6.1 so far?  Does it
work properly for you?  Are you and/or your company willing to test out
the -rc releases and provide feedback if it works or not for your
systems?  Do you have problems with 6.1.y vs. older kernels?  Is there
anything missing in it that you feel needs to be addressed with a newer
kernel instead?

We have been integrating and testing 6.1 on the Microblaze, ARM32 and
ARM64-bit architectures over the past few weeks.  These builds have
been successful and we are able to run our regression tests on hardware
targeting our FPGA SoC devices.

We're continuing our tests as new updates to the 6.1 kernel series
appear.

As Kris said AMD/Xilinx has already moved internal SOC tree to 6.1 based
kernel in expectation that 6.1 will become next LTS.
And I am not aware about any issue from testing team related to 6.1 kernel
version. And we are covering our AMD/Xilinx SOCs based on arm32/arm64 and
Microblaze CPUs.

It would be good to continue with the same strategy which using the latest
kernel at that year which is what I am hearing all the time from others that
6.1 was last kernel at that year and it should be LTS.

Generally yes, I pick the last release of the year but we need people to
verify and validate that both it works for them, and that they are going
to be using it in their systems and can provide testing results to us
(as well as providing a way for their devices to actually be updated to
the new releases, we've had previous stable kernel releases that were
never actually shipped out to devices...)

We are preparing base (every year) for our customers which is what it is
also shipped in our distribution.

What distribution does AMD create? Who uses it? Where is it published
and how is it updated?

AMD acquired Xilinx last year. That's why Xilinx PetaLinux distribution is under AMD wings. It means it is distribution for Xilinx SOCs (Microblaze, Zynq (arm32), ZynqMP/Versal/VersalNET(arm64).
https://www.xilinx.com/products/design-tools/embedded-software/petalinux-sdk.html


It is used initially by all Xilinx customers. Kernel itself is wired inside this distribution and also used by Xilinx meta Layers in Yocto.
Cannonical is also using it as a base for AMD/Xilinx derivative kernel in Ubuntu for supporting arm64 based SOCs.


Other distribution like Ubuntu with their derivative kernels prefer to stay
on the LTS kernels too.

I already discuss this with the various "normal" distributions all the
time so they know what is going on here already.

ok

Also users will be getting fixes against our base for extended period with
fixes going from vanilla kernel.

What is "your base"? Who uses this? How long is it supported?

For 2022 it was 5.15 + Xilinx SOC patches.
For 2023 it is going to be 6.1 + Xilinx SOC patches.

As I said above. It is wired in petalinux, yocto and also partners are using it.

Xilinx is supporting this kernel for one year till the next upgrade.
After it 3rd party partners are continue to support these kernels if required.


For code which is not upstream(which we unfortunately have it) we are
providing fixes only till the next LTS version base is created.

So you only support the image for one year?

From AMD/Xilinx point of view unfortunately yes but in Ubuntu via derivative kernel newer LTS version are merged and supported base on their support strategy.


That's why it is very beneficial to use for our base kernel which is going
to be LTS that users will get that extended period support.

But you should be keeping this up to date for each normal release,
right?

We have never had enough resources to support multiple kernels. In perfect world we should do it but there are all the time constraints.
I am keep pushing over years to do upstreaming as early as possible and then our changes will be spread to all kernels. But we are not there yet.

Never do only yearly jumps, that way causes way too much work
and is proven to be a bad idea.

I know that for many years but not the person who is deciding the upgrade strategy.


And why not just update your "customers" to every normal release? Why
stick them with one a year? What is preventing more frequent updates?

Questions are correct but I am not defining strategy.

My goal is very simple to push everything to upstream and then we don't need to have discussions about it.

Base Kernel version is taken after your decision about LTS. But this year
because of timing we couldn't wait for this decision. That's why 6.1 was
taken with expectation that your generic concept about picking up the last
release of the year is going to continue.

That's a great guess, but again, you should always be updating your
out-of-tree changes against every release so it will not matter what
kernel is picked as a stable one or not.

There are a lot of things what we should be doing but we don't have unlimited number of resources especially on testing side. For me it is not a problem to preparing that kernels but someone has to run regressions over it and fixing issues if any issue happens.


Tested wise. We have test result from 6.1 <.0> and I have also shared 6.1.5
based version for our distribution. As of today we can't see any issue with
6.1 kernel in general on features which we are using on AMD/Xilinx SOCs.

Great, can you also test the -rc releases to make sure we don't break
anything for your systems?

Request to test the HEAD of TF-A, U-Boot and Linux was sent long time ago.
And I hope that regressions are running based on available features internally.

And our ZynqMP board is the part of LTSI and KernelCI and I see that regression is running there properly

https://linux.kernelci.org/soc/zynqmp/job/next/
https://lava.ciplatform.org/scheduler/device_type/zynqmp-zcu102

Another regression is running on Canonical side.


Also Jonathan wrote at https://lwn.net/Articles/915435/
"Unless something extremely surprising happens, 6.1 will be the final kernel
release for 2022, and thus will become the next LTS kernel."
That's why I hope that 6.1 is going to be next LTS.

Testing and feedback is going to be what allows this to happen. I can't
happen in a vacuum, and nor should you want it to.

And that was the purpose of jumping to this thread to send you a message that we are going to use 6.1 kernel and we can't see any issue with it on our side.
If 6.1 is going to be LTS then good for us.
If different kernel is going to be pick up based on feedback you get then we will just stay on kernel which is not going to be LTS.
For me this discussion is worth as input for internal discussion for kernel selection for 2024.

As I said I prefer to push everything upstream, properly mark fixes and then kernel version decision is up to every distribution/project. I got there with TF-A project and we are pretty much there with U-Boot and the kernel is the last one to focus.


I didn't run any stats but normally also more patches are going to this
version to be the part of LTS.

What do you mean by this? The patches accepted so far since 6.1.0 was
released, or up until 6.1.0 was released? For the patches since 6.1.0
was released, that's due to more developers/maintainers tagging patches
during the -rc1 merge window for stable releases (honestly they should
have gotten them into the -final release first), and due to us having
better tools in digging up potential stable patches (i.e. Sasha's
AUTOSEL bot work.)

I am just saying that developers/driver owners can simple do calculation to
identify LTS version. When they know it they also know time when their
deadline is for upstreaming work. It means if patch is accepted between
6.0-r1 and 6.0-rc5/6 they know that it will get to 6.1 merge window.

That is what I am afraid of and if it causes problems I will purposfully
pick the previous release. This has happened in the past and is never
an excuse to get anything merged. Code gets merged when it is ready,
not based on a LTS release.

No doubt about it.

Thanks,
Michal