Re: [RFC] cxl_test: upgrade as a first class citizen selftests capable driver

[Luis Chamberlain]

On Mon, Dec 19, 2022 at 04:27:10PM -0800, Dan Williams wrote:
> Luis Chamberlain wrote:
> > On Fri, Dec 16, 2022 at 08:55:19PM -0800, Dan Williams wrote:
> > > In other words the suggestion that the current
> > > organization ultimately leads to bit rot has not been substantiated in
> > > practice.
> > 
> > On top of this patch I just added a custom debug patch to my tree which
> > enables CXL_BUS and CXL_TEST by default when this is currently allowed
> > and it got quite a bit of kernel build warnings. Although some of these
> > are specific to my change, some of them do not seem to be related to
> > that and likely could benefit from fixing:
> > 
> > https://gist.github.com/mcgrof/73dce72939590c6edc9413b0384ae4c2
> > 
> > And so although you may not see some build warnings so far, it does not
> > negate my suggestion that having cxl_test as a proper upstream driver strategy
> > gets you more build testing / coverage.
> 
> If autobuild coverage of test components is the main concern then
> cxl_test can copy what nfit_test is doing with CONFIG_NVDIMM_TEST_BUILD.
> No need for disruptive redesign of how this facility is integrated.

I've itemized a list of gains of having this properly integrated. What
gains are there of this being an external module other than a few folks
are used to it and it been done before for other subsystems?

> > > The proposed direction to move tests out of the ndctl.git repo into the
> > > kernel solves the wrong problem.
> > 
> > That's not in any way what I suggested and is not the point to my patch.
> > 
> > The proposed patch does not suggest to ditch ndctl unit tests but to
> > *enable* also sefltests to make use of cxl_test using the selftests
> > framework, which is very different. It is not saying "abandon" ndctl
> > unit tests, but rather, "also enable linux kernel selftests for CXL with
> > cxl_test".
> 
> I think centralizing test scripts is a virtue, and right now the
> momentum is with those located ndctl.git. This is why I jumped to the
> conclusion that the long term direction would be to pick one location
> for maintainer regression tests.

That's fine for ndctl unit tests.

> > But more importantly, it looks for the value of proper kernel
> > integration and making use of kconfig for the actual dependencies
> > and requirements. This is of high value.
> > 
> > In addition to this, one possible area I see of value with this change is the
> > ability to also use the wrap feature later, even without cxl_test to enable
> > error injection. What would this look like? You simply replace one built in
> > routine as you do with another which has sprinkled in should_fail() calls,
> > which otherwise would be an eyesore upstream. This shold also then not
> > depend the rest of cxl_test stuff, but can make use of building
> > alternative wrap routines which could be replacement for upstream ones.
> > 
> > Another benefit of this strategy is you can also test cxl_test *without*
> > the need for for requiring modules, which some folks prefer for testing.
> > At LSFMM this came up for instance and one of the biggest grudges with
> > testing some frameworks was the dependency on modules.
> 
> I do think this is the void that QEMU CXL testing would attempt to fill.

Not for error injection support. You need proper kernel support for that.

  Luis