[ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2021

From: Cyril Hrubis
Date: Mon Sep 27 2021 - 10:57:58 EST

Good news everyone,

the Linux Test Project test suite stable release for *September 2021* has been
released.

Since the last release 483 patches by 40 authors were merged.

This is a record breaking release by the number of patches as well as by the
amount of newly implemented functionality.

Many thanks to all contributors!

And even more thanks to the reviewers!

Patch review is what most of the projects struggle with and LTP is no
different. If you can spare some effort helping with the patch review is more
than welcomed.

NOTABLE CHANGES
===============

* New tests
- epoll_ctl05 checks for ELOOP error for circular loop of epolls
- epoll_ctl04 checks for EINVAL when epoll nesting is too deep
- epoll_ctl02 checks for EPERM when epoll is not supported for fd
- epoll_create01 checks that epoll create returns a valid fd
- epoll_create1_02 checks for EINVAL when epoll_create1() is passed invalid flags
- epoll_pwait02 test if events are reported properly
- epoll_pwait03 tests if epoll_pwait/epoll_pwait2 timeouts correctly
- epoll_pwait04 adds tests for zero timeout and invalid timespec values
- shmget05 test for /proc/sys/kernel/shm_next_id
- shmget06 test when shm_next_id is already in use
- io_getevents02 checks for EINVAL errors via libaio
- io_cancel02 checks for EFAULT errors via libaio
- io_submit03 checks for invalid input via syscall
- io_submit02 checks if correct operation via syscall
- io_setup02 checks for invalid inputs via syscall
- io_destroy02 checks for invalid inputs via syscall
- cfs_bandwidth01 a multi-level cgroup cpu controller test

* New regression tests
- creat09 privilege escalation using setgid directory aka CVE-2018-13405
- can_bcm01 race condition vulnerability in CAN BCM aka CVE-2021-3609
- setsockopt08 out-of-bounds write corrupting aka CVE-2021-22555
- perf_event_open03 kernel memory leak aka CVE-2020-25704
- squashfs01 regression test for broken sanity checks
- set_mempolicy05 information leak aka CVE-2017-7616
- userns08 broken id mappings in nested namespaces CVE-2018-18955
- tc01.sh regression test for tc qdisc command
- fanotify19 added regression test for:
a8b98c808eab ("fanotify: fix permission model of unprivileged group")
- kill13 INT_MIN negation reproducer aka CVE-2018-10124
- wait403 INT_MIN negation reproducer
- vsock01 race condition reproducer aka CVE-2021-26708
- icmp_rate_limit01 DNS cache poisoning attack aka CVE-2020-25705

* Increased coverage
- epoll_create02 checks for more EINVAL failures
- epoll_ctl03 checks that the syscall works with all valid flags combinations
- mbind01 more tests for MPOL_LOCAL
- epoll_pwait01 adds test variants for epoll_pwait2()

* 75 tests were rewritten to the new library

* Removed tests
- sched/stress as it was unmaintained and unused for years
- stress_cd and stress_floppy since these scripts were broken and unmaintained for years
- disktest as it was unmaintained and unused for years

* The CI was moved from Travis to GitHub actions

* 'make check' was introduced to the LTP build system
- which is a tool for developers that runs various sanity checks on the source code
- this includes:
- checkpatch.pl, custom set of sparse rules for C
- checkbashism.pl for shell
- please make sure you run 'make check' or 'make check-testname' and fix the
warnings before submitting patches

* Subset of our library tests i.e. tests for the test library is now executed
CI in github actions, this is ongoing effort to make sure that the test library
used by the testcases is working as expected

* The test library was fixed to print all messages to stderr

* Testcases were fixed not to expect certain users and groups to be present on the system
- some tests were expecting bin, daemon and similar groups and users to exist on the system
- now LTP depends only on user 'nobody' and group 'nogroup'

* New environment variable LTP_VIRT_OVERRIDE was added
- this is meant as a quick workaround for cases where the test library
cannot properly detect if system is virtualized or not
- see https://github.com/linux-test-project/ltp/wiki/User-Guidelines

* Fixed (again) timeouts in shell test library
- turns out you cannot make timeouts in shell race-free so the timeout logic
is now implemented in a small C helper binary

* Finally after the years we are slowly making tests limits-aware
- cgropup tests and a few others are now aware of pid limits imposed by the system
- next step would be doing rough estimates on the number of processes we can run
based on the free system memory

* There is ongoing effort to fix LTP cgroup tests, this release includes many
small fixes and improvements in the cgroup tests

* The aarch64 system call table was fixed not to include 32bit only syscall numbers

* The TST_EXP_*() macros were improved and a few bugs were fixed

* The test library developer documentation and tutorials were reorganized and improved
- https://github.com/linux-test-project/ltp/wiki/C-Test-API
- https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API
- https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

+ The more that the usual amount of fixes and cleanups

WHAT IS IN THE QUEUE
====================

Next release will hopefully include runltp-ng which is new and improved LTP
testrunner. The current runltp script will be maintained alongside with the
new solution for some time but removed after a few releases.

You are more than welcome to try it and make sure that it works for you before
it becomes the official LTP testrunner.

See: https://github.com/metan-ucw/runltp-ng

NOTABLE CHANGES IN NETWORK TESTS
================================
brought to you by Petr Vorel

* C network library
- Added device management helpers
- And documentation:
https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

* Shell network library
- Allow to skip initialization with TST_NET_SKIP_VARIABLE_INIT=1
- Disable IPv6 Duplicate Address Detection on link-local addresses

* Tests rewritten to the new library
- getaddrinfo01
- ssh-stress

* Removed tests
- broken_ip-totlen

+ More busybox and general fixes

DOWNLOAD AND LINKS
==================

The latest version of the test-suite contains 3000+ tests for the Linux
and can be downloaded at:

https://github.com/linux-test-project/ltp/releases/tag/20210927

The project pages as well as GIT repository are hosted on GitHub:

https://github.com/linux-test-project/ltp
http://linux-test-project.github.io/

If you ever wondered how to write a LTP testcase, don't miss our developer
documentation at:

https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines

https://github.com/linux-test-project/ltp/wiki/C-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Network-API

https://github.com/linux-test-project/ltp/wiki/Shell-Test-API

https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial

https://github.com/linux-test-project/ltp/wiki/BuildSystem

Patches, new tests, bugs, comments or questions should go to to our mailing
list at ltp@xxxxxxxxxxxxxx.

CREDITS
=======

Many thanks to the people contributing to this release:

git shortlog -s -e -n 20210927..

87 Petr Vorel <pvorel@xxxxxxx>
51 Xie Ziyao <xieziyao@xxxxxxxxxx>
51 Richard Palethorpe <rpalethorpe@xxxxxxxx>
50 Martin Doucha <mdoucha@xxxxxxx>
30 Krzysztof Kozlowski <krzysztof.kozlowski@xxxxxxxxxxxxx>
29 Cyril Hrubis <chrubis@xxxxxxx>
28 Alexey Kodanev <aleksei.kodanev@xxxxxxxxxxx>
27 Yang Xu <xuyang2018.jy@xxxxxxxxxxx>
20 Li Wang <liwang@xxxxxxxxxx>
18 zhanglianjie <zhanglianjie@xxxxxxxxxxxxx>
17 Joerg Vehlow <joerg.vehlow@xxxxxxxxxxx>
12 Vinay Kumar <vinay.m.engg@xxxxxxxxx>
8 Xie Ziyao <ziyaoxie@xxxxxxxxxxx>
7 sujiaxun <sujiaxun@xxxxxxxxxxxxx>
5 Bogdan Lezhepekov <bogdan.lezhepekov@xxxxxxxx>
5 Dai Shili <daisl.fnst@xxxxxxxxxxx>
5 Jan Stancek <jstancek@xxxxxxxxxx>
5 QI Fuli <qi.fuli@xxxxxxxxxxx>
4 Leo Liang <ycliang@xxxxxxxxxxxxx>
2 Alexander Egorenkov <egorenar@xxxxxxxxxxxxx>
2 Amir Goldstein <amir73il@xxxxxxxxx>
2 Shiyang Ruan <ruansy.fnst@xxxxxxxxxxx>
2 Thadeu Lima de Souza Cascardo <cascardo@xxxxxxxxxxxxx>
2 Zhao Gongyi <zhaogongyi@xxxxxxxxxx>
1 Alessio Balsini <balsini@xxxxxxxxxxx>
1 Bogdan Lezhepekov via ltp <ltp@xxxxxxxxxxxxxx>
1 Cao jin <caojin@xxxxxxxxxxxxx>
1 Egor Petrov <egor.petrov@xxxxxxxxxx>
1 Ian May <ian.s.may@xxxxxxxxx>
1 John Stultz <john.stultz@xxxxxxxxxx>
1 Leo Yu-Chi Liang <ycliang@xxxxxxxxxxxxx>
1 Marius Hillenbrand <mhillen@xxxxxxxxxxxxx>
1 Naresh Kamboju <naresh.kamboju@xxxxxxxxxx>
1 Radoslav Kolev <radoslav.kolev@xxxxxxxx>
1 Wang Xin <wangxin410@xxxxxxxxxx>
1 Xiao Yang <yangx.jy@xxxxxxxxxxx>
1 Zou Wei <zou_wei@xxxxxxxxxx>
1 dongshijiang <dongshijiang@xxxxxxxxxx>

And also thanks to patch reviewers:

git log 20210927.. | grep -Ei '(reviewed|acked)-by:' | sed 's/.*by: //' | sort | uniq -c | sort -n -r

289 Cyril Hrubis <chrubis@xxxxxxx>
113 Petr Vorel <pvorel@xxxxxxx>
75 Li Wang <liwang@xxxxxxxxxx>
25 Richard Palethorpe <rpalethorpe@xxxxxxxx>
16 Yang Xu <xuyang2018.jy@xxxxxxxxxxx>
12 Alexey Kodanev <aleksei.kodanev@xxxxxxxxxxx>
7 Jan Stancek <jstancek@xxxxxxxxxx>
6 Richard Palethorpe <rpalethorpe@xxxxxxx>
5 Xiao Yang <yangx.jy@xxxxxxxxxxx>
5 Martin Doucha <mdoucha@xxxxxxx>
3 Joerg Vehlow <joerg.vehlow@xxxxxxxxxxx>
3 Enji Cooper <yaneurabeya@xxxxxxxxx>
2 Krzysztof Kozlowski <krzysztof.kozlowski@xxxxxxxxxxxxx>
1 Yang Xu <xuyang2018.jy@xxxxxxxxxxx
1 Xie Ziyao <xieziyao@xxxxxxxxxx>
1 Stephen Kitt <steve@xxxxxxx>
1 Petr Vorel <pvorel@xxxxxxxx>
1 Leo Yu-Chi Liang <ycliang@xxxxxxxxxxxxx>
1 Joerg Vehlow <lkml@xxxxxxxxxxx>

--
Cyril Hrubis
chrubis@xxxxxxx