[patch V3 00/37] mm/highmem: Preemptible variant of kmap_atomic & friends
From: Thomas Gleixner
Date: Tue Nov 03 2020 - 05:37:51 EST
Following up to the discussion in:
https://lore.kernel.org/r/20200914204209.256266093@xxxxxxxxxxxxx
and the second version of this:
https://lore.kernel.org/r/20201029221806.189523375@xxxxxxxxxxxxx
this series provides a preemptible variant of kmap_atomic & related
interfaces.
This is achieved by:
- Removing the RT dependency from migrate_disable/enable()
- Consolidating all kmap atomic implementations in generic code including
a useful version of the CONFIG_DEBUG_HIGHMEM which provides guard pages
between the individual maps instead of just increasing the map size.
- Switching from per CPU storage of the kmap index to a per task storage
- Adding a pteval array to the per task storage which contains the ptevals
of the currently active temporary kmaps
- Adding context switch code which checks whether the outgoing or the
incoming task has active temporary kmaps. If so, the outgoing task's
kmaps are removed and the incoming task's kmaps are restored.
- Adding new interfaces k[un]map_local*() which are not disabling
preemption and can be called from any context (except NMI).
Contrary to kmap() which provides preemptible and "persistant" mappings,
these interfaces are meant to replace the temporary mappings provided by
kmap_atomic*() today.
This allows to get rid of conditional mapping choices and allows to have
preemptible short term mappings on 64bit which are today enforced to be
non-preemptible due to the highmem constraints. It clearly puts overhead on
the highmem users, but highmem is slow anyway.
This is not a wholesale conversion which makes kmap_atomic magically
preemptible because there might be usage sites which rely on the implicit
preempt disable. So this needs to be done on a case by case basis and the
call sites converted to kmap_local().
Note, that this is only tested on X86 and completely untested on all other
architectures (at least it compiles except on csky which does not compile
with the newest cross tools from kernel.org independent of this change).
The lot is available from
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git highmem
It is based on Peter Zijlstras migrate disable branch which is close to be
merged into the tip tree, but still not finalized:
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git sched/migrate-disable
The series has the following parts:
Patches 1 - 22: Consolidation work which is independent of the scheduler
changes
79 files changed, 595 insertions(+), 1296 deletions(-)
Patch 23: Needs to be folded back into the sched/migrate-disable
Patches 24 - 26: The preemptible kmap_local() implementation
9 files changed, 283 insertions(+), 57 deletions(-)
Patches 27 - 37: Cleanup of the less common kmap/io_map_atomic users
19 files changed, 114 insertions(+), 256 deletions(-)
Vs. merging this pile:
If everyone agrees, I'd like to take the first part (1-22) through tip so
that the preemptible implementation can be sorted in tip once the scheduler
prerequisites are there. The initial cleanups (27-37) might have to wait if
there are conflicts vs. the drm/gpu tree. We'll see.
>From what I can tell kmap_atomic() can be removed all together and
completly replaced by kmap_local(). Most of the usage sites are trivial and
just doing memcpy(), memset() or trivial operations on the temporarily
mapped page. The interesting ones are those which do either conditional
stuff or have copy_.*_user_inatomic() inside. As shown with the crash and
drm/gpu cleanups this allows to simplify the code quite a bit.
Changes vs. V2:
- Remove the migrate disable from kmap_local and only issue that when the
there is an actual highmem mapping. (Linus)
- Reordered the series so the consolidation is upfront
- Get rid of kmap_types.h and the associated cruft
- Fixup documentation and add function documentation for kmap_*
- Splitout the internal implementation into a seperate header
- More cleanups - removal of unused functions
- Replace a few of the less frequently used kmap_atomic and
io_mapping_map_atomic variants and remove those interfaces.
Thanks,
tglx
---
arch/alpha/include/asm/kmap_types.h | 15
arch/arc/include/asm/kmap_types.h | 14
arch/arm/include/asm/kmap_types.h | 10
arch/arm/mm/highmem.c | 121 -------
arch/ia64/include/asm/kmap_types.h | 13
arch/microblaze/mm/highmem.c | 78 ----
arch/mips/include/asm/kmap_types.h | 13
arch/nds32/mm/highmem.c | 48 --
arch/parisc/include/asm/kmap_types.h | 13
arch/powerpc/include/asm/kmap_types.h | 13
arch/powerpc/mm/highmem.c | 67 ----
arch/sh/include/asm/kmap_types.h | 15
arch/sparc/include/asm/kmap_types.h | 11
arch/sparc/mm/highmem.c | 115 -------
arch/um/include/asm/kmap_types.h | 13
arch/x86/include/asm/kmap_types.h | 13
b/Documentation/driver-api/io-mapping.rst | 92 ++---
b/arch/arc/Kconfig | 1
b/arch/arc/include/asm/highmem.h | 26 +
b/arch/arc/mm/highmem.c | 54 ---
b/arch/arm/Kconfig | 1
b/arch/arm/include/asm/fixmap.h | 4
b/arch/arm/include/asm/highmem.h | 33 +-
b/arch/arm/mm/Makefile | 1
b/arch/arm/mm/cache-feroceon-l2.c | 6
b/arch/arm/mm/cache-xsc3l2.c | 4
b/arch/csky/Kconfig | 1
b/arch/csky/include/asm/fixmap.h | 4
b/arch/csky/include/asm/highmem.h | 6
b/arch/csky/mm/highmem.c | 75 ----
b/arch/microblaze/Kconfig | 1
b/arch/microblaze/include/asm/fixmap.h | 4
b/arch/microblaze/include/asm/highmem.h | 6
b/arch/microblaze/mm/Makefile | 1
b/arch/microblaze/mm/init.c | 6
b/arch/mips/Kconfig | 1
b/arch/mips/include/asm/fixmap.h | 4
b/arch/mips/include/asm/highmem.h | 6
b/arch/mips/kernel/crash_dump.c | 42 --
b/arch/mips/mm/highmem.c | 77 ----
b/arch/mips/mm/init.c | 4
b/arch/nds32/Kconfig.cpu | 1
b/arch/nds32/include/asm/fixmap.h | 4
b/arch/nds32/include/asm/highmem.h | 22 -
b/arch/nds32/mm/Makefile | 1
b/arch/openrisc/mm/init.c | 1
b/arch/openrisc/mm/ioremap.c | 1
b/arch/powerpc/Kconfig | 1
b/arch/powerpc/include/asm/fixmap.h | 4
b/arch/powerpc/include/asm/highmem.h | 7
b/arch/powerpc/mm/Makefile | 1
b/arch/powerpc/mm/mem.c | 7
b/arch/sh/include/asm/fixmap.h | 8
b/arch/sh/mm/init.c | 8
b/arch/sparc/Kconfig | 1
b/arch/sparc/include/asm/highmem.h | 8
b/arch/sparc/include/asm/vaddrs.h | 4
b/arch/sparc/mm/Makefile | 3
b/arch/sparc/mm/srmmu.c | 2
b/arch/um/include/asm/fixmap.h | 1
b/arch/x86/Kconfig | 3
b/arch/x86/include/asm/fixmap.h | 5
b/arch/x86/include/asm/highmem.h | 13
b/arch/x86/include/asm/iomap.h | 13
b/arch/x86/include/asm/paravirt_types.h | 1
b/arch/x86/kernel/crash_dump_32.c | 48 --
b/arch/x86/mm/highmem_32.c | 59 ---
b/arch/x86/mm/init_32.c | 15
b/arch/x86/mm/iomap_32.c | 57 ---
b/arch/xtensa/Kconfig | 1
b/arch/xtensa/include/asm/fixmap.h | 4
b/arch/xtensa/include/asm/highmem.h | 12
b/arch/xtensa/mm/highmem.c | 46 --
b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 7
b/drivers/gpu/drm/i915/i915_gem.c | 40 --
b/drivers/gpu/drm/i915/selftests/i915_gem.c | 4
b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 8
b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/fbmem.h | 8
b/drivers/gpu/drm/qxl/qxl_image.c | 18 -
b/drivers/gpu/drm/qxl/qxl_ioctl.c | 27 -
b/drivers/gpu/drm/qxl/qxl_object.c | 12
b/drivers/gpu/drm/qxl/qxl_object.h | 4
b/drivers/gpu/drm/qxl/qxl_release.c | 4
b/drivers/gpu/drm/ttm/ttm_bo_util.c | 20 -
b/drivers/gpu/drm/vmwgfx/vmwgfx_blit.c | 30 -
b/fs/aio.c | 1
b/fs/btrfs/ctree.h | 1
b/include/asm-generic/Kbuild | 2
b/include/asm-generic/kmap_size.h | 12
b/include/linux/highmem-internal.h | 210 ++++++++++++
b/include/linux/highmem.h | 294 ++++++------------
b/include/linux/io-mapping.h | 28 -
b/include/linux/kernel.h | 21 -
b/include/linux/preempt.h | 38 --
b/include/linux/sched.h | 11
b/kernel/entry/common.c | 2
b/kernel/fork.c | 1
b/kernel/sched/core.c | 63 +++
b/kernel/sched/sched.h | 4
b/lib/smp_processor_id.c | 2
b/mm/Kconfig | 3
b/mm/highmem.c | 255 ++++++++++++++-
include/asm-generic/kmap_types.h | 11
103 files changed, 959 insertions(+), 1576 deletions(-)