Re: Proposal: /lib/modules/<version> matching /usr/src/linux

From: Keith Owens (kaos@ocs.com.au)
Date: Mon Jul 31 2000 - 03:21:06 EST


On Sun, 30 Jul 2000 19:13:58 -0700,
"Adam J. Richter" <adam@yggdrasil.com> wrote:
>[snip] I would like to change
>the layout of /lib/modules/<version> to mirror the layout of
>/usr/src/linux. I see the following advantages to this approach:

modutils 2.3.12 added /lib/modules/<version>/kernel for exactly this
reason. The idea is to copy modules from linux/modules/*.o to
/lib/modules/<version>/kernel (LMVK) but preserving the same directory
structure as the kernel source. So you would find sound drivers under
LMVK/sound/, block drivers under LMVK/drivers/block and the ide driver
would be LMVK/drivers/ide/ide.o.

I plan to remove all kernel top level directories under LMVK except for
kernel. External packages like pcmcia would still have their own
directory. depmod, insmod etc. would look under LMVK first then the
old directory names, in the same order as existing modutils.

> 1. simplifying development of automated tools that would benefit
> from being able to classify modules (e.g., for hardware
> autoconfiguration),

As long as the source tree has decent names. But anything will be
better than the existing crud. Just watch out for developers who like
to rename objects.

> 2. users who have one less directory layout to learn

Definitely.

> 3. It should eliminate a line from many Makefile's in kernel
> subdirectories

A line which just happens to be one of the most incorrectly used
features of the Makefiles. Developers do not understand how to use
MOD_LIST_NAME, it is completely manual and is horribly error prone.

As a test, compile some net drivers, including wan LMC, as modules and
make modules_install. Check which directory under
/lib/modules/<version> the net modules get installed in. Then

  touch drivers/net/wan/lmc/lmc.c
  make SUBDIRS=drivers/net/lmc modules
  make modules_install

Now see where the modules get installed. If you did not erase the old
net modules from LMV first, you now have two copies. Which one gets
loaded first?

khhtpd has the same problem but these are not the only places where
MOD_LIST_NAME is used incorrectly. MOD_LIST_NAME is Broken As Designed
(BAD).

> 4. It should eliminate the need for a staging area in kernel
> build process for installing modules.

I was planning to keep the staging area. Some external kernel packages
look in linux/modules so removing this directory would break those
packages. Also the staging area ensures that all module names are
unique. At one stage we had two modules with the same name in
different source subtrees and the flat staging area caught that error.

This is the patch I sent to linux-kbuild a couple of months ago. I was
planning on adding the recursive support to modutils a bit sooner but
got sidetracked on kdb and IA64 work.

----

How about this patch, against 2.3.99-pre6-5? MOD_SUB_DIRS and MOD_IN_SUB_DIRS are now equivalent, MOD_TO_LIST does nothing. The only problem is that make modules_install takes 11 seconds instead of 3, most of that is due to the recursive make overhead. It's a small price to pay to remove all the problems. If this is acceptable, I will update modutils to scan modules/kernel.

I suspect that we could get rid of MOD_SUB_DIRS and MOD_IN_SUB_DIRS and just use SUBDIRS throughout. AFAICT these variables are all part of the great modules_install kludge. I have not patched any sub Makefile yet, that cleanup can come later to remove MOD_... A change to just the top level Makefile and Rules.make is easier to explain, instead of a change to every Makefile.

Index: 99-pre6-5.1/Rules.make --- 99-pre6-5.1/Rules.make Sat, 01 Apr 2000 17:51:17 +1000 kaos (linux-2.3/d/b/27_Rules.make 1.4 644) +++ 99-pre6-5.1(w)/Rules.make Tue, 25 Apr 2000 02:31:20 +1000 kaos (linux-2.3/d/b/27_Rules.make 1.4 644) @@ -32,8 +32,6 @@ unexport MIX_OBJS unexport SYMTAB_OBJS -unexport MOD_LIST_NAME - # # Get things started. # @@ -132,37 +130,32 @@ PDWN=$(shell $(CONFIG_SHELL) $(TOPDIR)/scripts/pathdown.sh) endif -ifdef MOD_SUB_DIRS -$(patsubst %,_modsubdir_%,$(MOD_SUB_DIRS)) : dummy +unexport MOD_DIRS +MOD_DIRS := $(MOD_SUB_DIRS) $(MOD_IN_SUB_DIRS) +ifneq "$(strip $(MOD_DIRS))" "" +.PHONY: $(patsubst %,_modsubdir_%,$(MOD_DIRS)) +$(patsubst %,_modsubdir_%,$(MOD_DIRS)) : dummy $(MAKE) -C $(patsubst _modsubdir_%,%,$@) modules -endif -ifdef MOD_IN_SUB_DIRS -$(patsubst %,_modinsubdir_%,$(MOD_IN_SUB_DIRS)) : dummy - $(MAKE) -C $(patsubst _modinsubdir_%,%,$@) modules +.PHONY: $(patsubst %,_modinst_%,$(MOD_DIRS)) +$(patsubst %,_modinst_%,$(MOD_DIRS)) : dummy + $(MAKE) -C $(patsubst _modinst_%,%,$@) modules_install endif +.PHONY: modules modules: $(ALL_MOBJS) $(MIX_OBJS) $(MI_OBJS) dummy \ - $(patsubst %,_modsubdir_%,$(MOD_SUB_DIRS)) \ - $(patsubst %,_modinsubdir_%,$(MOD_IN_SUB_DIRS)) -ifneq "$(strip $(MOD_LIST_NAME))" "" - rm -f $$TOPDIR/modules/$(MOD_LIST_NAME) -ifdef MOD_SUB_DIRS - for i in $(MOD_SUB_DIRS); do \ - echo `basename $$i`.o >> $$TOPDIR/modules/$(MOD_LIST_NAME); done -endif -ifneq "$(strip $(ALL_MOBJS))" "" - echo $(ALL_MOBJS) >> $$TOPDIR/modules/$(MOD_LIST_NAME) -endif -ifneq "$(strip $(MOD_TO_LIST))" "" - echo $(MOD_TO_LIST) >> $$TOPDIR/modules/$(MOD_LIST_NAME) -endif -endif + $(patsubst %,_modsubdir_%,$(MOD_DIRS)) + +.PHONY: _modinst__ +_modinst__: dummy ifneq "$(strip $(ALL_MOBJS))" "" - echo $(PDWN) - cd $$TOPDIR/modules; for i in $(ALL_MOBJS); do \ - ln -sf ../$(PDWN)/$$i $$i; done + mkdir -p $(MODLIB)/kernel/$(PDWN) + cp $(ALL_MOBJS) $(MODLIB)/kernel/$(PDWN) endif + +.PHONY: modules_install +modules_install: _modinst__ \ + $(patsubst %,_modinst_%,$(MOD_DIRS)) # # A rule to do nothing Index: 99-pre6-5.1/Makefile --- 99-pre6-5.1/Makefile Sat, 22 Apr 2000 11:49:47 +1000 kaos (linux-2.3/M/b/40_Makefile 1.7.1.19.1.3.1.3.1.7.2.1.1.12.1.17 644) +++ 99-pre6-5.1(w)/Makefile Tue, 25 Apr 2000 02:26:13 +1000 kaos (linux-2.3/M/b/40_Makefile 1.7.1.19.1.3.1.3.1.7.2.1.1.12.1.17 644) @@ -76,6 +76,9 @@ # makefile but the arguement can be passed to make if needed. # +MODLIB := $(INSTALL_MOD_PATH)/lib/modules/$(KERNELRELEASE) +export MODLIB + # # standard CFLAGS # @@ -295,56 +298,24 @@ MODFLAGS += -DMODVERSIONS -include $(HPATH)/linux/modversions.h endif +.PHONY: modules modules: $(patsubst %, _mod_%, $(SUBDIRS)) -modules/MARKER: - mkdir -p modules - touch modules/MARKER - -$(patsubst %, _mod_%, $(SUBDIRS)) : include/linux/version.h include/config/MARKER modules/MARKER +.PHONY: $(patsubst %, _mod_%, $(SUBDIRS)) +$(patsubst %, _mod_%, $(SUBDIRS)) : include/linux/version.h include/config/MARKER $(MAKE) -C $(patsubst _mod_%, %, $@) CFLAGS="$(CFLAGS) $(MODFLAGS)" MAKING_MODULES=1 modules -modules_install: - @( \ - MODLIB=$(INSTALL_MOD_PATH)/lib/modules/$(KERNELRELEASE); \ - cd modules; \ - MODULES=""; \ - inst_mod() { These="`cat $$1`"; MODULES="$$MODULES $$These"; \ - mkdir -p $$MODLIB/$$2; cp $$These $$MODLIB/$$2; \ - echo Installing modules under $$MODLIB/$$2; \ - }; \ - mkdir -p $$MODLIB; \ - \ - if [ -f BLOCK_MODULES ]; then inst_mod BLOCK_MODULES block; fi; \ - if [ -f NET_MODULES ]; then inst_mod NET_MODULES net; fi; \ - if [ -f IPV4_MODULES ]; then inst_mod IPV4_MODULES ipv4; fi; \ - if [ -f IPV6_MODULES ]; then inst_mod IPV6_MODULES ipv6; fi; \ - if [ -f ATM_MODULES ]; then inst_mod ATM_MODULES atm; fi; \ - if [ -f IDE_MODULES ]; then inst_mod IDE_MODULES ide; fi; \ - if [ -f SCSI_MODULES ]; then inst_mod SCSI_MODULES scsi; fi; \ - if [ -f FS_MODULES ]; then inst_mod FS_MODULES fs; fi; \ - if [ -f NLS_MODULES ]; then inst_mod NLS_MODULES fs; fi; \ - if [ -f CDROM_MODULES ]; then inst_mod CDROM_MODULES cdrom; fi; \ - if [ -f HAM_MODULES ]; then inst_mod HAM_MODULES net; fi; \ - if [ -f SOUND_MODULES ]; then inst_mod SOUND_MODULES sound; fi; \ - if [ -f VIDEO_MODULES ]; then inst_mod VIDEO_MODULES video; fi; \ - if [ -f FC4_MODULES ]; then inst_mod FC4_MODULES fc4; fi; \ - if [ -f IRDA_MODULES ]; then inst_mod IRDA_MODULES net; fi; \ - if [ -f SK98LIN_MODULES ]; then inst_mod SK98LIN_MODULES net; fi; \ - if [ -f SKFP_MODULES ]; then inst_mod SKFP_MODULES net; fi; \ - if [ -f USB_MODULES ]; then inst_mod USB_MODULES usb; fi; \ - if [ -f USB_SERIAL_MODULES ]; then inst_mod USB_SERIAL_MODULES usb; fi; \ - if [ -f IEEE1394_MODULES ]; then inst_mod IEEE1394_MODULES ieee1394; fi; \ - if [ -f PCMCIA_MODULES ]; then inst_mod PCMCIA_MODULES pcmcia; fi; \ - if [ -f PCMCIA_NET_MODULES ]; then inst_mod PCMCIA_NET_MODULES pcmcia; fi; \ - if [ -f PCMCIA_CHAR_MODULES ]; then inst_mod PCMCIA_CHAR_MODULES pcmcia; fi; \ - if [ -f PCMCIA_SCSI_MODULES ]; then inst_mod PCMCIA_SCSI_MODULES pcmcia; fi; \ - \ - ls -1 -U *.o | sort > $$MODLIB/.allmods; \ - echo $$MODULES | tr ' ' '\n' | sort | comm -23 $$MODLIB/.allmods - > $$MODLIB/.misc; \ - if [ -s $$MODLIB/.misc ]; then inst_mod $$MODLIB/.misc misc; fi; \ - rm -f $$MODLIB/.misc $$MODLIB/.allmods; \ - ) +.PHONY: modules_install +modules_install: _modinst_ $(patsubst %, _modinst_%, $(SUBDIRS)) + +.PHONY: _modinst_ +_modinst_: + rm -rf $(MODLIB)/kernel + mkdir -p $(MODLIB)/kernel + +.PHONY: $(patsubst %, _modinst_%, $(SUBDIRS)) +$(patsubst %, _modinst_%, $(SUBDIRS)) : + $(MAKE) -C $(patsubst _modinst_%, %, $@) modules_install # modules disabled....

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Mon Jul 31 2000 - 21:00:32 EST