RFD: One-shot table driven system setup

[Harald Becker]

Hi !

*Intention:*

My original intention was (beside geting netlink operation), to add some 
extensions to mdev, to have the ability to do a simple table driven 
one-shot startup of the device file system.

The same extensions would also allow to bring up the other virtual file 
systems, at no extra cost, but let Laurent correctly state, that this 
would mix functionality into mdev, logically not belonging to the device 
file system.

So the decision was to split the intended initial function (xdev -i) and 
build the table driven startup feature into it's own applet (confsh), 
independed from any used device file system management.

This also allows to extend the intended functionality, without mixing 
logical functionality, bringing this to a hopefully general purpose 
method for easy table driven setup operations.


*Primary goal:*

The primary idea of confsh is simplification of the system setup, not 
replacing one type of complexity with a different one. Where the novice 
admin shall get the possibility to setup a simple system in an easy 
manner, and the expert shall gain from simplicity for the default things 
done a thousand times, but still have the ability to insert hookups 
(call helper scripts) to do any special kind of operation he likes to do.

For this approach the focus lies on description on what is required, has 
to be done or setup in *your* system (table of descriptive lines), 
instead of the usual shell script approach, describing more how this 
operation is done (all the commands to achieve the result).

Anyway shall confsh *not break* the KISS principal, Busybox is 
following, *not wasting* resources otherwise useless, and *not build* 
any specific system setup operation into the binary code.


*Planned operation:

The planned operation is a shell pre-processor, which read a specified 
configuration file for a table driven system setup operation, create a 
shell script and pass (pipe) that to /bin/sh (or may be better $SHELL 
with default /bin/busybox ash ?). This approach allow table driven 
operation for general kinds of environmental setups, either system or 
application related.

The lead in code send to the shell, instructs that to read a 
configurable script file (setup by system maintainer, example provided 
in distribution), defining the required shell functions for the 
operation of confsh. The generated script code will never do any 
specific system operation, but pre-parse and check the conf file format, 
then create shell friendly function calls with a constant order of 
arguments.

e.g. generating shell script code for a mount line from table format
(the brackets mean optional):

MOUNTPOINT [OWNER[:GROUP] [MODE]] @ FSTYPE[:DEVICE] [OPTION] [#PASS]

=> create this shell script line (passed to shell via pipe):

cf_mount MOUNTPOINT OWNER GROUP MODE FSTYPE DEVICE OPTION [PASS]


where cf_mount() is defined in confsh-script.sh:

# cf_mount - create mount point and mount given file system
#   $1 = MOUNTPOINT
#   $2 = OWNER or ''
#   $3 = GROUP or ''
#   $4 = MODE or ''
#   $5 = FSTYPE
#   $6 = DEVICE or ''
#   $7 = OPTIONS or ''
#   $8 = fsck PASS number or absent
#   (giving a pass number include this entry in a created fstab)
cf_mount()
{
   ... do all required operation to create the mount point
   ... set owner, group, mode, and mount the file system
   if [ -n "$8" ]
     then ... add a line to variable CF_TABLE_FSTAB
   fi
}


*More complete example:*

/etc/devfs-setup.conf:
(this what the system maintainer write)

---cut-here---
#!/bin/busybox confsh
# (confsh may be used as script interpreter)
# (or called: /bin/busybox confsh CONF_FILE_NAME [ARGS])

# ... here may go other setup stuff

# setup the initial device file system:

# (create empty mount point with owner, group, mode)
# (explicit creation of mount points allow to setup any permission)
/dev  root:hotplug  0751 @

# (mount the file system with the given options)
# (if mount point does not exit, try auto create with root:root 0751)
/dev  root:root  0755  @  tmpfs relatime,size=10240k

# (create empty mount points in the device file system)
/dev/pts     root:root  0751 @
/dev/shm     root:root  0751 @
/dev/mqueue  root:root  0751 @

# (mount the virtual file systems in /dev)
/dev/pts     root:root    0755  @  devpts  #0
/dev/shm     root:root    1777  @  tmpfs   #0
/dev/mqueue  root:mqueue  1777  @  mqueue  #0

# create devfstab table
write fstab /var/run/config.d/devfstab root:admins 0644

# ... more stuff to setup
---cut-here---


=> shall create a shell script to invoke the desired operations, which 
also write the file /var/run/config.d/devfstab (with perm root:admins, 
mode 0644):

(this file is create due to write fstab command):

---cut-here---
# *** auto generated file, don't edit, for changes see ... ***
virtual /dev/pts devpts defaults 0 0
virtual /dev/shm tmpfs defaults 0 0
virtual /dev/mqueue mqueue defaults 0 0
---cut-here---

Note: The /dev virtual file system is not included in the devfstab file, 
as this mount line is not given with a fsck pass number at the end of 
the line (intentional as an example).


Rough approximation of script passed to the shell (details are a bit 
finer, and without the comments):

(this is created by the operation of confsh binary):

---cut-here---
# load the cf_* shell functions
. confsh-script 'CONF_FILE_NAME'
while :
  do
   ...

   # the /dev file system
   cf_mountpoint /dev root hotplug 0751
   cf_mount /dev root root 0755 tmpfs 'relatime,size=10240k'

   # other file systems in /dev
   cf_mountpoint /dev/pts root root 0751
   cf_mountpoint /dev/shm root root 0751
   cf_mountpoint /dev/mqueue root root 0751
   cf_mount /dev/pts root root 0755 devpts '' 0
   cf_mount /dev/shm root root 1777 tmpfs '' 0
   cf_mount /dev/mqueue root mqueue 1777 mqueue '' 0

   # write the devfstab file
   cf_write fstab /var/run/config.d/devfstab root admins 0644

   ...
   exit 0
  done
---cut-here---


*Attention:*

The intended operation, and details, are nevertheless complete. I'm 
currently working on a proposal of the conf file format, trying to find 
an intuitive syntax, but at least easily readable, still giving the 
ability to configure and trigger invocation of most of the usual system 
setup operation. Where my start point for the syntax the base format of 
mdev.conf is. Suggestions for possible setup operations and the intended 
syntax, are highly appreciated.


*Functionality:*

The following functionalities for table driven setups are intended (List 
and details as base for discussion):

- allow including of other conf files or all files from a directory

- emit messages on stderr
   (usually the console)

- file operations
   (touch, permissions, move, copy, remove)

- directory operations
   (create, permissions, move, copy, remove)

- symlink operations
   (create, remove)

- mount point operations
   (create, permissions, unmount, remove)
   (I like to auto create an empty file of name "not mounted" with mode
    0000 in empty mount point directories, as a novice human visible
    indicator; this is pure preference, but all such things can be done
    by logically separating mount points from other directories)

- mount operations
   (auto create, mount, permissions, fstab creation)

- easily setting sysctl values
   (write value to /proc/sys/...)

- loading a keyboard table

- loading the console font

- auto creating or replacing configuration files
   (fstab, mdev.conf, ...)
   (what seams to be interesting to be setup from central place?)

- wait for accessibility of a specific device / file
   (wait until device / file exist and is readable)
   (with a possible timeout)

   can be checked with (?):
     [ -r DEVICE ] && dd if=DEVICE of=/dev/null bs=1 count=1 2>/dev/null

- wait for accessibility of a specific file system (LABEL/UUID)
   (use blkid to get device for LABEL/UU1D, then wait for device)

- loading kernel modules
   (invoke modprobe with given module names)
   (this not for module hotplug, but for manual module loading)

- removing kernel modules
   (trigger removing of specified modules)

- invoke (source) other shell scripts with arguments
   (for the more novice usage of functional extension)

- embed any snippet of shell commands into output piped to shell
   (for the experts usage)
   (the shell snippet shall go inside a sub shell '('...')')


*This list is not complete*, let me know what you are missing!

*Please* keep in mind: My intention is to create a shell pre-processor 
and a shell script with default implementation of such operations, not 
to reinvent or creation of a new shell, or to implement all such 
operation details in binary code. Only the table parser, format checks, 
slight sanity checks, and creation of a minimum for shell function 
invocation is to be done in the binary code. The remaining operations 
are controlled by the called shell functions (example implementation 
will be provided, but may be modified to your likeness).

Anyway: Using the preprocessor approach allow to benefit from shell 
substitutions and expansions ($ENVVAR, etc.), without reinventing such 
things.

(Thanks to Isaac and Laurent, for pre-release suggestions to this.)

--
Harald