RFD: Rework/extending functionality of mdev

Harald Becker ralda at gmx.de
Wed Mar 11 13:30:13 UTC 2015


Hi Natanael !
Hi Isaac !

Looks like you misunderstand my approach for the mdev changes ... ok, 
may be I explained it the wrong way, so lets go one step back and start 
with a wider view:

IMO Busybox is a set of tools, which allow to setup a working system 
environment. How this setup is done, is up to the distro / system 
maintainer, that is it is up their preferences.

I really like the idea to have an optimized version using netlink for 
the hotplug events and avoid unnecessary forking on each event, but 
there are other people which dislike that approach and prefer to use the 
hotplug handler method, or even ignore the hotplug feature completely 
and setup device nodes only when required (semi automatic, not manual 
mknod, but manual invoking of mdev).

The world is not uniform, where all people share the same preferences, 
so we need to be polite to accept those different kinds of preferences 
and don't try to force someone to setup their system in a specific way.

Right? ... else we would be at the end of discussion and the end of my 
project approach :(

... but I think you will agree:

As Busybox is the used tool set, it shall provide the tools for all 
users, and shall not try to force those users to use netlink, etc.

So how can we handle different preferences, but still allow each one to 
setup there system the way he likes?

We either need different versions of the commands / applets and allow to 
enable / disable them in the configuration, or we try to find a modular 
solution, which allows a maximum of code sharing, but still give the 
flexibility to setup the system up to the user preferences.

The config approach may be the simpler solution, but will add more and 
more options to the config system, and needs to do all those decisions 
before you have a working system. So many users neglect to build there 
own Busybox version (for different reasons) and grab a prebuild binary 
version (or are forced to stay on a specific version) ... but still all 
those users will have there own preferences on how to setup their 
system. The conclusion is, to have a modular tool set, which on default 
include all functionality and allow to select the desired usage on 
system setup. Still with the wish to minimize resource usage and a 
maximum of code sharing between the different functionalities.

Beside having an universal modular tool set, with all functions 
included, it may still be a good idea to allow disabling unwanted 
applets / functionalities, for those who are picky and / or low on 
system resources.

... using this wider view, I tried to find a modular solution for the 
device system management (here mdev):

Maximum code sharing means to have a look on flow of data and try not to 
duplicate same functionality into different commands. For the device 
management I see the following different usage scenarios:

- not using hotplug feature, semiautomatic device node setup
   (currently user calls "mdev -s" when required)

- using the hotplug handler approach of the kernel
   (current operation of "mdev")

- using a netlink based hotplug approach
   (currently not in Busybox, but external tools exit, e.g. nldev)


The current implementation suffers on event bursts, due to massive 
forking a separate parser for each event. So one of the major design 
decisions has to be avoiding those unnecessary parallelism. It does not 
only suffer from resource consumption, the parallelism is contra 
productive for device event management, due to not serialized operation.

So how can we avoid that unwanted parallelism, but still enable all of 
the above usage scenarios *and* still have a maximum of code sharing 
*and* a minimum of memory usage *without* delaying the average event 
handling too much?

The gathering parts need to acquire the device information, sanitize 
this information, and serialize the event operations to the right order. 
The device node handling part shall receive the information from the 
gathering part(s) (whichever is used) and call the required operations, 
but shall avoid reparsing the conf on every event (speed up) *and* drop 
as much memory usage as possible, when the event system is idle.

My idea is a fifo approach. This allows splitting the device management 
functionalities. Nevertheless which approach to gather the device 
information is used, the parser and device handling part can be shared 
(even on a mixed usage scenario).

So we have the following functional blocks for our device management:

- initial setup of the device file system environment
   (yes, can be done by shell scripting, but it is a functional block)

- starting the fifo management and automatic parser invocation
   (long lived minimalistic daemon)

- manual scanning of the sys file system and gathering device info

- setting up the usage of the hotplug helper system
   (check fifo availability and set hotplug helper in kernel)

- an hotplug helper spawned by the kernel on every event
   (should be as small / fast as possible)

- a netlink based event receiptor
   (long lived daemon, small memory foot print)

- the device node handling part
   (conf table parser / calling required operation)

Where the gathering parts may be used according to the user preferences 
(and may be opted out on BB configuration).

Every gathering part grabs the required information, sanitizes, 
serializes and then write some kind of command to the fifo. The fifo 
management (a minimalist daemon) starts a new parser process when there 
is none running and watches it's operation (failure handling). The 
parser process reads the conf file on startup, then start to read 
commands from the fifo (written by a gathering part). For each device 
command (event) the in memory table (build from the conf) is scanned and 
required event operations are invoked. The parser process can (has to) 
exit and free all it's memory usage, when the event system is idle for 
as some time. The fifo management has to catch this and restart a new 
parser when more events arrive.

Adding initial setup of the device file system as a functional block to 
the device management, allows to arrange for having a one-shot device 
system startup. Consider emergency shell usage, with a minimalist system 
started in memory (e.g. starting kernel wit /bin/sh as init). Then you 
need to do some base setup get a usable system, with bringing up the 
device management of one central part of operation. Then you would most 
benefit from a one-shot startup. At startup which is *not* hard coded in 
it's operation, it is just controlled by a configuration file, focusing 
on description on what is to create and not on how it is created (as in 
shell scripts). The finally invoked commands for this initialization 
should be this way nearly the same as most existing shell scripts for 
system startup doe ... and no one blocks this shell script approach (let 
out the special setup operation from conf files and even when calling 
the init function - which can be opted out, it will be a no operation 
call, not affecting surrounding shell script handling). Adding the 
initial setup functionality is for those who like the idea of having 
such a one-shot startup, but otherwise does not affect the device 
handling management (optional feature under full control of user, no 
hard coded setup in a binary).

Adding the hotplug helper setup to the list of functional blocks of the 
device management, is on one hand kind of completeness, and on the other 
side also part of the one-shot-startup approach (Not every average admin 
remember the exact names where to put each setting, so just adding a 
single command or adding an single option to a one-shot command call 
simplifies usage, at least for the more novice).


Not (yet) included in this description is the module loader 
functionality (MODALIAS handling). Have not got around to think on the 
required details, but should be possible doing in a similar approach.

The following questions to this topic:

- are MODALIAS requests send to same netlink socket as device events?

- does the module handling need and / or benefit from synchronization 
with the device node handling, or is choosing parallelism the better 
approach?


Later on, I will pick up your messages again to comment on some more 
specific points, but in general I hope this clarifies the intention of 
my approach ... was my mistake to start the wrong way from behind the 
back to the usual beginning (or at least not to allow you to follow my 
initial thoughts) ... I apologize.

--
Harald



More information about the busybox mailing list