RFD: Rework/extending functionality of mdev

[Harald Becker]

Hi Isaac !

On 12.03.2015 02:05, Isaac Dunham wrote:
>>> I just don't think you're quite thinking through exactly what this means.
>> In which sense? Let me know, what I got wrong. I'm a human, making mistakes,
>> not a perfect machine.
>
> It seems like you want to force everyone who uses mdev to use a multi-part
> setup.

Whops, you got that definitely wrong! Who told you, I want to force you 
to use a different setup. My clear intention, was adding some extra 
flexibility, without breaking existing usage.

... but criticism arose, which poked for more clarity. Only due to this 
and the wish to allow fitting as much preferences of people as possible, 
there might result a slight change: One extra operation may be required 
(when not not combined / hidden in the mdev system for clarity). At the 
location you setup the hotplug handler or do "mdev -s" you either need a 
slight change of the command parameters or need to insert a single extra 
command (details on this have not been discussed yet). But that's it.

Are you really concerned about inserting a single extra line in your 
startup scripts or having a modification to one of those lines? That way 
you would block any innovation, *and* the needs of other people.


> Specifically, you are proposing to *replace* the standard hotplugger design
> (where the kernel spawns one program that then sets up the device itself)
> with a new design where there's a hotplugger that submits each event to
> a longer-lived program, which in turn submits the events to a program
> that creates/sets up the devices.

You say I propose a change of the device management system?

The mdev system suffer on event bursts and this has been grumbled about 
several times in the past. What I'm now trying is to build a solution 
for this into Busybox. Not reinventing the wheel, but implementing known 
solutions.

> I am saying, don't force this design on people who want the hotplug helper
> that the kernel spawns to finish things up.

The only way to solve this, would be to let mdev as it is, and create an 
additional netlink-mdev, which brings us as at the situation where it 
gets complicated (or at least complex) to share code and work between 
both of them. Selection between both (when you don't want to include 
both hunks), needs to be done with BB configuration, which make all 
thinks like pre-build binaries a mess (how many different versions shall 
be maintained? Only yours? Only mine?).

To solve this conflict and to also give the current device management 
system some speed improvement on burst, I tried to find a solution. This 
solution centers on the most important problem, the parallelism and 
parsing the conf for each single hotplug event.

So I *propose* to split the function of "mdev" into two separate parts. 
Part one will contain the kernel hotplug helper part (and should be as 
small and fast as possible), and part two the parsing of the conf and 
device node operations ... with the requirement of communication between 
any number of running part ones and the single part two. To reduce cost 
on those "part one" hunks (remember they shall be as fast as possible), 
and the availability of failure management (which BB currently 
completely lack), the communication between the parts need to be watched 
by a small helper daemon. A very minimalist daemon, which doesn't touch 
any data, it just fire up the device operation stuff, when it's required 
and wait until this process dies. Then goes back to wait for more events 
to arrive (this not like udev).

Do I wish to overcome the suffering of Busybox device management? Yes.

Do this need some changes? Yes, I propose as one or two line change in 
your startup files, with the benefit of speed improvement.

Do I otherwise propose to change your system setup or flip to using 
netlink operation? *NO*

Hence, I do not force anybody to change from using the kernel hotplug 
feature. I improve, that mechanism ... with the ability to add further 
mechanisms, without duplicating code. One shall be a netlink reader in 
BB, the other may be external programs of others, with a better 
interface to use device management functions from there.

Are your really complaining against any kind of innovation? Any step to 
overcome long problems, discussed several times ... but dropped, mostly 
due to the amount of required work?

Do I *propose* some innovation? Yes.

Do I propose to force someone changing to different mechanism? No.

>>> Agreed.
>>> But I would include "hotplug daemons" under "etc."
>>
>> I used "etc." so add any similar type of program you like, including
>> "hotplug daemons" ... but stop, what are "hotplug daemons"? Do you mean
>> daemons like udev? udev uses netlink reading. Otherwise I know about hotplug
>> helper programs (current mdev operation), but not about "hotplug daemons".
>
> That's the best description I can come up with for the FIFO reader that
> you proposed that would read from a fifo and write to a pipe, monitoring
> the state of "mdev -p"

Wow? You got it wrong!

It won't be a FIFO reader, it will be a FIFO watcher, which is a big 
difference. That is in functionality comparable to the operation of 
tcpsvd, which opens a network socket waiting for incoming connections, 
accept them, fork a handler program and pass the socket of the incoming 
connection. This is a well known and proven practice on Unix like 
systems. And what is wrong with this?


>>>> The gathering parts need to acquire the device information, sanitize this
>>>> information, and serialize the event operations to the right order. The
>>>> device node handling part shall receive the information from the gathering
>>>> part(s) (whichever is used) and call the required operations, but shall
>>>> avoid reparsing the conf on every event (speed up) *and* drop as much memory
>>>> usage as possible, when the event system is idle.
>>>
>>> That *shall* is where you're forgetting about the first principle you
>>> mentioned (assuming that you mean "shall" in the normative sense used
>>> in RFCs).
>>
>> ??? Sorry, may be it's I'm not a native English speaker. Can you explain me
>> what is wrong? How should it be?
>
> The idea that the hotplug helper is *required* to fit into a redesign
> where the parser runs as a different process from the backend and handles
> multiple hotplug events is what I'm objecting to.

So you are complaining at the wrong location. The problem lies at the 
principle of the kernel hotplug feature, where the kernel spawns a 
separate process for each new event. I should only spawn *one* soch 
hotplug process and then send this process the events.

I didn't create that brain damaged method, but I have to live with it. 
So I'm trying to overcome some of the problems ... still with the 
possibility of using the hotplug mechanism (for those who dislike or 
can't use netlink), beside additional over all speed improvement.

... and what does you complain about "hotplug helper required"? How else 
other than starting a program, can you gather the information from the 
kernel when using the hotplug helper mechanism? The hotplug helper is 
nothing new, it is only the first part of code from current mdev, 
avoiding duplicating the second part for any further parallel arriving 
hotplug event.


> roughly, architecture = design.
> Not how the scripts/config files are written, but the fact that you're
> changing "kernel starts hotplug helper, hotplug helper sets up device"
> into "kernel starts hotplug helper, hotplug helper writes to fifo,
> fifo reader supervises and writes message to tool that sets up device"

So you prefer pure parallelism with known problems, over splitting the 
operation into separate threads, using a communication method?

And see above, it is not a FIFO reader. The extra daemon is just a 
process who fires up the device handling "thread" when that thread is 
not active, and may catch failures of that "thread".


>> Do you really like forking a separate conf parser for each hotplug event,
>> even if they tend to arrive in bursts?
>
> Yes, depending on the circumstances (I assume that by "forking a separate
> conf parser", you mean "letting the kernel spawn a new hotplug helper that
> parses a config file").

Then this means blocking any innovation, and forcing other people to use 
different software or do it the same way you like, if you are not 
willing do accept some slight modifications / big improvements.

> *I* certainly like it better than spawning a queue submitter on each
> hotplug event, which will then submit it to a supervisor, which will
> then submit it to a device creator.

One submit to much: The supervisor fire up a device creator if required, 
which processes the incoming events until no more to do. Splitting of 
that queue submitter also add some buffering, which allows the hotplug 
helper process to exit early (freeing system resources).


> In short, don't expect people to buy into your design before it gets merged,
> and don't expect it to replace mdev before everyone buys into your design.
> Come up with an extra applet that lets people test it without replacing
> a known-working mdev, and you might see a lot more people test it.

And I highly dislike that, as others too. I asked ahead to gather as 
much preference wishes and ideas as possible, the usual way of 
programming, not hacking. No one asked you or others to use untested 
stuff. The first step is planning, the second hacking, the third testing 
... then a patch can be offered for those who like to test ... and if no 
one more complains on those tests, it may be merged into the main stream.

... but I neglect having different versions of mdev, not talking about 
you are forcing me to change the name of a well known operation.


>> Won't you like to get a faster system startup with mdev, without changing
>> system setup / configuration?
>
> Absolutely NOT without changing system setup/configuration.
> This is where I see no room for negotiation.

???

> Think about this:
> Would it be nice if someone changed "mount" so that instead of parsing
> /etc/fstab and mounting drives, it just submitted a job request to a
> service that then supervised another service that uses a pre-parsed
> fstab to determine which drive to mount?

Beside your mistake about forwarding data a step to much, what would be 
wrong, if the overall operation of the mount system does not change 
(neither the mount call nor the fstab format change)?

> What if modprobe and insmod did that?

This is Natanaels idea and request, and could be a good practice to 
speed up module loading on system startup and device changes.


> Wouldn't it be a nightmare if you upgraded, mount or modprobe broke,
> and you went to strace it and found that it's just writing messages
> and you need to figure out where those are going/failing to go?

You are tracing mount or modprobe operation???


> Even if it *did* work better, you *don't* make changes that large as a
> transparent mandatory change, where one upgrades and all of a sudden,
> it works completely different behind the scenes.
>
> Your new code *will* fail at some point when dealing with a system
> configuration that you never had access to or heard of, probably sooner
> than later. That's the way Murphy works.
> And if you make this happen without changing configuration, you make it
> much harder to troubleshoot, because the person who first troubleshoots
> it will likely try to use their knowledge of how things have always
> worked.
>
> Now, if you can do it faster by adopting this approach, people will
> be willing to change their configurations to use it. It would be one word
> to change to another hotplug helper, so they can trivially switch between
> your proposal and mdev.

That is: New automobiles may be dangerous, don't even think about 
creating a newer one ... sorry, but I'm not able to understand your 
point of view :(


>>> So, in order to respect that preference, it would be nice if you could
>>> let the hotplug part of mdev keep doing what it does.
>>
>> What expect you to be the hotplug part? The full hotplug handler including
>> conf file parser, spawned in parallel for each event?
>
> Yes, at least for mdev.

Then I expect you, staying at current versions of Busybox, saving the 
current version of mdev, and replacing this version in may be newer 
versions ... that's more or less just the opposite of what you expect, 
that I do all that on a private basis.


> My main point is that what you propose for the default action of mdev
> should be in a separate applet;

Which mean doubling of code, as it can't share the code ... 
(provocative) why not just copying mdev ahead, renaming into 
mdev-hp-old, for those who are really picky about that usage. Giving the 
average users an improved, but compatible mdev implementation?


>>> This would become the FIFO-reading daemon if need be, and dump its
>>> environment into the fifo or (as daemon) read from the fifo, spawn
>>> mdev -p, and write to a pipe.
>>> I'd assume it needs a *simple* config file, which could be just a line
>>> or two:
>>> /sbin/mdev -p
>>> timeout <n>
>>
>> ??? <confused> :(
>
> The first line specifies the hotplug parser to use (so if someone wants
> to write a parser for udev rules or something else, they're able to use
> the fifo daemon.
>
> Rather than using mdev -h to set things up, it sets itself up from the
> (extremely minimal) config file as soon as possible.
>
> This is significant, since there are at least four ways to set the
> hotplugger:
> -In the kernel .config
> -On the kernel command line
> -Via /proc (also accessible via sysctl)
> -Via /sys
>
> So someone might not realize that they need to start the fifo-watching
> daemon/set the hotplugger.

You completely misunderstand, on how this will work!

--
Harald