RFD: Rework/extending functionality of mdev

Thu Mar 12 01:05:15 UTC 2015

On Wed, Mar 11, 2015 at 09:55:39PM +0100, Harald Becker wrote:
> Hi Isaac !
> 
> > Agreed, whole-heartedly.
> >I just don't think you're quite thinking through exactly what this means.
> 
> In which sense? Let me know, what I got wrong. I'm a human, making mistakes,
> not a perfect machine.

It seems like you want to force everyone who uses mdev to use a multi-part
setup.
Specifically, you are proposing to *replace* the standard hotplugger design
(where the kernel spawns one program that then sets up the device itself)
with a new design where there's a hotplugger that submits each event to
a longer-lived program, which in turn submits the events to a program
that creates/sets up the devices.

I am saying, don't force this design on people who want the hotplug helper
that the kernel spawns to finish things up.

> >Agreed.
> >But I would include "hotplug daemons" under "etc."
> 
> I used "etc." so add any similar type of program you like, including
> "hotplug daemons" ... but stop, what are "hotplug daemons"? Do you mean
> daemons like udev? udev uses netlink reading. Otherwise I know about hotplug
> helper programs (current mdev operation), but not about "hotplug daemons".

That's the best description I can come up with for the FIFO reader that
you proposed that would read from a fifo and write to a pipe, monitoring
the state of "mdev -p"

> >>The gathering parts need to acquire the device information, sanitize this
> >>information, and serialize the event operations to the right order. The
> >>device node handling part shall receive the information from the gathering
> >>part(s) (whichever is used) and call the required operations, but shall
> >>avoid reparsing the conf on every event (speed up) *and* drop as much memory
> >>usage as possible, when the event system is idle.
> >
> >That *shall* is where you're forgetting about the first principle you
> >mentioned (assuming that you mean "shall" in the normative sense used
> >in RFCs).
> 
> ??? Sorry, may be it's I'm not a native English speaker. Can you explain me
> what is wrong? How should it be?

The idea that the hotplug helper is *required* to fit into a redesign
where the parser runs as a different process from the backend and handles
multiple hotplug events is what I'm objecting to.

> >Yes, some people find the hotplugger too slow.
> >But that doesn't mean that everyone wants a new architecture.
> 
> What do you mean with new architecture? Different system setup? Changing
> configuration files?
> 
> My first approach was not to change usage of current systems. Except a
> slightly bigger BB, you should not have noted from my modifications. Then
> came Laurent with some questions and suggestions to split of some things for
> clarification, so the changes may result in slightly modified applet names
> and/or parameter usage (still under discussion), to be able to adopted all
> functionality ... but otherwise you won't need to change your setup, if you
> do not like.

roughly, architecture = design.
Not how the scripts/config files are written, but the fact that you're
changing "kernel starts hotplug helper, hotplug helper sets up device"
into "kernel starts hotplug helper, hotplug helper writes to fifo,
fifo reader supervises and writes message to tool that sets up device"

> >Some people, at some points in time, would prefer to use a plain, simple,
> >hotplugger, regardless of whether it's slow.
> 
> ??? Didn't you notice the following:
> 
> > - using the hotplug handler approach of the kernel
> >    (current operation of "mdev")
> 
> 
> >(Personally, I'd like a faster boot, but after that a hotplugger that
> >doesn't daemonize is fine by me.)
> 
> Do you really like forking a separate conf parser for each hotplug event,
> even if they tend to arrive in bursts?

Yes, depending on the circumstances (I assume that by "forking a separate
conf parser", you mean "letting the kernel spawn a new hotplug helper that
parses a config file").

*I* certainly like it better than spawning a queue submitter on each
hotplug event, which will then submit it to a supervisor, which will
then submit it to a device creator.
That's a good design for cron, but I've yet to be convinced that it's
appropriate for creating devices, and the proof is not in the concept
but how the code runs.

In short, don't expect people to buy into your design before it gets merged,
and don't expect it to replace mdev before everyone buys into your design.
Come up with an extra applet that lets people test it without replacing
a known-working mdev, and you might see a lot more people test it.

> Won't you like to get a faster system startup with mdev, without changing
> system setup / configuration?

Absolutely NOT without changing system setup/configuration.
This is where I see no room for negotiation.

Think about this:
Would it be nice if someone changed "mount" so that instead of parsing
/etc/fstab and mounting drives, it just submitted a job request to a
service that then supervised another service that uses a pre-parsed
fstab to determine which drive to mount?
What if modprobe and insmod did that?
Wouldn't it be a nightmare if you upgraded, mount or modprobe broke,
and you went to strace it and found that it's just writing messages
and you need to figure out where those are going/failing to go?

Even if it *did* work better, you *don't* make changes that large as a
transparent mandatory change, where one upgrades and all of a sudden,
it works completely different behind the scenes.

Your new code *will* fail at some point when dealing with a system
configuration that you never had access to or heard of, probably sooner
than later. That's the way Murphy works.
And if you make this happen without changing configuration, you make it
much harder to troubleshoot, because the person who first troubleshoots
it will likely try to use their knowledge of how things have always
worked.

Now, if you can do it faster by adopting this approach, people will
be willing to change their configurations to use it. It would be one word
to change to another hotplug helper, so they can trivially switch between
your proposal and mdev.

> >So, in order to respect that preference, it would be nice if you could
> >let the hotplug part of mdev keep doing what it does.
> 
> What expect you to be the hotplug part? The full hotplug handler including
> conf file parser, spawned in parallel for each event?

Yes, at least for mdev.

> Won't you like to benefit from a faster system startup, only why you fear
> there is another (minimalistic) daemon sitting in the back? Sounds like
> "automobiles are dangerous, I won't use them"? ... sorry if this sounds bad,
> I try to understand what exactly you are fearing ... I expect you did
> misunderstand something (or I explained / translated it wrong).

To use your analogy, I'm objecting to you upgrading all the carriages to
automobiles.
If automobiles are so much better, people can go buy them.
If your way of doing it is so much better, people can change the hotplugger.

Also, I suspect that if you multiply processes involved in handling one
event, you multiply the chances of having undelivered events.

> >But if you prefer to have it in busybox, I'd suggest doing a new applet
> >named (for example) "afifohp" (A FIFO HotPlugger, so named because it
> >would be one of the first entries in the multiplexer and that's what
> >it does).
> 
> Ugly name :( ... but otherwise please don't stick on specific names, they
> where examples and are under discussion. My concern is the functionality.
> Changing names and parameter letters is hacking code.

Fine. 
My main point is that what you propose for the default action of mdev
should be in a separate applet; a detail is that the second half
of what "mdev -h" does under your proposal becomes one of two paths
in the alternate hotplugger; if the daemon isn't running, the hotplugger
automatically starts it.

> >This would become the FIFO-reading daemon if need be, and dump its
> >environment into the fifo or (as daemon) read from the fifo, spawn
> >mdev -p, and write to a pipe.
> >I'd assume it needs a *simple* config file, which could be just a line
> >or two:
> >/sbin/mdev -p
> >timeout <n>
> 
> ??? <confused> :(

The first line specifies the hotplug parser to use (so if someone wants
to write a parser for udev rules or something else, they're able to use
the fifo daemon.

Rather than using mdev -h to set things up, it sets itself up from the
(extremely minimal) config file as soon as possible.

This is significant, since there are at least four ways to set the
hotplugger:
-In the kernel .config
-On the kernel command line
-Via /proc (also accessible via sysctl)
-Via /sys

So someone might not realize that they need to start the fifo-watching
daemon/set the hotplugger.

The "timeout" line might not belong there; it's based on the premise that
the fifo-watching daemon will time out and close a pipe to the parser if
nothing happens.

Thanks,
Isaac Dunham