[RFC] Proof-of-concept for netlink listener for mdev -i

[Natanael Copa]

Harald,

On Fri, 13 Mar 2015 10:04:57 +0100
Harald Becker <ralda at gmx.de> wrote:
...

> You hacked a solution, for the mechanism of your preference, throwing 
> out those who want to use one of the other mechanisms ...
> this is forcing others to do it your way?

The RFC in subject means "request for comments", not "I force you do it
my way".

> ... and what is the major difference of your hacked code and my intended 
> solution?

My way does not use anonymous pipe instead of named pipe? (no mkfifo
call)

(I now realize the pipes are not necessary at all - more on that below)

> > Instead of just using words to tell how to make a, express it with
> > example code.
> 
> Ok, example code, which show some of the work required for the intended 
> methods, but lacking other ...

The example was to show *one* way to solve a problem I am interested
in. Not all possible ways to solve it - nor was the intention to
provide a general solution for all kinds of problems. (I'm still not
sure we try solve same problem)

> and then you start adding and modifying 
> your hacked code, before even finishing the planning, and even before 
> finishing the gathering of the required functionalities? ... Shruck! ... 

I did trivial fixes based on comments. I am sorry if that offended you.

> I prefer doing it the way vise versa.

I am not trying to stop you do it vise versa.

Are you trying to force me to discuss ideas your way?

> ... you did nothing more, than putting together some well known code 
> examples, for something incomplete ... I already pushed similar code 
> examples into the trash can ... sorry, don't see any / much usage of this :(

I'm not trying to force you use it. I will not be offended if you put
my ideas in your trashcan, or add my email to your spam filter.

> >> What was your intention to do that code hacking?
> >
> > To show that a long-lived netlink daemon can be very small and simple -
> > probably much smaller than a fifo manager would be.
> 
> LOL ... sorry, but can't resist ...
> 
> ... you clearly does not understand the purpose of that "fifo manager": 
> Have you ever used supervisor daemons like tcpsvd, which accept incoming 
> TCP connections to fire up a handling server process? This is the job of 
> the "fifo manager", fire up a parser / device handler process, when it's 
> required ... and it is required because kernel spwaned hotplug helper 
> otherwise can't access the pipe to deliver the event to the handling 
> process, or still need to spawn a full parser / handler for each event. 
> So what is the benefit of your code snippet, other than forcing other to 
> do it your way?
> 
> 
> > If someone have a better idea how to solve it, I would love to hear it.
> > (No, I don't fifos will make it smaller/simpler)
> 
> What? You use pipes and say fifos will suffer? You clearly did not 
> understand the operation of named pipes (aka Linux fifos)?

I think you have a point here though. It should be possible to solve it
without pipes/fifos at all.

> Have you ever thought of the problem, how a separate spawned process can 
> access and write something into a pipe? The Unix solution for this are 
> named pipes (fifos), that is separate spawned processes may get access 
> to the pipe by just opening the name in the files system, receiving the 
> pipe descriptor ... so what is the difference?
> 
> One may be my decision to split even more functionality into smaller 
> modular blocks, instead of copying there code snipped into different 
> programs.
> 
> You will need the logic to fire up a process to handle the event, and 
> react on failure exit. Or you need to let that process stay in the back 
> forever. Right?
> 
> My intended "fifo manager" does exactly this, beside creating the named 
> pipe. That's it, a supervisor comparable in it's function to tcpsvd ...
> 
> ... and may be it is a good idea to add this functionality as a separate 
> applet into Busybox, where other functions (e.g. Natanaels modprobe 
> wishes, and others) may benefit from same functionality:
> 
> e.g. (not a code example, but a usage example)
> 
> usage:  fifosvd [-e ERROR_SCRIPT] FIFO_NAME PROG [ARGS]
> 
> Create the named pipe FIFO_NAME, wait for any other process writing 
> something into this pipe, then fire up PROG with given ARGS connecting 
> it's stdin to the read end of the fifo (stdout to /dev/null, stderr to 
> same as the one of fifosvd). When PROG dies, check the exit status and 
> fire up ERROR_SCRIPT with name of PROG, exit status or failure reason, 
> and a count of successive failures.
> 
> In addition fifosvd should forward signals SIGINT, SIGQUIT, SIGHUP, 
> SIGTERM, SIGUSR1, and SIGUSR2 to the currently running PROG, or just die 
> on SIGTERM otherwise (ignoring the others).
> 
> ... and no, this is not an extra thing. This *is* the function of the 
> intended "fifo manager", with the benefit of splitting process fire up 
> code from possibly different programs, replacing it with a single and 
> simple open of FIFO_NAME for writing.
> 
> ... and (as an idea) when FIFO_NAME is dash, use stdin of fifosvd as 
> pipe (skip fifo creation), but still wait until any data arrive in the 
> pipe, then fire up a PROG, respawning on more data, until write end of 
> pipe get closed ... and you gain a dynamic pipe manager:
> 
> ANY PROG | fifosvd - CONSUMER_PROG

Ok. This time it does not sound as stupid as I originally though.

It should be possible to solve the hotplug problem by setting up
netlink listener, wait for event, when event arrives fork helper and
just hand over the netlink socket filedescriptor to the child. That way
we avoid pipes/fifos alltogehter. And we avoid the splitted messages
problem too.

That would also make it possible to replace the netlink socket with a
named pipe for those who wants that.

In theory, the netlink socket (or named pipe) could be set up of a
separate process. That way we avoid the init code in memory of longtime
running process. Not sure its worth it thoug.

> Will delay start of CONSUMER_PROG until data arrive in the pipe, and 
> allows the consumer process to exit when idle (timeout needs to be added 
> to consumer, rest is done by fifosvd) ... think of running a shell 
> script with notable memory consumption of the shell, only running when 
> data is send to the pipe, otherwise freeing resources:
> 
> #!/bin/sh
> while read -tTIMEOUT line
>    do case "$line" in
>      ...
>    esac
> done
> exit 0

I think we agree on the design of consumer process. earlier presented
as mdev -i.

> --
> Harald
> 

Thanks for your comments! They have been useful even if I don't agree
with everything.

-nc