mdev -d can (silently) die with "read: no buffer space available"

Jan Klötzke jan at kloetzke.net
Sat Dec 14 13:49:20 UTC 2019


On Sat, Dec 14, 2019 at 06:47:43PM +1000, Alexander Zangerl wrote:
> the mdev -d daemon mode introduced in 1.31.0 works pretty well on
> my (systemd- and udev-less) box, but i've hit a few minor issues:
> 
> 1. after a few days the mdev daemon tends to die.
> 
> this is on a laptop that suspends and resumes relatively frequently, and
> there's a fair bit of uevent activity.
> 
> i've run mdev -df for a while and found that it eventually dies with
> "read: no buffer space available", ie. in daemon_loop().
> 
> as far as i can tell the default 2kb buffer allocated for
> reading from the netlink fd doesn't always suffice.
> in my build i've upped this BUFFER_SIZE to 64kb, and haven't had any
> mdev crashes since.

Just to double check: did you increase BUFFER_SIZE or RCVBUF? I'm
asking because the error that you observed (ENOBUFS) is returned by the
kernel if the socket ran out of buffer space, that is mdev did not read
the socket fast enough. OTOH if BUFFER_SIZE would be too small then the
event would just be silently truncated.

Looking at the bigger picture I think the error handling is sub-par at
the moment. In high load situations it could always be the case that
mdev does not keep up reading the events. There should be a fallback to
re-synchronize instead of dying silently. I'll see what I can do to add
such a proper error handling.

> 1. mdev in background daemon mode does die utterly silently.
> 
> even though i built busybox with CONFIG_FEATURE_SYSLOG=y
> mdev doesn't set logmode so it seems that that remains set to
> LOGMODE_STDIO but after bb_daemonize_or_rexec() both stdout and stderr
> are dup'd to /dev/null.
> 
> i think that logmode = LOGMODE_SYSLOG; should be added after the
> bb_daemonize_or_rexec call, or at least before the
> bb_perror_msg_and_die() call in the daemon_loop().

I fully agree here. I'll look into this too.

> 2. /dev/mdev.log is ignored unless it exists before the daemon starts,
> and is completely ignored for mdev actions related to initial_scan().

This has always been this way. I think it makes sense for a plain "mdev
-s" which is a synchronous invocation. But for "mdev -d" it might indeed
be better to write everything to mdev.log, including the initial scan.

> i think open_mdev_log() should be called in daemon_loop, not just
> after initial_scan() and bb_daemonize_or_rexec().

That would incur a considerable syscall overhead for the regular case
where mdev.log does not exist. Then mdev would try to open mdev.log for
every event it receives.

> that you can't create an mdev.log on demand, just before debugging
> some uncooperative device makes debugging a bit harder than necessary;
> it might also be good if the docs explicitly state the (reasonable) fact
> that mdev -d reads mdev.conf only once at startup.

That should indeed be documented. Maybe adding SIGHUP handler that
re-reads mdev.conf and SIGUSR1 to (re-)open mdev.log is a good idea...

Regards,
Jan

> regards
> az
> 
> 
> -- 
> Alexander Zangerl + GPG Key 2FCCF66BB963BD5F + http://snafu.priv.at/
> Then I dream of a world where idiots are hunted like wild pigs. 
>  -- Stephen Edwards



> _______________________________________________
> busybox mailing list
> busybox at busybox.net
> http://lists.busybox.net/mailman/listinfo/busybox



More information about the busybox mailing list