Busybox and standards/direction
rob at landley.net
Sun Dec 11 19:40:51 UTC 2005
On Sunday 11 December 2005 07:10, Glenn McGrath wrote:
> Trying to state the issues as i see it...
> Standard always exists, its just "standard" behaviour, peoples
> behaviour changes over time, so for any definition of a standard to
> have meaning it needs to change also.
Formal standards and de facto standards.
> When trying to _define_ a standard there will always be people who
> think this should be in the standard or that shouldnt be in it. So
> multiple standards can be defined at the same time targeting different
Standards should document, not legislate. "This is the existing best
practice" is a good thing. "We think you should do it this way" is seldom a
good thing to come out of a committee.
> Busybox's demographics has been those wanting a minimal runtime
> environment, and to a much smaller extent a minimal build environment.
I see three definitions of "minimal".
1) Smallest possible binary.
2) Smallest run-time memory usage.
3) Least complexity.
They're weighted in approximately that order, but that doesn't mean that lower
numbers always win over higher numbers. Doubling the complexity to save five
bytes out of a 12k applet probably isn't a good trade off. So the question
is how to balance them.
My objection here isn't to your recent checkin itself, it's that I'm trying to
figure out where the trend goes from here. Extra config knobs have a
complexity cost for people trying to build busybox. What's the best way to
balance that cost off against the gain? (Which means, step 1, define the
gain so we know what the goal is here.)
> AFAIK GNU is backward compatable with SuS, so SuS is definetly a good
> standard to aim for, it caters to our demographics.
Agreed. What is _in_ SuS, we can assume is likely to be used.
> As much as we aim for SuS we do need to be able change the behaviour of
> specific applets so users can have a system suited to them, wether its
> GNUism, old SuS behaviours or whatever.
Yup. "In theory, there is no difference between theory and practice. But, in
practice, there is."
> I guess im not a pragmatist, because i dont see the relavence of how
> cheaply we can implement default functionality that shouldnt be default.
My objection isn't to what is and isn't default. (The defaults that interest
me are generally "make allyesconfig" and "make allnoconfig". Those are the
two ends people are likely to start from.) My objection is really the
granularity of configuration.
I'm not saying it's a _bad_ idea to make a #config option that removes this
stuff. I'm saying it's a bad idea to confront anybody trying to configure
busybox for the first time with a half-dozen separate #config options for
every single applet. Is there a larger strategy this fits in, and if so
should that strategy have a master switch?
(I'm not even against having a half-dozen config options per applet someday,
as long as it's hidden behind a master CONFIG_NITPICK switch so most people
don't have to deal with it.)
If you're interested in going down the road of disabling extensions (more
power to you), there should probably be a CONFIG_EXTENSIONS switch to group
them together somehow. And that requires some thought and design work before
we can figure out how to do that (the real _meat_ of the size savings is
likely to be in disabling gnu extensions, since there are more of those than
obsolete features), and that very much makes it a 1.2 issue. :)
And this is _adding_ complexity to potentially reduce size, so we need to
think about the tradeoffs a bit in order to make intelligent decisions here.
> But the the real question isnt what you or i think should or shouldnt be
> in there, but how do we give users the ability to customise
> functionality without creating an overly complex configuration system ?
We're on the same page. :)
> We cant give users more choices of behaviour without asking more
> questions... so i think we have to accept busybox is always going to be
> complex to configure.
Yes and no. If we group options intelligently, it doesn't have to be.
Look at it this way:
Some users want to use busybox to replace entire packages, so they're thinking
in terms of coreutils, util-linux, vi, and so on. What _they_ want is to
look at just the top-level menus and "select all" so they can slap something
together really quickly. (The ability to then switch off a few obviously
unnecessary things in those menus is a bonus. Right now we have make
allyesconfig, but not "select all in menu".)
Some users want to select a group of specific applets (audit some script to
see everything it calls and enable that list of functions). And they just
want those applets to work, so they probably want all the functions in that
Some are trying to be as small as possible, and will use every config option
we give them.
And some are hand-tailoring a system that is on such a tight budget that
they're willing to spend all day figuring out how to save another 153 bytes
to squeeze it down into the next size rom, and will go into our source code
to chop out stuff that we never made configurable.
Group #1 currently either goes through menus and selects everything (which
gets a bit tedious) or does a "make allyesconfig"and goes through the menus
they don't want switching stuff off (which is faster: they don't have to deal
with sub-features that way, but this is entirely an indiosyncrasy of our menu
Group #2 is probably the majority. They start from "allnoconfig" and work
their way up, and are decently served right now unless they want to configure
something like "tar", which has a distracting number of config options. If
everything was like tar, our config menus would be quicksand for these people
since the "allyesconfig and trim" thing that lets them easily ignore
subfeatures doesn't help them.
Groups #3 and #4 aren't that far apart to me. Beyond a certain point, us
giving them more config features is probably _less_ convenient for them than
just pointing them at clean source they can chop irrelevant bits out of.
We'll never be able to guess what strange uses our code will be put to, and
trying just makes it a mess for the 99% of people who aren't going to do
I don't know where the tradeoffs are. My mental rule of thumb has been that
if a config option saves less than 100 bytes and only a single-digit
percentage of our users would ever want to disable it, it's probably not
worth the effort of offering a config option for that. We're often _better_
off making our C code as clean as possible so people can tailor it to their
needs more easily.
That said, one nice thing about the move from CONFIG_BLAH to ENABLE_FEATURE is
that the C code is much cleaner, and can handle a higher level of feature
removals. (The config menus may not be able to, but the C code can.) On a C
code level, the ENABLE_ tags can even be good documentation of what's easy to
rip out, where a corresponding amount of #ifdefs would make the code
The issue is then scaling the config menus to handle a higher number of
sub-features, without overwhelming people who just want to slap together
"mount, sh, sed, find, ls, tar, and cat" for an initramfs. And thus having
"CONFIG_SUBFEATURES" in the general options menu that hides all the applet
sub-features when it's enabled (and stuff like CONFIG_EXTENSIONS and
CONFIG_NITPICK under CONFIG_SUBFEATURES to control how _much_ granularity you
want, or perhaps automatically select or deselect certain categories of
features (CONFIG_SUSV3_ONLY)) makes sense to me.
But again, a 1.2 issue.
> Perhaps one thing we could do is group all the SuS utilities together
> into their own menu, that way any generic configure options related to
> obsolete SuS behaviour (e.g numeric arguments) would be isolated... not
> sure if that would make any difference.
Mirroring the packages people get these things from elsewhere has a certain
appeal, but then again that could just be one configuration option.
One approach that has a certain attraction to me is to put every single applet
together in one big menu, alphabetically, and then have another menu with
selection items for categories, which we could set to Y, N, or M. (Which to
_us_ would mean Yes, No, Maybe.) Y means select everything in that category,
N means hide everything in that category, and M means make everything in that
category show up for individual selection in the big list...
We have several different options...
> It would be great to have a testsuite that would check standards
> conformance, but i suspect there owuld be years of work in developing
> it. Ideally such an effort would be supported by other utilties
> projects as well.
I'm working on it in the 1.2 timeframe. I think that "uniq.tests" is there,
sort is more than halfway, I have about the first 1/3 of such a test for
sed... I was more worried about getting the testing infrastructure right,
and now I'm focusing on shipping 1.1, but this _is_ a priority.
The hard ones are the ones requiring root access (mount, losetup, the
attribute preserving portions of tar), but the new test infrastructure is
basically just a big shell script so I can put "[ `id -u` -ne 0 ] && echo
"Skipping root-only tests" && exit 0" before anything requiring root access
(or a similar if statement around it).
I'm also making it so the busybox tests aren't dependent on busybox. We can
run them against the gnu stuff as well to see how we match up. (That's an
important part of the new testing infrastructure.)
Steve Ballmer: Innovation! Inigo Montoya: You keep using that word.
I do not think it means what you think it means.
More information about the busybox