mark obsolete usage of fold
rob at landley.net
Sun Dec 11 10:05:37 UTC 2005
Warning: philosophical differences ahead. You have been warned.
On Sunday 11 December 2005 02:29, Glenn L. McGrath wrote:
> On Sun, 11 Dec 2005 00:30:11 -0600
> Rob Landley <rob at landley.net> wrote:
> > On Saturday 10 December 2005 16:53, Glenn L. McGrath wrote:
> > > Attached is a patch sets a feature to allow 'fold -40' type arguments
> > > to fail, this behaviour is obsolete even in susv2.
> > >
> > > Im sure some scripts (e.g. configure) are still using it, so its
> > > usefull to support it.
> > >
> > > The patch doesnt change default behaviour. Its really just to start to
> > > organise things so we can one day have a global setting to choose what
> > > posix standard to compile to.
> > 1) If you're going to do this kind of thing, could you please have a
> > global argument to disable nonstandard extensions? (I'm trying to keep
> > the amount of configuration people have to do down to a dull roar.
> > Getting buried in options doesn't help. I'm thinking of adding things
> > like CONFIG_NEEDS_PROC and CONFIG_LONG_OPTIONS to 1.2, for example.)
> Ive commit some changes, i added 2 config options to Build Options, One
> for SuSv2, the other for pre SuSv2.
Both are "not Susv3". I don't care whether it's a gnu extension, solaris
compatibility, or something left over from system 5. We're not currently
fully implementing _any_ standard, and I believe the one we're trying to move
towards (and that I'd like to audit stuff against) is susv3.
And yes, there are cases where we'll still violate that standard. We're not
doing the full internationalization thing with locale environment variables,
_ever_ if I can help it, because it's just too much bloat for our mandate to
absorb. (UTF-8 seems like the low hanging fruit here. You followed those
> > 2) I'm trying to get 1.1 out by new year's. Could this kind of change
> > please wait until January? (See big long bug list posted a couple days
> > ago.)
> umm, woops, too late.
I saw before I posted this, and didn't complain. No biggie. :)
> > 3) I really think we should pick one standard we care about, and try to
> > implement all of. (Right now it's that Open Group thing that's on the
> > web, which is roughly susv3.) Trying to distinguish between susv2 and
> > susv3 is just going to drive us _crazy_.
> The files i just patched are some obvious ones, but we can also get
> information about changes between SuSv2 and SuSv3 from coreutils or
> maybe the Heirloom Project.
Something we need to do, totally separate from this, is audit all the apps to
see which ones are susv3 compliant and which aren't. In my head, this is
tied up with extending the test suites to _test_ all of susv3 (using the new
"all tests in one script" format so that the subversion history is actually
So I don't personally care what susv2 said, unless the spec we're getting off
the open group's website is susv2. (The one that all our URLs point to, like
the one in the comment towards the start of sed.c.)
> I think we need a compatability layer for when things break, e.g.
> configure scripts still use the numeric arguments in head/tail i think.
Yes, they due. If real world scripts would break due to the lack of numeric
arguments, we should be able to handle the numeric arguments. I really don't
care what the standard says if a 100% standards compliant implementation is
useless in the real world. To me, the standard is an important point of
reference, but not the whole picture.
For example, uClibc 0.9.28 won't install unless tar has the --exclude option.
(I had to fix this to get uClibc 0.9.28 to install with busybox tar.) You
still can't install uClibc 0.9.28 with busybox 1.01, and those are the most
recent official releases of each...
That's not the only gnu extension we need to be useful. Far from it.
The ./configure stage of all sorts of gnu extensions in sed and sort in order
to complete. (This gets silly at times: look in editors/sed.c for the
comment "Lie to autoconf". Yes, this is needed to make things like binutils
build correctly. It shouldn't be, but it is.)
What standard is bzip part of? Does this mean people should stop using it?
> We can try for the latests standard, but still have the old behaviour
> to fall back on.
This is a case where I consider the additional feature to be an extension. We
provide gnu extensions left and right, so where's the downside of providing
obsolete behavior as an extension?
There are several reasons for removing old behavior. Bloat is high on the
list for us, but there's also potential security issues, causing interference
with other functionality, and so on.
Removing it just _because_ it's old, however... why? We configure out the
old tar format because A) it saves space, B) almost nobody ever uses it so
it's not a feature there's much demand for. (And we do that because _both_
of those are true.)
If the argument starts with "saves x bytes", I'm all for it. If the argument
is "because people should convert their scripts"... Busybox provides
features that aren't in _any_ standard, and it does so because people want
and use those features.
(I realise this conflicts with my position that devfsd is evil and should die.
The truth is, devfsd is intrusive (breaking stuff like losetup when people do
allyesconfig and accidentally enable it), never worked properly, and
something I actively don't want to maintain. If Tito steps up to defend it,
and is passionate about it and wants to maintain it I'll probably back down
on that one. But allyesconfig still shouldn't select it, because it breaks
If you're going to try to wean people off of obsolete things that some
standard depreicates (yet which people choose to widely ignore anyway), then
why aren't you trying to wean people off of gnu extensions that were never
standardized in the first place?
> The advantage i see of being pedantic about the latest standard is that
> it helps developers fix their scripts, standards are meaningless if
> people dont/wont use them.
We're talking about the embedded space. Developers here violate standards
left and right. Most of them are going to use the small version of sort.
I am a ruthless pragmatist. (You may have noticed. :) My computing history
dates back to the Atari 800 and Commodore 64, and wandered to Linux by way of
Dos with Desqview and OS/2. All of that was based on "what people got to
work", usually by reverse engineering.
Saying we implement a standard and can prove it, I'm all for. People who
write to the standard can then be pretty sure they can run our code, and
that's good to know. But intentionally choking on anything that isn't in the
standard, yet is widely understood elsewhere, is counterproductive.
"Standards are great, there's so many to choose from, and if you didn't pick
the same one we did expect busybox to break"... That's not so good.
Offering a debugging tool that lets people who _want_ to be pedantic, I can
see. But wouldn't a better way be some kind of debug warning that emits when
you use an obsolete feature? (So your script generates messages rather than
Of course this gets you back to the problem that running your script under
busybox, things are likely to break 8 ways from sunday right now. This means
that right now, the option you added is merely useless, because "it doesn't
run under busybox" is not a useful datapoint anyway. 1.1 will be a _lot_
better about that, but real world use will find stuff we don't handle
properly, all over the place. (For example, linking busybox against uClibc
right now, "tar xvjfC file.tgz directory ." doesn't work because the argument
that "f" gets isn't "file.tgz", it's "C". This is a uClibc bug, but it hits
My goal right now is to get everything out there to work with a "make
allyesconfig" busybox (which means lots of gnu extensions, but not all of
them because many nobody we care about actually uses), and someday being able
to say that we implement all of susv3 except for the internationalization
If we intentionally break people's scripts in the name of "standards
compliance" before we ever get everything working long enough and widely
enough that people start to _trust_ that everything works, then all that
happens is nobody will ever think of BusyBox as a real environment.
In a real world tool, "Standards compliance" doesn't mean you _stop_ at the
standard unless you compile the thing with -pedantic and set POSIX_ME_HARDER,
and there's a _reason_ both of those options have derogatory names.
Right now, our shells (all four of them) are useless for any scripts that
people didn't write expicitly for busybox, because in the real world people
use bash syntax and we can't handle it. We have four different shells that
don't work in different ways, and I want to clean that up and have one
scalable shell that runs real scripts. And it's going to implement bash
syntax first, and be checked against susv3 later once it works on real
scripts. (That's how Linus wrote the Linux kernel originally: he implemented
a lot of stuff from the the sun man pages and then got tired of that and
started implementing all the system calls bash was trying to make on its' way
If I focused on implementing an susv3 shell, I'd just have another shell that
nobody can use. Instead, I intend to focus on real scripts people might want
to run, and then once those work listen to bug reports from people trying to
run other real scripts and finding stuff we can't handle. That's all that
_matters_. Compliance with the spec is bragging rights, really, and if it
was _important_ there'd already bee a test suite that tells you whether or
not your command line is compliant with it. (Writing such a free compliance
test suite might is a todo item of mine. Because NOBODY ELSE HAS. Even
after all these years...)
Don't worship the spec. Please...
> The easier it is for people to move on to
> the latest standard the quicker we can remove all the compatability
> code. But that is a long term issue, not a short term one.
If busybox becomes a hare shirt for developers, they'll stop using it. Having
debugging modes, yay. But let's debug busybox itself first before we start
expecting people to use our tools to find stuff wrong with their scripts.
Right now it's hard enough to get them to find stuff wrong with busybox,
which _I_ can do at the drop of a hat...
> > 6) I can see the advantage of disabling things like gnu extensions when
> > people want to use our code to test standards compliance, but right now
> > we don't implement anything close to the whole standard yet so do we want
> > to start worrying about that now?
> I guess those few cases are some obvious ones that had bugged me for a
> while, they have obsolete in the standard for like four years...
And nobody cares. They continue to use them as long as they work. And no
matter what the standard says, I see nothing wrong with that.
> I really think people who are using numeric arguments need to fix their
> scripts so we can all move on.
Why? What's the downside of saying -40? (Is it ambiguous?)
> The changes looked pretty safe to me, i.e. not likely to break
> anything, and its something that has been talked about for a long time
> i think, so figured why not do it now.
Talked about by who? I missed this.
In any case, this is very much a 1.2 issue. Could we comply with susv3 before
we worry about enforcing it?
Steve Ballmer: Innovation! Inigo Montoya: You keep using that word.
I do not think it means what you think it means.
More information about the busybox