mark obsolete usage of fold

Rob Landley rob at landley.net
Sun Dec 11 10:05:37 UTC 2005


Warning: philosophical differences ahead.  You have been warned.

On Sunday 11 December 2005 02:29, Glenn L. McGrath wrote:
> On Sun, 11 Dec 2005 00:30:11 -0600
>
> Rob Landley <rob at landley.net> wrote:
> > On Saturday 10 December 2005 16:53, Glenn L. McGrath wrote:
> > > Attached is a patch sets a feature to allow 'fold -40' type arguments
> > > to fail, this behaviour is obsolete even in susv2.
> > >
> > > Im sure some scripts (e.g. configure) are still using it, so its
> > > usefull to support it.
> > >
> > > The patch doesnt change default behaviour. Its really just to start to
> > > organise things so we can one day have a global setting to choose what
> > > posix standard to compile to.
> >
> > 1) If you're going to do this kind of thing, could you please have a
> > global argument to disable nonstandard extensions?  (I'm trying to keep
> > the amount of configuration people have to do down to a dull roar. 
> > Getting buried in options doesn't help.  I'm thinking of adding things
> > like CONFIG_NEEDS_PROC and CONFIG_LONG_OPTIONS to 1.2, for example.)
>
> Ive commit some changes, i added 2 config options to Build Options, One
> for SuSv2, the other for pre SuSv2.

Both are "not Susv3".  I don't care whether it's a gnu extension, solaris 
compatibility, or something left over from system 5.  We're not currently 
fully implementing _any_ standard, and I believe the one we're trying to move 
towards (and that I'd like to audit stuff against) is susv3.

And yes, there are cases where we'll still violate that standard.  We're not 
doing the full internationalization thing with locale environment variables, 
_ever_ if I can help it, because it's just too much bloat for our mandate to 
absorb.  (UTF-8 seems like the low hanging fruit here.  You followed those 
threads?)

> > 2) I'm trying to get 1.1 out by new year's.  Could this kind of change
> > please wait until January?  (See big long bug list posted a couple days
> > ago.)
>
> umm, woops, too late.

I saw before I posted this, and didn't complain.  No biggie. :)

> > 3) I really think we should pick one standard we care about, and try to
> > implement all of.  (Right now it's that Open Group thing that's on the
> > web, which is roughly susv3.)  Trying to distinguish between susv2 and
> > susv3 is just going to drive us _crazy_.
>
> The files i just patched are some obvious ones, but we can also get
> information about changes between SuSv2 and SuSv3 from coreutils or
> maybe the Heirloom Project.

Something we need to do, totally separate from this, is audit all the apps to 
see which ones are susv3 compliant and which aren't.  In my head, this is 
tied up with extending the test suites to _test_ all of susv3 (using the new 
"all tests in one script" format so that the subversion history is actually 
intelligible).

So I don't personally care what susv2 said, unless the spec we're getting off 
the open group's website is susv2.  (The one that all our URLs point to, like 
the one in the comment towards the start of sed.c.)

> I think we need a compatability layer for when things break, e.g.
> configure scripts still use the numeric arguments in head/tail i think.

Yes, they due.  If real world scripts would break due to the lack of numeric 
arguments, we should be able to handle the numeric arguments.  I really don't 
care what the standard says if a 100% standards compliant implementation is 
useless in the real world.  To me, the standard is an important point of 
reference, but not the whole picture.

For example, uClibc 0.9.28 won't install unless tar has the --exclude option.  
(I had to fix this to get uClibc 0.9.28 to install with busybox tar.)  You 
still can't install uClibc 0.9.28 with busybox 1.01, and those are the most 
recent official releases of each...

That's not the only gnu extension we need to be useful.  Far from it.  
The ./configure stage of all sorts of gnu extensions in sed and sort in order 
to complete.  (This gets silly at times: look in editors/sed.c for the 
comment "Lie to autoconf".  Yes, this is needed to make things like binutils 
build correctly.  It shouldn't be, but it is.)

What standard is bzip part of?  Does this mean people should stop using it?

> We can try for the latests standard, but still have the old behaviour
> to fall back on.

This is a case where I consider the additional feature to be an extension.  We 
provide gnu extensions left and right, so where's the downside of providing 
obsolete behavior as an extension?

There are several reasons for removing old behavior.  Bloat is high on the 
list for us, but there's also potential security issues, causing interference 
with other functionality, and so on.

Removing it just _because_ it's old, however...  why?  We configure out the 
old tar format because A) it saves space, B) almost nobody ever uses it so 
it's not a feature there's much demand for.  (And we do that because _both_ 
of those are true.)

If the argument starts with "saves x bytes", I'm all for it.  If the argument 
is "because people should convert their scripts"...  Busybox provides 
features that aren't in _any_ standard, and it does so because people want 
and use those features.

(I realise this conflicts with my position that devfsd is evil and should die.  
The truth is, devfsd is intrusive (breaking stuff like losetup when people do 
allyesconfig and accidentally enable it), never worked properly, and 
something I actively don't want to maintain.  If Tito steps up to defend it, 
and is passionate about it and wants to maintain it I'll probably back down 
on that one.  But allyesconfig still shouldn't select it, because it breaks 
stuff.)

If you're going to try to wean people off of obsolete things that some 
standard depreicates (yet which people choose to widely ignore anyway), then 
why aren't you trying to wean people off of gnu extensions that were never 
standardized in the first place?

> The advantage i see of being pedantic about the latest standard is that
> it helps developers fix their scripts, standards are meaningless if
> people dont/wont use them.

We're talking about the embedded space.  Developers here violate standards 
left and right.  Most of them are going to use the small version of sort.

I am a ruthless pragmatist.  (You may have noticed. :)  My computing history 
dates back to the Atari 800 and Commodore 64, and wandered to Linux by way of 
Dos with Desqview and OS/2.  All of that was based on "what people got to 
work", usually by reverse engineering.

Saying we implement a standard and can prove it, I'm all for.  People who 
write to the standard can then be pretty sure they can run our code, and 
that's good to know.  But intentionally choking on anything that isn't in the 
standard, yet is widely understood elsewhere, is counterproductive.  
"Standards are great, there's so many to choose from, and if you didn't pick 
the same one we did expect busybox to break"... That's not so good.

Offering a debugging tool that lets people who _want_ to be pedantic, I can 
see.  But wouldn't a better way be some kind of debug warning that emits when 
you use an obsolete feature?  (So your script generates messages rather than 
breaking?)

Of course this gets you back to the problem that running your script under 
busybox, things are likely to break 8 ways from sunday right now.  This means 
that right now, the option you added is merely useless, because "it doesn't 
run under busybox" is not a useful datapoint anyway.   1.1 will be a _lot_ 
better about that, but real world use will find stuff we don't handle 
properly, all over the place.  (For example, linking busybox against uClibc 
right now, "tar xvjfC file.tgz directory ." doesn't work because the argument 
that "f" gets isn't "file.tgz", it's "C".  This is a uClibc bug, but it hits 
us.)

My goal right now is to get everything out there to work with a "make 
allyesconfig" busybox (which means lots of gnu extensions, but not all of 
them because many nobody we care about actually uses), and someday being able 
to say that we implement all of susv3 except for the internationalization 
bits.

If we intentionally break people's scripts in the name of "standards 
compliance" before we ever get everything working long enough and widely 
enough that people start to _trust_ that everything works, then all that 
happens is nobody will ever think of BusyBox as a real environment.

In a real world tool, "Standards compliance" doesn't mean you _stop_ at the 
standard unless you compile the thing with -pedantic and set POSIX_ME_HARDER, 
and there's a _reason_ both of those options have derogatory names.

Right now, our shells (all four of them) are useless for any scripts that 
people didn't write expicitly for busybox, because in the real world people 
use bash syntax and we can't handle it.  We have four different shells that 
don't work in different ways, and I want to clean that up and have one 
scalable shell that runs real scripts.  And it's going to implement bash 
syntax first, and be checked against susv3 later once it works on real 
scripts.  (That's how Linus wrote the Linux kernel originally: he implemented 
a lot of stuff from the the sun man pages and then got tired of that and 
started implementing all the system calls bash was trying to make on its' way 
up.)

If I focused on implementing an susv3 shell, I'd just have another shell that 
nobody can use.  Instead, I intend to focus on real scripts people might want 
to run, and then once those work listen to bug reports from people trying to 
run other real scripts and finding stuff we can't handle.  That's all that 
_matters_.  Compliance with the spec is bragging rights, really, and if it 
was _important_ there'd already bee a test suite that tells you whether or 
not your command line is compliant with it.  (Writing such a free compliance 
test suite might is a todo item of mine.  Because NOBODY ELSE HAS.  Even 
after all these years...)

Don't worship the spec.  Please...

> The easier it is for people to move on to 
> the latest standard the quicker we can remove all the compatability
> code. But that is a long term issue, not a short term one.

If busybox becomes a hare shirt for developers, they'll stop using it.  Having 
debugging modes, yay.  But let's debug busybox itself first before we start 
expecting people to use our tools to find stuff wrong with their scripts.  
Right now it's hard enough to get them to find stuff wrong with busybox, 
which _I_ can do at the drop of a hat...

> > 6) I can see the advantage of disabling things like gnu extensions when
> > people want to use our code to test standards compliance, but right now
> > we don't implement anything close to the whole standard yet so do we want
> > to start worrying about that now?
>
> I guess those few cases are some obvious ones that had bugged me for a
> while, they have obsolete in the standard for like four years...

And nobody cares.  They continue to use them as long as they work.  And no 
matter what the standard says, I see nothing wrong with that.

> I really think people who are using numeric arguments need to fix their
> scripts so we can all move on.

Why?  What's the downside of saying -40?  (Is it ambiguous?)

> The changes looked pretty safe to me, i.e. not likely to break
> anything, and its something that has been talked about for a long time
> i think, so figured why not do it now.

Talked about by who?  I missed this.

In any case, this is very much a 1.2 issue.  Could we comply with susv3 before 
we worry about enforcing it?

Rob
-- 
Steve Ballmer: Innovation!  Inigo Montoya: You keep using that word.
I do not think it means what you think it means.



More information about the busybox mailing list