[PATCH] Updates on busybox POSIX compliance

Fri Jun 19 08:16:51 UTC 2009

On Thursday 18 June 2009 19:45:07 Mike Frysinger wrote:
> On Thursday 18 June 2009 17:38:49 Rob Landley wrote:
> > On Thursday 18 June 2009 09:35:32 Mike Frysinger wrote:
> > > On Thursday 18 June 2009 07:33:48 David Krakov wrote:
> > > >  * I've seen some tests compare with GNU tools output - does it
> > > > require a specific version of GNU utilities?
> > >
> > > we only care about the latest versions.  that means some tests may have
> > > been written for older versions and so fail with latest ones.
> >
> > An implementation is not a standard, especially where different gnu tool
> > versions give different results.
>
> the latest version of a GNU tool is the the GNU standard

Which part of "an implementation is not a standard" did you not understand?  
(Or are you calling what this week's bugfix release of Microsoft Word does a 
standard?  I'm sure the old Star Office developers would love to discuss that 
with you over a beer and several large boards with nails in.)

I'm fairly certain I put this in the FAQ years ago:

  http://busybox.net/FAQ.html#standards

> > > or they're
> > > written for  the latest and so it is expected they'll fail on older
> > > ones. we should document in the test the last known tested GNU version,
> > > but we should always be targeting the latest GNU releases.
> >
> > We should not be testing against gnu tools.
>
> considering we're implementing GNU features, yes we should.

A) We've implementing some features beyond SUSv3, but by no means every weird 
thing gnu tools do.  If you want that, use the gnu tools.

B) If you can't run the test suite on a system that hasn't got the gnu tools 
installed, it's a useless test suite.

> the entire
> point of these features is to replicate the GNU functionality.

No, the entire point of these features is to serve the needs of users out 
there, most of which don't really care what the gnu tools do.  They just care 
that their scripts run.

For example, nobody really cares that gnu tools create info pages.  It's a 
dead format the FSF clings to out of inertia.  I doubt anybody using busybox 
cat has ever noticed the lack of a "--squeeze-blank" long option.  Or the fact 
that if you go "./busybox cat --version" it spits out an error message at you 
because we don't support that and have never supported that even though every 
gnu tool _does_ support that.

And in some cases, we threw out the existing implementation entirely and wrote 
our own things (for example Erik did dumpkmap/loadkmap and I did mdev and 
catv) that don't pretend to work like any other implementation.  You can't 
test them against "the gnu version" because there ISN'T ONE.  (This isn't even 
counting things like pipe_progress where there was no similar existing 
functionality we could have chosen to copy.)

> and tests
> are designed exactly to make sure the functionality remains working.

So it's impossible to create a regression test suite for something like a 
Python interpreter because the Free Software Foundation never issued a python 
interpreter, and your definition of a regression test suite is "run the FSF 
version and check the output".  (Oh, and if the FSF version changes it 
behavior between releases, then we broke even if our code didn't change.) 

That's _insane_.

> > The test suite should run and
> > produce usable results on a system that _only_ has busybox installed.
>
> these things are not mutually exclusive.  no GNU tools -> GNU tests are
> skipped.

The most gnu-specific test I can think of is in testsuite/sed.tests:

  # Test lie-to-autoconf

  testing "sed lie-to-autoconf" "sed --version | grep -o 'GNU sed version '" \
        "GNU sed version \n" "" ""

Notice that even for a test for our version's ability to explicitly replicate 
a GNU identification string, which exists to work around a _bug_ in gnu 
software, the test isn't running the gnu version of the command, nor depending 
on its existence on the system.  Because that would be pointless, unnecessary, 
brittle, and stupid.

> > > > Though there are tests that can be used for this objective, the test
> > > > suite is lacking a simple way to mark existing tests with some kind
> > > > of a flag like POSIXTEST. How do you propose to do that?
> > >
> > > i think we should split the tests so that POSIX and GNU conformance is
> > > kept separate.
> >
> > Annotating existing tests which what context they test is useful.
> > Splitting the test suite into separate scripts means each one now gets
> > half as much attention.
>
> how the scripts are organized is irrelevant to "attention".  whether they
> get executed is relevant.  my suggestion of splitting the tests was to keep
> things organized and easily skipped when GNU functionality was not
> available ... it did not imply we would stop running the tests.

Gnu functionality is not _special_.  We've already had requests for posix, and 
even when gnu violates posix we might not want to.  We may want to implement 
MacOS X idiosyncrasies at some point in the future, after the Linux developers 
acknowledge this whole "desktop" thing didn't work out for us because whenever 
"shut up and show me the code" isn't an appropriate response to user feedback 
the open source development process winds up forking itself to death as badly 
as proprietary unix ever did.  (Which is sad, but after 20 years of trying to 
break 1% market share, and having now dropped below 10% market share in the 
subnotebook market Linux systems _invented_,  I think we can acknowledge this 
is a structural problem and not something that will go away if we wait long 
enough.  Oh well.)

Also, "gnu" not a synonym for "standard linux behavior": util-linux, procps, 
bunzip2, less, sysklogd, sysvinit, vim, iproute2, rpm, e2fsprogs, fun things 
like ifenslave...

Heck, the gnu versions are _themselves_ either reimplementations or code they 
adopted after it was created.  Stallman's version of Emacs wasn't even the 
first reimplementation (James Gosling did "gosmacs" before creating Java, Jamie 
Zawinski did xemacs back before Netscape).

I started using netcat before there was a gnu version.  I wrote my own 
reimplementation of netcat before there was a gnu version.  Here's a page on 
the history of netcat: http://m.nu/program/util/netcat/netcat.html which the 
"gnu netcat" page doesn't exactly highlight: http://netcat.sourceforge.net/

GNU did not invent tar.  GNU did not invent Unix.  They didn't invent 
_anything_ of note.  Nope, not even gzip: deflate was created by Phil Katz for 
pkzip version 2.  Obviously gcc reimplemented pcc (and Stallman's first 
instinct was to extend the pastel compiler.  He gave up on that and did 
actually write his own, but it was really Michael Tiemann, founder of Cygnus, 
who made the result interesting).

Larry Wall wrote patch back in May 1985 and posted it to usenet:

  http://groups.google.com/group/mod.sources/browse_thread/thread/c5240ceb77b7f586/488b0929254d936a

And here's the here's a release notes for version 2 from 1988 which _still_ 
had nothing to do with the FSF (no mention of gnu or fsf, copyright ntocie is 
by Larry when the FSF requires copyright be assigned to them, his contact 
email is his work address at nasa...):

  ftp://garbo.uwasa.fi/unix/patch/README

The FSF is still trying to take credit for Linux, but Linux is clearly if 
anything a successor to _Minix_ (original posted and discussed on 
comp.os.minix, the famous tanenbaum-torvalds debate, Linus's own book 
describing the history of the thing, etc).  Minix had nothing to do with the 
FSF, neither did BSD, neither did Open Solaris, neither did Bell Labs' 
original Unix.

I can go on if you like, but hopefully I've made my point.  The GNU tools are 
not special.  Half the point of busybox is to offer an ALTERNATIVE to software 
written by that insane group of religious extremists.  (You at least get 
_different_ crazy from us. :)

> -mike

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds