[PATCH] Updates on busybox POSIX compliance

[Rob Landley]

On Thursday 18 June 2009 06:33:48 David Krakov wrote:
> Hi,
>
> Table can be updated only manually - documenting differences between
> POSIX and busybox behaviour on options that do exist.

Are non-existing options considered non-compliant?

Is it possible to write tests for these differences?

> It can serve as a mini-TODO file.

We haven't exactly got a shortage of TODO files. :)

> The only easy way I see to go with tests is to check for full
> compliance by comparing with GNU (which is not strictly compliant
> itself).

Look at testsuite/seq.tests for an example (and notice the comment near the 
top which shows the arguments of the "teesting" macro).  That one claims to 
test SUSv3 compliance, but SUSv4 is out now and it doesn't actually have any 
annotations to say what behavior is required by the standard and what we're 
just checking for, nor to announce any kind of standards-related conclustion.  
Just whether or not the tests were passed.

You can also run the tests on the gnu version of each command, but it's not 
required.  (That's a way to test the test script itself.)

> I ran the test suite - there are some FAILs:
>  * Is it up to date?

Unlikely.  Like documentation, it's a low priority that bit rots unless 
somebody notices.

>  * I've seen some tests compare with GNU tools output - does it
> require a specific version of GNU utilities?

No, and the tests that compare against GNU tools output are _broken_.  They 
shouldn't do that, they should test against known output.

Before I left in 2006 I was migrating the command subdirectpry kind of tests 
into the command.tests file kind of tests.  I've been wandering back recently, 
but I dunno what the current status of the test suite is and looking into it 
isn't near the top of my TODO list.

> Though there are tests that can be used for this objective, the test
> suite is lacking a simple way to mark existing tests with some kind of
> a flag like POSIXTEST. How do you propose to do that?

Add another argument to the "testing" command that indicates what context 
requires that.  (Basically append a sixth argument, "SUSv4", to tests that 
indicate SUSv4 compliance, and then update the infrastructure to make use of 
that info.)

> P.S
> attached `runtest -v` stdout and stderr.

There's a "make test" target.

The test suite is another thing I did rather a lot of work on in toybox, so 
I've been away from busybox here for 2 years and don't really remember what 
subset of the testing infrastructure I'm familiar with it implements, and what 
they've added since I left...

I do note that "optional" to test config options seems to be a toybox thing...

Rob

P.S.  I said it was the right thing to do long-term, I didn't say it was 
_easy_. :)
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds