Cross-compilation (was: Thank you for not using autoconf)

Rich Felker dalias at aerifal.cx
Mon Jun 10 20:39:21 UTC 2013


On Fri, Jun 07, 2013 at 01:46:38PM +0200, Laurent Bercot wrote:
> > I think you misread what I wrote. The problem is needing to _run_ the
> > test program. Proper autoconf tests simply test preprocessing,
> > compiling, or linking, but not running the resulting executable. It's
> > rare that you need to run a test program, and when you do, it's often
> > a bug for other reasons.
> 
>  I understand what you wrote, but I disagree here. Having to run a
> test program is pretty common if you want to work around system bugs
> or idiosyncrasies.
>  Some old rand() implementations have a bug where the least significant
> bit isn't random at all.

The solution is not to use rand() at all. It's not thread-safe,
usually of abysmally low quality, and easy to replace. If you do
insist on using it, just using the upper bits is usually a reasonable
solution. Amusingly this issue just came up in musl, and we switched
from a 32-bit LCG to a 64-bit LCG to make the folks who were using the
low bits happy. :-)

> Some poll() implementations report EOF via
> POLLIN, others via POLLHUP, others via both.

If you want to support non-conforming implementations, just check for
both. It's not that hard.

> Some pipe implementations
> constantly report EOF to a reader once a writer has been opened then
> closed, others report EOF on the first check only then do not trigger
> poll() anymore. How do you test that kind of behaviour without running
> a test program ?

Just write your program not to care which way it works.

> > For instance you might test for a buggy
> > version of some syscall, and find that it's working, but then get the
> > buggy one at application-runtime when somebody runs your binary on an
> > older kernel.
> 
>  I remember making that exact point to you about signalfd() in musl,
> arguing that the workaround you wrote might break build-time bug
> detection. ;)

And I vaguely remember the counter-argument being that it's not wise
to encode the behavior of the kernel you happened to use when running
configure into the binary, since it's quite likely that the user might
be running an older kernel or a newer kernel. At least with libraries
you may have some version control/dependency information available,
but when it comes to the kernel, it's generally frowned upon to tell
users they need to use specific kernel versions...

By the way, the signalfd issue was SFD_NONBLOCK and SFD_CLOEXEC not
being supported on older kernels, and musl emulating them. Emulating
SFD_NONBLOCK should be entirely transparent to the application (it has
no atomicity property), and while emulating SFD_CLOEXEC does introduce
a fd-leak race on old kernels, there's absolutely no way to work
around such leaks on old kernels that lack atomic open-with-cloexec
features, so any fallback the application could do when libc reports
failure would be just as bad or worse than doing it at the libc level.

> > So, basically what I'm saying is that if you need to probe _behavior_
> > of an interface, that probing should be in your application itself,
> > and it should be done when the application is run, rather than being
> > part of the application's configure process.
> 
>  Ah, OK, now I understand your stance on the matter - but I very much
> disagree. You cannot ask applications to do run-time tests ! That is
> totally dodging the main cross-compilation issue. Detecting
> peculiar behaviours and making sure the application runs correctly is
> not the application's job - can you imagine the duplicated code and
> wasted cycles here ? The application's job is to run, end of story.
> Making sure it runs well on a system is clearly a job for the
> compilation/installation process, which does require running some
> tests, which is precisely why cross-compiling is hard.

I agree with you to a point. Putting ridiculous amounts of run-time
tests all over the place is ugly and harmful. But most of the time --
think of the examples you cited above, like POLLHUP, pipe EOF, etc. --
the least expensive solution is just to write your application in such
a way that it's agnostic to the behavior that varies from system to
system. I'm sure you can find counterexamples where relying on a
non-portable behavior that's only detectable at runtime could give you
major performance benefits, but I believe such cases are exceptions,
not the typical case.

Rich


More information about the busybox mailing list