[Buildroot] Build reproducibility

[Thomas Petazzoni]

Dear Thomas De Schampheleire,

On Mon, 2 Sep 2013 15:18:09 +0200, Thomas De Schampheleire wrote:
> > Of course, if within the Buildroot project we are interested in
> > fixing such missing dependencies, then we can find a way of adding
> > randomness into the build order in our autobuilders. But clearly,
> > we do want to expose this randomness to our users.
> 
> I think indeed we should try to set the dependencies right some way
> or another.
> 
> If we assume that a package does not have any configurable options
> that would change its dependencies, a simple way to check if all
> dependencies are properly expressed is through:
> make clean toolchain foo

This is already done by the autobuilders. Thanks to the randomness of
the configuration, if a package fails to express a mandatory
dependency, sooner or later the autobuilders will generate a
configuration that has the package enabled but not one of its unknown
dependencies. The autobuilders have triggered such cases very quickly
in the past when a new package was added, so I'm pretty confident we
have good coverage on this one.

However, I think this suggestion misses the point of the current
discussion: we are talking about *optional* dependencies, i.e
functionality of a given package that are enabled if OpenSSL is
available, or disabled if OpenSSL is not available. Those optional
dependencies cannot be checked/discovered by a build such as "make
clean toolchain foo".

> Also, it's not necessary that each package gets build every night:
> once the dependencies are correct, they will stay correct until a
> version bump. This means we can spread out the execution of this type
> of tests over time, interleaving them with the already existing
> autobuilds with random configurations.

As stated above, for the mandatory dependencies, the autobuilders are
already finding them very quickly. Introduce a new package that lacks a
mandatory dependency expressed in its .mk file, and you'll see it very
quickly in the autobuilders.

> The make targets of buildroot itself are executed sequentially.
> Suppose that we keep a list of all targets executed, something like:
> python-source
> python-extract
> python-patch
> python-configure
> python-build
> python-install-target
> pyfoo-source
> pyfoo-extract
> pyfoo-build
> ...
> 
> To reproduce a build, we can explicitly pass this list on the make
> command-line, roughly like:
> cat <target-list> | xargs make clean toolchain

Could be doable.

I've lost track. Are we talking about all of this to use "include
package/*/*.mk" in make 3.82 which doesn't guarantee sorting, or to be
able to do top-level parallel build?

Seeing how simple it is to get make 3.82 to behave like make 3.81 by
sorting the "include package/*/*.mk" inclusions, I don't think it's
worth doing anything but the fix that Jérôme has suggested.

Only the top-level parallel build would be a good enough to worry about
reproducibility of more complicated builds (and therefore the need to
list in which order the targets were built).

Best regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com