[Buildroot] Call for autobuild fixing for 2013.11

[Thomas De Schampheleire]

Hi Thomas,

On Mon, Dec 2, 2013 at 1:32 PM, Thomas Petazzoni
<thomas.petazzoni at free-electrons.com> wrote:
> Dear Thomas De Schampheleire,
>
> On Mon, 2 Dec 2013 12:11:02 +0100, Thomas De Schampheleire wrote:
>
>> Yes, obviously I did not mean to stop fixing autobuild issues when
>> 2013.11 is released.
>> In fact, the amount of remaining autobuild issues at the release of
>> 2013.11 was relatively small that it is imaginable that we could close
>> them all... It would be great if we could reach 0 failures at some
>> point for a few days. Then any new failure could be attributed to a
>> recent commit, and should be fixed at once (or the commit reverted).
>
> Yes, I must say I was impressed by how much we managed to reduce the
> number of build failures.

A wild idea I have is to keep count of how many autobuild problems
were fixed by each contributor, and add this ranking to the buildroot
release e-mail, similar to how commits are counted. This would give
some extra incentive to developers in fixing such problems.

>
> However, it's hard to be 100% sure a failure showing is a new failure,
> even if we had several days in a row with 0 failures. The number of
> possible combinations is so huge that I don't think it's possible to
> test all of them within a reasonable amount of time. So a "new failure"
> may just be something that did exist since quite some time was that we
> couldn't trigger (like was indeed by another failure, for example).

True. But, imagine we had zero failures for a few days, and then there
is a new failure. The logical thing to do is check which package
fails, in which way, and have a look at the recent commits. In many
cases, it would be obvious whether the new failure is or could be
caused by such a recent commit, or if it is an 'old' issue that wasn't
seen for a while.
So, while we cannot automatically blame the last set of commits,
having close-to-zero failures would certainly help a lot in the
analysis.

>
> Also, another effect of having more success in the builds is that we do
> less builds (statistically, successful builds take more time than
> failed builds, since all successful builds run completely to the end).
> Therefore, the number of builds we do each day is now below 100, which
> is not enough. I think we should look at adding more autobuilders in
> the future. I'll try to clean up my script so that other people can set
> up autobuilders.

That would be very useful, indeed. Thanks in advance...

Best regards,
Thomas