[Buildroot] [PATCH v2 RFC] support/scripts: don't build board defconfigs with Gitlab's pipelines trigged on tag

[Yann E. MORIN]

Thomas, All,

On 2021-11-11 11:02 +0100, Thomas Petazzoni spake thusly:
> On Thu, 11 Nov 2021 09:25:48 +0100
> "Yann E. MORIN" <yann.morin.1998 at free.fr> wrote:
> > On 2021-11-09 23:03 +0100, Romain Naour spake thusly:
> > > In order to reduce the number of long jobs, don't build board
> > > defconfigs with pipelines trigged on tag, keeping only the runtime
> > > tests and the Qemu's defconfigs.
> > Applied to master, thanks.
> I hadn't had the chance to chime in on this one,

Sorry, I could have waited a bit more indeed...

> but I don't entirely
> agree with the change (I also don't entirely disagree!). Indeed, I
> believe we do want to have the results of the defconfigs builds on
> tags. If we missed something, and one of our release has defconfigs
> broken, we definitely want to know it and fix it.

On principle, I do agree with that, and that's what I initially coded in
that script.

> This disabling of defconfig builds is only done because there is an
> infrastructure issue. But really, from a proper QA/CI point of view, we
> really want to build the defconfigs on tags I think.

Again, agreed on principle.

But as Romain explained, that creates too much jobs when a tag is cut,
so much so that a lot of them do not even stand a chance to even start.

In that case, the CI is not usefull at all either.

On the other hand, the defconfigs are already built reasonably often,
every week, with a scheduled pipeline. It is thus pretty quick to notice
a build failure.

So, my opinion is that I prefer a partial CI that is stable and
dependable, where each failure is an actual failure, rather than a
full-range CI that has a very low SNR, where people eventually stop
caring about failure reports because they are often spurious failures.

Of course, we can reverse course on that commit; after all, that's what
git-revert is for! ;-)

The better course of action, obviously, would be to have an infra that
is strong enough to absorb all that load when we push a tag, but that's
an order of magnitude more complex to achieve.

One thing, though: maybe some of the runners could be configured to
accept more jobs at once, so even if each job then takes longer to run,
we start them faster and thus do not hit the 24-hour blockage limit.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'