[Buildroot] [PATCH 3/6] support/test-pkg: report number and types of failures

[Thomas De Schampheleire]

On Thu, Feb 9, 2017 at 10:59 PM, Thomas Petazzoni
<thomas.petazzoni at free-electrons.com> wrote:
> Hello,
>
> On Wed,  8 Feb 2017 21:15:26 +0100, Yann E. MORIN wrote:
>
>> +    if [ ${nb_dl} -ne 0 ]; then
>> +        printf "%d configurations could not be downloaded\n" ${nb_dl}
>> +    fi
>> +    if [ ${nb_cfg} -ne 0 ]; then
>> +        printf "%d configurations could not be applied\n" ${nb_cfg}
>> +    fi
>> +    if [ ${nb_skip} -ne 0 ]; then
>> +        printf "%d configurations were skipped\n" ${nb_skip}
>> +    fi
>> +    if [ ${nb_clean} -ne 0 ]; then
>> +        printf "%d configurations could not be dircleaned\n" ${nb_clean}
>> +    fi
>> +    if [ ${nb_build} -ne 0 ]; then
>> +        printf "%d configurations would not build\n" ${nb_build}
>> +    fi
>
> This is really verbose and not very useful. Instead what would be much
> more useful is just:
>
>         successes = %d, failures = %d, skipped = %d
>
> (possibly presented in a different way, my point is really that number
> of successes, number of failures and numbers of skipped configurations
> is really all what matters)
>

I agree with Thomas here. Especially because a failure in download,
olddefconfig and dirclean steps is exceptional. Most failures will be
due to build problems or to a lesser extent invalid cofigurations
(either completely invalid == developer error in providing the
snippet, or invalid in some toolchains only).
Since the amount of toolchains that will be checked is limited, it is
not a lot of work to scroll up to view the actual problems.

/Thomas