[Buildroot] [RFC] [PATCH v2 2/2] support/kconfig: Bump to kconfig from Linux 4.17-rc2

Thu Aug 2 11:02:19 UTC 2018

On 01-08-18 21:42, Yann E. MORIN wrote:
> Arnout, Thomas, All,
> 
> On 2018-07-31 10:20 +0200, Arnout Vandecappelle spake thusly:
>> On 31-07-18 10:06, Thomas Petazzoni wrote:
>>> On Tue, 31 Jul 2018 09:55:42 +0200, Arnout Vandecappelle wrote:
>>>>>>> We're only talking about dropping the need for host-flex and host-bison
>>>>>>> for the kconfig stuff. In this case, we don;t care that the user
>>>>>>> generates C code one way or another: it's only a host tool...    
>>>>>>
>>>>>>  I was replying to Thomas's "we should remove host-flex and host-bison
>>>>>> entirely". Yes we can rely on system-installed flex and bison for kconfig, but
>>>>>> we can't rely on that for target packages, e.g. target dtc.  
>>>>>
>>>>> Why so ?  
>>>>
>>>>  Reproducibility. I tested this for something (I think it was flex, but as usual
>>>> I don't remember exactly), and the code generated by my system's flex was wildly
>>>> different from what was generated by Buildroot's host-flex, even though the
>>>> versions were pretty close. Unfortunately, I don't remember exactly, but that
>>>> was my concern.
>>>
>>> True, but then if we have this reproducibility issue, it also applies
>>> to building the kconfig code, no ?
>>
>>  kconfig is a host package, (target) dtc is a target package.
>>
>>  The point of reproducibility is that when someone gives me a core dump four
>> years from now, I can still rebuild the same source and see what that core dump
>> really means, and I can make a patch that just changes one little thing in there
>> without upgrading the world. Since host packages are not deployed, it's not
>> needed to have the same level of reproducibility for them.
> 
> Agreed.
> 
> I.e. we don't care about the reproducibility of the host tools that
> contaon generated code, like flex / bison.
> 
> Even if the generated code changes with different versions of flex
> and/or bison, the _behaviour_ of that generated code is predictable.
> It night be less or more optimised in size amd/or speed, but the
> behaviour is predictable. Which is all we care about for host tools,
> imho.
> 
> See how the kconfig code dropped use of gperf, yet still behaves as
> previously, with different (less) generated code.
> 
> But...
> 
> Now that we have the 'sdk' feature, people are going to distribute that,
> and they will have to come in compliance for this sdk. So they will want
> the build of the host tools to also be as reproducible as possible.

 I don't see the reasoning behind this statement. For GPL compliance,
reproducibility is definitely not an issue, otherwise every GPL binary
distributed in the last 30 years would be in violation...

> Amd since we now will have to have the toolchain before we can parse the
> linux kconfig stuff, building host-flex and host-bison is the least of
> our speed problems...

 It is a big speed problem if you're using an external toolchain and you could
use the system flex and bison. This thread (long ago, in a galaxy far away) was
about requiring flex and bison to be installed on the system, because
Buildroot's own kconfig would also no longer ship the generated files. The
general agreement was that that would be OK, and we could remove host-flex and
host-bison entirely (speeding up the build for a bunch of other packages).

 My statement was (and still is): drop host-flex and host-bison for building
host tools, but keep them for building target binaries (in casu, dtc).

 Thomas (if I understand correctly) is questioning the need for keeping
host-flex and host-bison even for building the target binaries. And I have to
agree that the need for host-flex and host-bison for target binaries is not that
strong.

> So, I'd say we should keep the dependency on host-flex and host-bison
> for the linux kernel...

 I would really prefer to not double the build time of a simple kernel...

 Regards,
 Arnout

> 
> Regards,
> Yann E. MORIN.
> 
>>> Or do you assume that because kconfig is so widely used, it is tested
>>> against lots of bison/flex version, so we don't need to build
>>> bison/flex for kconfig, but we should still build it for other packages
>>> that need bison/flex, as those ones may be less tested against random
>>> versions of flex/bison ? Or because the generated flex/bison code goes
>>> into the target, and we ideally want binary-identical results for a
>>> given Buildroot configuration ?
>>
>>  It's not just for fully binary identical (i.e. BR2_REPRODUCIBLE).
>>
>>  It's basically for the same reason that we have versioned docker images, to
>> make sure that we really know what we're building for the target.
>>
>>
>>  I realize now that this rule is not something we've written down anywhere. I
>> just always assumed that Buildroot tries to make sure that for a given config,
>> whatever your build machine, you generate binaries that will behave the same and
>> will expose the same bugs. If you use a different flex, a bug in the parsing
>> code will manifest differently.
>>
>>  The more I think about it, the more I come to the conclusion that maybe this
>> rule (which I thought was something that I learned from the Buildroot community,
>> not something I invented myself) is not so useful. Because in practice, the kind
>> of bugs where this is relevant are so sensitive to even tiny changes that
>> changing the generated code is not going to make all that much of a difference.
>>
>>  Regards,
>>  Arnout
>>
>>>
>>> Best regards,
>>>
>>> Thomas
>>>
>>
>> -- 
>> Arnout Vandecappelle                          arnout at mind be
>> Senior Embedded Software Architect            +32-16-286500
>> Essensium/Mind                                http://www.mind.be
>> G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
>> LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
>> GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF