[Buildroot] [PATCH v2 05/12] support/download/go-post-process: implement Go vendoring support

Yann E. MORIN yann.morin.1998 at free.fr
Sun Aug 1 07:08:30 UTC 2021


Christian, All,

On 2021-07-31 21:59 -0700, Christian Stewart spake thusly:
> Vincent said:
> >I think Yann already calls out what may be going on in this patch
> >https://patchwork.ozlabs.org/project/buildroot/patch/20201219153525.1361175-5-thomas.petazzoni@bootlin.com/
> 
> I think this is probably correct, the .tar.gz needs to have
> deterministic formatting.

Can you try usign the tar helper in support/download/helpers instead?

> In this particular case the user was trying to build on Kali 2.
> 
> Doesn't buildroot compile host-gzip and host-tar for this? I'm
> wondering why it's not the same between host machines.

Since 2020.02, we are no longer building host-tar, unless for very old
versions (i.e. before 1.27), thanks to some tar trickery vy Vincent.
See:

    cbe95b1a455b support/download: add helper to generate a reproducible archive

We also only build host-gzip if the host gzip is pigz (or missing).

> On Thu, Jul 29, 2021 at 1:50 PM Thomas Petazzoni
> <thomas.petazzoni at bootlin.com> wrote:
> 
> > > This works fine in most cases, however, this causes inconsistent hashes.
> > Based on the difficulties, I would really like to understand a little
> > bit why the archive is not deterministic. Could you compare the
> > contents of the download stuff between your different machines, and the
> > contents of the tarball, and see what changes?
> The two versions are attached. The actual contents of the two are
> identical. But for some reason the formatting of
> embiggen-disk-bad.tar.gz is different:
> 
>   00000030: 8232 63de 4282 92dd acff fb79 6740 eae2
> 
> Starting at "2dd acff" - in "embiggen-disk-good" it's
> 
> 00000030: 8232 63de 4282 96dd acff fb79 6740 eae6

So, I zcat both archives, and the two tar are identical!

    $ sha1sum *.tar
    dbb8338ab4acaff1e232c67d3b46602d61114b53  embiggen-disk-bad.tar
    dbb8338ab4acaff1e232c67d3b46602d61114b53  embiggen-disk-good.tar

So I also hashed the archives you provided, and they too are identical:

    $ sha1sum *.tar.gz
    bd887a60c4ca60e55d062067132ccab7d85e7d95  embiggen-disk-bad.tar.gz
    bd887a60c4ca60e55d062067132ccab7d85e7d95  embiggen-disk-good.tar.gz

Did you mess up when sending the archives?

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'


More information about the buildroot mailing list