[Buildroot] Autobnuilders timeouts [was: Re: autobuilder flock hangs]
Hollis Blanchard
hollis_blanchard at mentor.com
Fri Aug 24 23:45:52 UTC 2018
On 08/24/2018 12:02 AM, Yann E. MORIN wrote:
> Hollis, Julien, Matthew, Thomas, All,
>
> On 2018-08-16 11:10 +0200, Thomas Petazzoni spake thusly:
>> On Mon, 13 Aug 2018 11:18:10 -0700, Hollis Blanchard wrote:
>>> flock was hanging. I don't know why; have you experienced anything like
>>> that?
>> This happened again:
>> http://autobuild.buildroot.net/results/ddb/ddbc96b24017f2a2b06c6091dea3e19520bf2dd1/
> I think we now can see a pattern in those timeouts from Hollis'
> autobuilder: they only ever happen on a slect group of 4 packages that
> are downloaded with git:
> - linux-firmware (kernel.org)
> - f2fs-tools (kernel.org)
> - azure-iot-sdk-c (github.com)
> - uhttpd (openwrt.org)
>
> Could you try to download them manually from your autobuilder, and see
> if that works or not, please?
>
> Is there something on your network that systematically makes those
> packages fail to download? Do you have firewalling restrictions or
> extreme traffic shapping?
azure-iot-sdk-c takes just 26 seconds to clone by hand.
I stopped the autobuilder, then ran a br-reproduce job that took all
day, which ended up at the "rauc" error that y'all just fixed. So... I
guess it didn't time out.
While the autobuilder was still running, I did see some strange
processes though:
init(1)-+
|-flock(3937)
|-flock(4688)
|-flock(4774)
|-flock(9733)
|-flock(10710)
|-flock(10885)---bash(10886)---bash(10889)---git(10898)---git-remote-http(10899)---git(10902)
|-flock(11942)
|-flock(13311)---bash(13312)---bash(13315)---git(13324)---git-remote-http(13325)---git(13328)
|-flock(13681)
|-flock(13915)
|-flock(14113)
|-flock(17018)
|-flock(18869)
|-flock(19152)---bash(19153)---bash(19156)---git(19227)---git-submodule(19228)---git-submodule(19329)---git-submodule(20382+
|-flock(19819)
|-flock(21944)
|-flock(22375)
|-flock(25233)
|-flock(25622)
|-flock(26921)
|-flock(28424)---bash(28425)---bash(28428)---git(28437)---git-remote-http(28438)---git(28441)
|-flock(30945)
|-flock(31269)
|-flock(31271)
|-flock(32627)
|-sh(20815)---bash(20816)---bash(20818)---git(20823)---git-remote-http(20824)---git(20826)
They disappeared when I killed the autobuilder (which is surprising --
seems like they're children of init, so why did they die?).
I suspect a) something goes wrong with the buildroot job, b) it's killed
in a way that leaves a dangling flock, c) future buildroot jobs run
headlong into the lingering flock and triggers a timeout.
--
Hollis Blanchard
Mentor Graphics Emulation Division
More information about the buildroot
mailing list