[Buildroot] [TO-BE-TESTED] support/download/hg: implement repository cache

[Arnout Vandecappelle]

On 08/02/2019 20:54, Yann E. MORIN wrote:
> Arnout, All,
> 
> On 2019-02-08 20:27 +0100, Arnout Vandecappelle spake thusly:
>> On 08/02/2019 17:54, Yann E. MORIN wrote:
>>> On 2019-02-07 22:33 +0100, Arnout Vandecappelle spake thusly:
>>>> On 05/02/2019 21:24, Thomas De Schampheleire wrote:
>>>>> From: Thomas De Schampheleire <thomas.de_schampheleire at nokia.com>
>>>>>
>>>>> Similar to the git download helper, implement a repository cache for
>>>>> Mercurial repositories.
>>>>>
>>>>> The code is mostly guided by the implementation for git, with certain parts
>>>>> copied almost verbatim.
>>>>>
>>>>> Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire at nokia.com>
>>>>> ---
>>>>> index efb515fca5..bb5cc87969 100755
>>>>> --- a/support/download/hg
>>>>> +++ b/support/download/hg
>>>>> @@ -1,7 +1,7 @@
>>>>>  #!/usr/bin/env bash
>>>>>  
>>>>>  # We want to catch any unexpected failure, and exit immediately
>>>>> -set -e
>>>>> +set -E
>>>>  Ow, this really is inherited from git, introduced by commit b7efb43e86da96.
>>>> Yann, care to explain?
>>>
>>> Yes, I care to explain. Even though I do write some crap most of the
>>> time,
>>  I never said that!
> 
> I said it! :-)
> 
> [--SNIP--]
>>> Like all backends, the actual tool is wrapped into a _git() function,
>>> which was introduced more than three years ago, by commit 3f2bdd070
>>> (support/download: protect from custom commands with spaces in args), as
>>> a way to actually fix a bug actually reported by Thomas DS.
>>>
>>> Yes, this makes it that the set -E is needed, after all.
>>
>>  But then the comment is wrong, no? Because -E does not imply -e and without -e
>> there is no immediate exit. Or am I missing something else?
> 
> Yes, it is partly incorrect. But consider that, with the ERR trap, we
> _do_ catch unexpected failures, 

 It seems I'm unable to interpret documentation correctly, because I read the
trap help text as stating that trap ERR only works if -e is set...

                                                                A SIGNAL_SPEC
    of ERR means to execute ARG each time a command's failure would cause the
    shell to exit when the -e option is enabled.


> but we no longer _immediately_ exit
> anymore, indeed. We just postpone the exit to after the second
> tentative, in which case we do exit immediately from the ERR trap.
> 
> However, I was thinking that maybe we could revisit this try-again code,
> and just exit on the first error and rely on the user to actually clean
> the git tree if it ever gets corrupted.
> 
> We initially added that, in case the repository was uterly broken,
> because our backend itself  was creating situations where it _could_
> leave the repo in an inconsistent state, or the autobuilder code would
> do that (e.g. timeout-kill-9 the build right in the middle of a git
> action).

 Kill -9 of git should never leave the repo in an inconsistent state anyway. The
only reasonable way to get an inconsistent repo state is randomly deleting files
from it (and disk failure, of course). But doesn't the autobuilder do that,
randomly delete files?


 Regards,
 Arnout

> 
> However, the git backend was fixed to supposedly always leave the tree
> in a consistent state now, and the autobuilder code now timeouts from
> the last step, not from the build start, so we can assume that not git
> action would now be interrupted abruptely anymore...
> 
> Yet, I am a bit contrived, as the git backend is partly my fault, and so
> I'm not entirely happy with it...
> 
> Regards,
> Yann E. MORIN.
>