[PATCH 1/2] wget: replace set_alarm with non blocking functions to support retries

[Martin Lewis]

Hi Denys,

Thank you for the review.
I moved the buffer to G, and added an incremental sleep between the retries
(as in GNU wget).

Even if we use poll() and read(), if we don't want to read byte by byte we
need to use a buffer (like fgets does), as if we read two newlines at once
we need to parse the buffer twice.
As for the default number of retries, if it won't be 1 that might cause
problems with already written scripts that assume wget doesn't try 20 times.
Also, using 'retries' as the variable name might be a little confusing as
when 'retries' = 1 it won't retry the download at all.

Martin
On Sun, 20 Jan 2019 at 18:46, Denys Vlasenko <vda.linux at googlemail.com>
wrote:

> Throws a warning:
>
> networking/wget.c: In function 'fread_buffered':
> networking/wget.c:575: error: declaration of 'read' shadows a global
> declaration
>
> Way too large, and adds to bss:
>
> function                                             old     new   delta
> fgets_buffer                                           -    4096   +4096
> fgets_trim_sanitize                                  128     621    +493
> retrieve_file_data                                   579     775    +196
> open_socket                                           49     117     +68
> fgets_buffer_len                                       -       4      +4
> wget_main                                           2437    2435      -2
> set_alarm                                             27       -     -27
>
> ------------------------------------------------------------------------------
> (add/remove: 2/1 grow/shrink: 3/1 up/down: 4857/-29)         Total: 4828
> bytes
>    text       data        bss        dec        hex    filename
>  979050        485       7296     986831      f0ecf    busybox_old
>  979775        485      11400     991660      f21ac    busybox_unstripped
>
> The second patch is mangled by gmail, please resend as attachment.
>
> I played with a version where one try is done in a child,
> see attached z.diff.
>
> function                                             old     new   delta
> download_one_url                                       -    2221   +2221
> retrieve_file_data                                   579     602     +23
> ftpcmd                                               133     151     +18
> get_sanitized_hdr                                    156     162      +6
> fgets_trim_sanitize                                  128     131      +3
> base64enc                                             46      49      +3
> packed_usage                                       33070   33042     -28
> wget_main                                           2437     565   -1872
>
> ------------------------------------------------------------------------------
> (add/remove: 1/0 grow/shrink: 5/2 up/down: 2274/-1900)        Total: 374
> bytes
>
> I'm not entirely happy with this approach either...
> ...probably need to rewrite existing code to get rid of fgets(),
> use poll() + read().
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/busybox/attachments/20190124/2fc144e5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-wget-replace-set_alarm-with-non-blocking-functions-t.patch
Type: text/x-patch
Size: 7147 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/busybox/attachments/20190124/2fc144e5/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-wget-add-support-for-retries-in-http-requests.patch
Type: text/x-patch
Size: 13112 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/busybox/attachments/20190124/2fc144e5/attachment-0003.bin>