bzcat/zcat does eat huge amounts of memory.

[Markus Gothe]

Maybe this has been fixed since 1.20.2 or maybe is due to the pipe.

For me it is most notable with bzcat. Will have to investigate this further on Tuesday.

//M

Sent from my BlackBerry — the most secure mobile device


	  Original Message  



From: vda.linux at googlemail.com
Sent: 20 April 2019 14:37
To: nietzsche at lysator.liu.se
Cc: busybox at busybox.net
Subject: Re: bzcat/zcat does eat huge amounts of memory.


On Fri, Apr 19, 2019 at 4:23 PM Markus Gothe <nietzsche at lysator.liu.se> wrote:
>
> Seems to me that the implementation is seriously flawed, it decompress the whole file into the heap and then print it out.

Can't reproduce:

$ ./busybox zcat 392mbyte_gzipped_file.gz >/dev/null & pid=$!; pmap
$pid; while pmap $pid; do sleep 1; done
30164:   ./busybox zcat 392mbyte_gzipped_file.gz
0000000008048000    960K r-x-- busybox
0000000008138000      4K rw--- busybox
0000000008139000      4K rw---   [ anon ]
00000000083f6000      4K rw---   [ anon ]
00000000f7741000     64K rw---   [ anon ]
00000000f7751000     12K r----   [ anon ]
00000000f7754000      8K r-x--   [ anon ]
00000000ffa06000    132K rw---   [ stack ]
total             1188K
30164:   ./busybox zcat 392mbyte_gzipped_file.gz
0000000008048000    960K r-x-- busybox
0000000008138000      4K rw--- busybox
0000000008139000      4K rw---   [ anon ]
00000000083f6000      4K rw---   [ anon ]
00000000f7741000     64K rw---   [ anon ]
00000000f7751000     12K r----   [ anon ]
00000000f7754000      8K r-x--   [ anon ]
00000000ffa06000    132K rw---   [ stack ]
total             1188K
30164:   ./busybox zcat 392mbyte_gzipped_file.gz
0000000008048000    960K r-x-- busybox
0000000008138000      4K rw--- busybox
0000000008139000      4K rw---   [ anon ]
00000000083f6000      4K rw---   [ anon ]
00000000f7741000     64K rw---   [ anon ]
00000000f7751000     12K r----   [ anon ]
00000000f7754000      8K r-x--   [ anon ]
00000000ffa06000    132K rw---   [ stack ]
total             1188K
30164:   ./busybox zcat 392mbyte_gzipped_file.gz
0000000008048000    960K r-x-- busybox
0000000008138000      4K rw--- busybox
0000000008139000      4K rw---   [ anon ]
00000000083f6000      4K rw---   [ anon ]
00000000f7741000     64K rw---   [ anon ]
00000000f7751000     12K r----   [ anon ]
00000000f7754000      8K r-x--   [ anon ]
00000000ffa06000    132K rw---   [ stack ]
total             1188K
...
...
30164:   ./busybox zcat 392mbyte_gzipped_file.gz
0000000008048000    960K r-x-- busybox
0000000008138000      4K rw--- busybox
0000000008139000      4K rw---   [ anon ]
00000000083f6000      4K rw---   [ anon ]
00000000f7741000     64K rw---   [ anon ]
00000000f7751000     12K r----   [ anon ]
00000000f7754000      8K r-x--   [ anon ]
00000000ffa06000    132K rw---   [ stack ]
total             1188K

I see no memory usage growth. At all.