Significant performance problem with modprobe

[Timo Teräs]

On 06/14/2011 10:48 PM, Timo Teräs wrote:
> On 06/14/2011 06:15 PM, Ed W wrote:
>> On 14/06/2011 16:10, Timo Teräs wrote:
>>> The proper thing to do still is the implement the binary file format.
>>> This is just trying to do micro-optimizations compared to it.
>>
>> Could be - however, Lauri's results were: 28% faster using binary
>> modutils and 23% faster using text modutils.  Given that our "readline"
>> function is used in quite a few places in busybox, getting this nailed
>> has the potential to cause significant speedups in other places such as
>> sed (often used on reasonably sized files?)
>>
>> I hope you will consider looking slightly further into this? It does
>> seem like low hanging performance fruit that would have an effect more
>> widely than just "modprobe"?
> 
> Looking at callgrind, it says that the line reader is the biggest CPU
> hog. I'm not really sure how to further speed it up other than using the
> unlocked getc. Also, glibc vs. uclibc implementations have difference.

Duh. It looks like the bb_get_chunk_with_continuation always allocates
new memory for the returned string. That could be kinda slow. It'd be
faster if the line reading buffer was allocated only once (and grown
only when necessary).

But this realloc/free business is probably another culprit that has
global effect. Some one should redesign bb_get_chunk_with_continuation
so it can be used with pre-allocated buffers.

- Timo