[Buildroot] [PATCH v3 1/2] support/scripts/pkg-stats: add support for CVE reporting

Thomas Petazzoni thomas.petazzoni at bootlin.com
Wed Feb 19 18:48:58 UTC 2020


Hello Titouan,

On Sat, 15 Feb 2020 13:44:16 +0100
Titouan Christophe <titouan.christophe at railnova.eu> wrote:

> This commit extends the pkg-stats script to grab information about the
> CVEs affecting the Buildroot packages.

Here the script consumes too much memory. On my 4 GB RAM server, the
script gets killed by the OOM killer. It goes like this:

Cannot parse package 'mysql' version ''
Cannot parse package 'mysql' version ''
Cannot parse package 'mysql' version ''
Killed

In the logs, I have:

[273138.062124] sh invoked oom-killer: gfp_mask=0x3000d0, order=2, oom_score_adj=0
[273138.062131] sh cpuset=/ mems_allowed=0
[273138.062139] CPU: 1 PID: 27711 Comm: sh Not tainted 3.14.32-xxxx-grs-ipv6-64 #9
[273138.062142] Hardware name: OVH KS/D425KT, BIOS MWPNT10N.86A.0083.2011.0524.1600 05/24/2011
[273138.062145]  0000000000000000 ffffc90006383b78 ffffffff81efb3fc 00000000003000d0
[273138.062152]  ffffc90006383be0 ffffffff81ef50fb ffffc90006383bc8 ffffffff81157741
[273138.062157]  ffffc90006383bc8 ffffffff81694cf5 0000000001320122 0000000000000206
[273138.062162] Call Trace:
[273138.062177]  [<ffffffff81efb3fc>] dump_stack+0x45/0x56
[273138.062183]  [<ffffffff81ef50fb>] dump_header+0x93/0x1f7
[273138.062191]  [<ffffffff81157741>] ? delayacct_end+0x81/0x90
[273138.062198]  [<ffffffff81694cf5>] ? gr_task_acl_is_capable_nolog+0x15/0xb0
[273138.062204]  [<ffffffff81169751>] oom_kill_process+0x201/0x350
[273138.062210]  [<ffffffff810f180c>] ? has_ns_capability_noaudit+0x2c/0x40
[273138.062215]  [<ffffffff810f1832>] ? has_capability_noaudit+0x12/0x20
[273138.062220]  [<ffffffff81169f3d>] out_of_memory+0x4bd/0x500
[273138.062227]  [<ffffffff8116faf8>] __alloc_pages_nodemask+0x878/0x8c0
[273138.062234]  [<ffffffff810e635a>] copy_process.part.49+0x11a/0x1840
[273138.062239]  [<ffffffff810e7c3b>] do_fork+0xcb/0x340
[273138.062246]  [<ffffffff811db708>] ? set_close_on_exec+0x28/0x50
[273138.062252]  [<ffffffff811baa21>] ? filp_close+0x51/0x70
[273138.062257]  [<ffffffff810e7f31>] SyS_clone+0x11/0x20
[273138.062263]  [<ffffffff81f04935>] stub_clone+0x65/0x90
[273138.062269]  [<ffffffff81f0469e>] ? system_call_fastpath+0x16/0x1b
[273138.062271] Mem-Info:
[273138.062273] Node 0 DMA per-cpu:
[273138.062278] CPU    0: hi:    0, btch:   1 usd:   0
[273138.062280] CPU    1: hi:    0, btch:   1 usd:   0
[273138.062282] Node 0 DMA32 per-cpu:
[273138.062286] CPU    0: hi:  186, btch:  31 usd:   0
[273138.062289] CPU    1: hi:  186, btch:  31 usd:   0
[273138.062291] Node 0 Normal per-cpu:
[273138.062294] CPU    0: hi:  186, btch:  31 usd:   0
[273138.062297] CPU    1: hi:  186, btch:  31 usd:   0
[273138.062305] active_anon:705765 inactive_anon:254745 isolated_anon:0
                 active_file:89 inactive_file:100 isolated_file:0
                 unevictable:0 dirty:0 writeback:3 unstable:0
                 free:8553 slab_reclaimable:2431 slab_unreclaimable:6643
                 mapped:823 shmem:4774 pagetables:4050 bounce:0
                 free_cma:0
[273138.062308] Node 0 DMA free:15632kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15984kB managed:15900kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:40kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[273138.062321] lowmem_reserve[]: 0 3201 3904 3904
[273138.062327] Node 0 DMA32 free:13744kB min:6540kB low:8172kB high:9808kB active_anon:2490620kB inactive_anon:686120kB active_file:312kB inactive_file:300kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3374452kB managed:3279848kB mlocked:0kB dirty:0kB writeback:12kB mapped:2940kB shmem:18728kB slab_reclaimable:7512kB slab_unreclaimable:19620kB kernel_stack:4000kB pagetables:13164kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:3700 all_unreclaimable? yes
[273138.062339] lowmem_reserve[]: 0 0 702 702
[273138.062345] Node 0 Normal free:4836kB min:1436kB low:1792kB high:2152kB active_anon:332440kB inactive_anon:332860kB active_file:44kB inactive_file:100kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:786432kB managed:719824kB mlocked:0kB dirty:0kB writeback:0kB mapped:352kB shmem:368kB slab_reclaimable:2212kB slab_unreclaimable:6912kB kernel_stack:1280kB pagetables:3036kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:908 all_unreclaimable? yes
[273138.062357] lowmem_reserve[]: 0 0 0 0
[273138.062362] Node 0 DMA: 0*4kB 0*8kB 1*16kB (U) 0*32kB 0*64kB 0*128kB 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (U) 3*4096kB (MR) = 15632kB
[273138.062383] Node 0 DMA32: 3296*4kB (UEM) 8*8kB (MR) 17*16kB (R) 0*32kB 1*64kB (R) 2*128kB (R) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 13840kB
[273138.062404] Node 0 Normal: 1137*4kB (EMR) 15*8kB (MR) 2*16kB (R) 1*32kB (R) 2*64kB (R) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 4860kB
[273138.062425] 6049 total pagecache pages
[273138.062428] 1018 pages in swap cache
[273138.062431] Swap cache stats: add 1138680, delete 1137662, find 493695/665941
[273138.062433] Free swap  = 0kB
[273138.062434] Total swap = 525308kB
[273138.062436] 1044217 pages RAM
[273138.062438] 0 pages HighMem/MovableOnly
[273138.062439] 16652 pages reserved
[273138.062441] 0 pages hwpoisoned
[273138.062443] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[273138.062468] [  148]     0   148    10899      927      26       48             0 systemd-journal
[273138.062474] [  171]     0   171    23689        0      17       52             0 lvmetad
[273138.062480] [  178]     0   178    10542       11      22      206         -1000 systemd-udevd
[273138.062486] [  410]   108   410    25077       10      19       51             0 systemd-timesyn
[273138.062492] [  455]     0   455     1095        0       8       38             0 acpid
[273138.062497] [  457]     0   457    69070      408      36       84             0 accounts-daemon
[273138.062503] [  460]     0   460     7465        0      19       61             0 cgmanager
[273138.062508] [  465]     0   465     6507        0      18       59             0 atd
[273138.062514] [  467]   102   467    10722       56      25       64          -900 dbus-daemon
[273138.062519] [  512]     0   512     7075       16      20       59             0 cron
[273138.062524] [  518]     0   518     6301       11      17      179             0 smartd
[273138.062530] [  519]   101   519    64094      240      29      232             0 rsyslogd
[273138.062536] [  523]     0   523     7132       37      19       52             0 systemd-logind
[273138.062541] [  615]     0   615     4864       25      15       48             0 irqbalance
[273138.062546] [  622]     0   622    69274       57      39      127             0 polkitd
[273138.062552] [  623]     0   623     3340        7      11       29             0 mdadm
[273138.062557] [  839]   103   839    81055     3108      76    11814             0 named
[273138.062563] [  852]     0   852     3808        0      13       35             0 agetty
[273138.062569] [  866]   106   866   393554    19089     246    64828             0 mysqld
[273138.062574] [  869]     0   869    16374        6      35      171         -1000 sshd
[273138.062580] [  879]    33   879    13620      602      31      283             0 lighttpd
[273138.062585] [  887]    33   887    64908        0      87      869             0 php-cgi
[273138.062591] [  996]    33   996    67392     1467      87     2273             0 php-cgi
[273138.062596] [  997]    33   997    67696     1845      89     2226             0 php-cgi
[273138.062601] [  998]    33   998    67718      823      89     3254             0 php-cgi
[273138.062607] [  999]    33   999    67734     1720      89     2362             0 php-cgi
[273138.062612] [ 2503]  1000  2503    11253        1      26      153             0 systemd
[273138.062618] [ 2504]  1000  2504    15961        6      34      608             0 (sd-pam)
[273138.062624] [ 2578]  1000  2578     6861      142      17      233             0 screen
[273138.062629] [ 2579]  1000  2579     5509        2      16      507             0 bash
[273138.062634] [ 2593]  1000  2593    38787      892      49     1786             0 irssi
[273138.062640] [ 2657]  1000  2657    15173     2553      32      580             0 mosh-server
[273138.062645] [ 2658]  1000  2658     6587       14      19       57             0 screen
[273138.062651] [10896]  1001 10896    11253       48      26      106             0 systemd
[273138.062657] [10897]  1001 10897    15961        6      34      608             0 (sd-pam)
[273138.062663] [11013]  1001 11013     6786      224      16       68             0 screen
[273138.062668] [11014]  1001 11014     5552      262      16      292             0 bash
[273138.062673] [13729]  1001 13729     5555       77      16      479             0 bash
[273138.062679] [22603]     0 22603    24231        9      51      235             0 sshd
[273138.062684] [22703]  1001 22703    24231       44      50      205             0 sshd
[273138.062690] [22704]  1001 22704     5522        2      15      521             0 bash
[273138.062695] [22719]  1001 22719     6588        8      19       62             0 screen
[273138.062701] [22986]  1001 22986  1057556   917096    1867    16394             0 python
[273138.062804] [27001]     0 27001    13411       83      32       30             0 cron
[273138.062810] [27002]     0 27002     1122        1       8       21             0 sh
[273138.062816] [27005]     0 27005     1123        0       8       23             0 sessionclean
[273138.062821] [27006]     0 27006     1123        1       8       23             0 sessionclean
[273138.062826] [27008]     0 27008     4288        1      12       58             0 sort
[273138.062832] [27009]     0 27009     4288        1      11       59             0 sort
[273138.062837] [27010]     0 27010     1123        0       8       23             0 sessionclean
[273138.062842] [27016]     0 27016     1123        0       8       24             0 sessionclean
[273138.062848] [27017]     0 27017    45942      297      78      260             0 php7.0
[273138.062854] [27019]     0 27019    13411        9      32      104             0 cron
[273138.062859] [27021]  1001 27021     1122        1       8       21             0 sh
[273138.062865] [27023]  1001 27023     2951        9      11       38             0 bash
[273138.062871] [27024]  1001 27024     8496       49      20       64             0 git
[273138.062877] [27226]     0 27226     1122        1       8       23             0 50-motd-news
[273138.062883] [27293]     0 27293    47531     2442      61     2836             0 curl
[273138.062888] [27383]     0 27383    13411       64      32       50             0 cron
[273138.062894] [27385]  1001 27385     1122       20       8        1             0 sh
[273138.062899] [27386]  1001 27386     2950       37      11        9             0 bash
[273138.062905] [27388]  1001 27388     8496       73      20       40             0 git
[273138.062911] [27527]  1001 27527    27219       76      34       47             0 git
[273138.062917] [27598]  1001 27598    27219       89      31       19             0 git
[273138.062922] [27654]     0 27654    13411       68      32       38             0 cron
[273138.062928] [27704]     0 27704     1122       22       8        0             0 sh
[273138.062934] [27705]     0 27705     5125      140      14       17             0 rtm
[273138.062939] [27708]     0 27708    13411       89      32       24             0 cron
[273138.062944] [27711]     0 27711     1122       22       7        0             0 sh
[273138.062947] Out of memory: Kill process 22986 (python) score 823 or sacrifice child
[273138.063047] Killed process 22986 (python) total-vm:4230224kB, anon-rss:3668384kB, file-rss:0kB

So Python needs more than 4.2 GB of virtual memory, and 3.6 GB of
resident memory. To me, it feels like there is something wrong going on
with the NVD files.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com


More information about the buildroot mailing list