[Buildroot] [PATCH v2 0/7] CPE validation

[Thomas Petazzoni]

Hello,

We have recently merged support to allow packages to express their CPE
identifier, which is a unique identifier used by the NIST database to
identify software products, and associate CVEs to them.

This patch series extends that by ensuring that CPEs affected to
packages in Buildroot do exist in the official CPE dictionary from
NIST, and if they do not exist, assist in generating the XML snippet
that the CPE dictionary maintainers expect.

In more details:

 - PATCH 1 introduces a new "name" field in the show-info JSON output,
   with the raw name of the package. This is used in PATCH 2.

 - PATCH 2 fixes "make pkg-stats" to properly account for host
   packages, which were currently ignored. It makes use of the new
   "name" field introduced in PATCH 1 in the show-info JSON output.

 - PATCH 3 introduces support/scripts/cpedb.py, which is a new set of
   helper that is capable of downloading the CPE dictionary, parsing
   it, and generating XML snippets to submit new CPE entries.

 - PATCH 4 modifies support/scripts/pkg-stats to leverage the features
   of cpedb.py to include validation of CPE identifiers of Buildroot
   patches.

 - PATCH 5 adds support/scripts/gen-missing-cpe, which allows to
   generate the XML snippets expected by the CPE dictionary mainainers

 - PATCH 6 hooks up support/scripts/gen-missing-cpe as the "make
   missing-cpe" top-level make target

 - PATCH 7 extends the documentation with more details about
   Buildroot features related to vulnerability management.

Changes since v1:

 - Drop patches that have been merged upstream.

 - New patches adding the "name" property in the show-info JSON
   output, and using it so that "make pkg-stats" takes into account
   host packages.

 - Fix the parsing of the XML CPE databse for Python versions prior to
   3.7. This was reported by Matt, and fixed by Grégory Clement.

 - Improve how the namespaces are added to the XML file generated by
   "make missing-cpe", with the proper API instead of using a
   regexp. This was done by Grégory Clement.

 - Add encoding in the XML declaration found in the XML file generated
   by "make missing-cpe". This was done by Grégory Clement.

 - Reduced the size of the CPE database cache, with the side effect of
   also having prettier output for "make missing-cpe". This was done
   by Grégory Clement.

This is based on work from Matt Weber, which has been further
refactored/rewritten by Grégory Clement and myself.

To test this:

 $ make pkg-stats
 $ firefox output/pkg-stats.html
 $ make missing-cpe
 $ ls output/cpe-updates/

This work is also available at:

  https://github.com/tpetazzoni/buildroot/commits/cpe-validation

Thomas

Matt Weber (2):
  support/scripts/cpedb.py: new CPE XML helper
  support/scripts/gen-missing-cpe: add new script

Thomas Petazzoni (5):
  package/pkg-utils.mk: introduce "name" field in show-info output
  support/scripts/pkg-stats: properly handle host packages with -c
    option
  support/scripts/pkg-stats: check CPE existence in CPE dictionnary
  Makefile: add new missing-cpe target
  docs/manual: add details about vulnerability management

 Makefile                        |   9 ++
 docs/manual/common-usage.txt    |  42 +++++++
 package/pkg-utils.mk            |   1 +
 support/scripts/cpedb.py        | 203 ++++++++++++++++++++++++++++++++
 support/scripts/gen-missing-cpe |  65 ++++++++++
 support/scripts/pkg-stats       |  32 +++--
 6 files changed, 342 insertions(+), 10 deletions(-)
 create mode 100644 support/scripts/cpedb.py
 create mode 100755 support/scripts/gen-missing-cpe

-- 
2.29.2