[Buildroot] [PATCH 0/5] Runtime testing infrastructure
Ricardo Martincoski
ricardo.martincoski at gmail.com
Thu Mar 2 01:45:00 UTC 2017
Thomas,
On Tue, Feb 07, 2017 at 09:35 AM, Thomas Petazzoni wrote:
> At last, here is a first version of a small runtime testing
> infrastructure for Buildroot. The first patch introduces the
> infrastructure itself, the other 4 patches just add a few patches.
Thank you very much for this series.
In overall, the infra works very well.
I tested it and I will reply to each patch.
Here is a summary of what I found:
1) emulator supports only arm, making test_iso9660 testcases to fail;
2) testcases using i386 emulated system can fail occasionally due to a message
from the kernel;
3) the testcases *Internal from test_iso9660 expect the rootfs to be read-only,
I am not sure this is correct or not, but it makes the tests fail and it is
different from your previous iteration (before the stylistic rework);
4) output dir needs to be an absolute path;
5) there are some emulator.login("root") that I think should be .login()
because BR2_TARGET_GENERIC_ROOT_PASSWD it not set, except for dropbear;
6) using "-q" with commands to the emulated system is unnecessary and make
debug harder when a testcase fails;
7) the code is not compatible to python 3
- print without parenthesis
- exception handling using comma
- importing urllib2 (split in libraries in python3)
- probably will need .encode() and .decode() when using telnetlib because of
the bytes vs. string introduced in python 3
This is what I think about this list:
(1) and (2) should be fixed;
(3) should be checked and fixed if it is not correct;
(4), (5) and (6) can wait;
(7) should *not* be fixed now, but in a followup patch (it can be postponed
until 2020, when python 2 probably will retire);
> ./support/testing/run-tests -d ~/dl/ -o ../outputs/ tests.fs.test_ubi.TestUbi
Using relative path for output dir seems to cause problems with few testcases,
as tests.core.test_post_scripts.TestPostScripts .
Regards,
Ricardo
More information about the buildroot
mailing list