[Buildroot] [PATCH v2 08/18] package/pkg-generic.mk: move python fixup to generic package infrastructure
Herve Codina
herve.codina at bootlin.com
Thu Jul 8 15:38:34 UTC 2021
Hi,
On Wed, 7 Jul 2021 22:12:57 +0200
"Yann E. MORIN" <yann.morin.1998 at free.fr> wrote:
> Hervé, Arnout, All,
>
> On 2021-07-07 14:49 +0200, Herve Codina spake thusly:
> > On Wed, 7 Jul 2021 14:21:16 +0200
> > Arnout Vandecappelle <arnout at mind.be> wrote:
> > > > These finds are now called for all packages and .../lib/python* can be not
> > > > present. In this case, find fails.
>
> OK, I see.
>
> However, the commit log was lacking these explanations, because the
> change is not simply "the python fixup is moved to the generic package
> infrastructure", because the fixups had to be adapted.
>
> > > A simpler way of doing that would be:
> > > $(if $(wildcard $(HOST_DIR)/lib/python* $(STAGING_DIR)/usr/lib/python*),
> > > $(Q)find $(HOST_DIR)/lib/python* $(STAGING_DIR)/usr/lib/python*
> > I think this does not solve the problem.
> > find fails if at least one of the 2 paths are not present.
> >
> > Something like this (not tested) is needed.
> >
> > $(if $(wildcard $(HOST_DIR)/lib/python* $(STAGING_DIR)/usr/lib/python*),
> > $(Q)find $(wildcard $(HOST_DIR)/lib/python* $(STAGING_DIR)/usr/lib/python*)
>
> Yes, this would keep the pre-existing behaviour.
>
> However, I think the traditional way we usually do that would be with a
> foreach:
>
> $(foreach dir, $(wildcard $(HOST_DIR)/lib/python* $(STAGING_DIR)/usr/lib/python*), \
> $(Q)find $(dir) -name "_sysconfigdata*.py" -print0 \
> |xargs -0 --no-run-if-empty \
> $(SED) "s:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/:g"
> $(Q)find $(dir) -name "_sysconfigdata*.pyc" -delete
> )
>
> And thus you're back to a basic, non-parameterised macro that can be
> used as a POST_PREPARE_HOOK. Wee! :-)
>
> Note: optimisation: we should be able to use a single find call to do
> both, though:
>
> $(foreach dir, $(wildcard $(HOST_DIR)/lib/python* $(STAGING_DIR)/usr/lib/python*), \
> $(Q)find $(dir) \( -name "_sysconfigdata*.pyc" -delete \) \
> -o \( -name "_sysconfigdata*.py" -print0 \) \
> |xargs -0 --no-run-if-empty \
> $(SED) 's:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$($(PKG)_NAME)/:g'
> )
>
> Maybe the groupings '\( ... \)' are superfluous. Adapt appropriately...
>
I tried this kind of things and I ran into trouble with $(wildcard ...)
Indeed, $(wildcard ...) does not see my directories.
A simple 'ls $(HOST_DIR)/lib/python*' just before $(foreach dir, $(wildcard ...), ...)
lists the directory.
Suppose this stupid package:
---- 8< -----
define TLP_PKGA_INSTALL_TARGET_CMDS
echo "$$(date)" > $(TARGET_DIR)/pkga1
echo "$$(date)" > $(TARGET_DIR)/pkga2
mkdir -p $(TARGET_DIR)/foo/bla
touch $(TARGET_DIR)/foo/bla/file1
touch $(TARGET_DIR)/foo/bla/file2
endef
define TLP_PKGA_LIST_CMD
echo "ls:" && ls $(TARGET_DIR)/foo/* || true
echo "wildcard: $(wildcard $(TARGET_DIR)/foo/*)"
endef
TLP_PKGA_PRE_INSTALL_TARGET_HOOKS += TLP_PKGA_LIST_CMD
TLP_PKGA_POST_INSTALL_TARGET_HOOKS += TLP_PKGA_LIST_CMD
$(eval $(generic-package))
---- 8< -----
'make pkg' leads to:
>>> pkg Installing to target
echo "ls:" && ls /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/foo/* || true
ls:
ls: cannot access '/home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/foo/*': No such file or directory
echo "wildcard: "
wildcard:
echo "$(date)" > /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/pkga1
echo "$(date)" > /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/pkga2
mkdir -p /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/foo/bla
touch /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/foo/bla/file1
touch /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/foo/bla/file2
echo "ls:" && ls /home/hcodina/project/nagra/buildroot/output/per-package/tlp_pkga/target/foo/* || true
ls:
file1 file2
echo "wildcard: "
wildcard:
$(wildcard ...) uses internal make caching.
Depending on when it is called related to make internal caching updates, it can
lead to unexpected results.
In the example provided, make cannot known that $(TARGET_DIR)/foo/bla was updated by
TLP_PKGA_INSTALL_TARGET_CMDS and so it didn't update its internal caching.
$(wildcard ...) does not see the new files.
Herve
--
Hervé Codina, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
More information about the buildroot
mailing list