[Buildroot] [RFC PATCH] host-localedef: Compile against glibc

Samuel Mendoza-Jonas sam at mendozajonas.com
Wed Aug 1 01:31:16 UTC 2018


On Tue, 2018-07-31 at 22:55 +0200, Thomas Petazzoni wrote:
> Hello Samuel,
> 
> Thanks for taking a look into this!
> 
> On Mon, 30 Jul 2018 15:39:36 +1000, Samuel Mendoza-Jonas wrote:
> > In glibc 2.27 the following change occured:
> > "Statically compiled applications attempting to load locales compiled
> > for the GNU C Library version 2.27 will fail and fall back to the
> > builtin C/POSIX locale."
> 
> I don't really understand this explanation. Isn't the problem that
> glibc 2.27 doesn't read properly locale files generated by localedef
> from glibc 2.24 ?
> 
> I kind of fail to parse the explanation given here :-/

I think it's a bit of both; 2.27 introduces new locale descriptions which
localedef can't handle, eg:

Generating locale ru_RU.UTF-8
/scratch/jenkins-workspace/op-build-upstream/output/host/powerpc64le-buildroot-linux-gnu/sysroot/usr/share/i18n/locales/ru_RU:125: LC_TIME:syntax error
/scratch/jenkins-workspace/op-build-upstream/output/host/powerpc64le-buildroot-linux-gnu/sysroot/usr/share/i18n/locales/ru_RU:149: LC_TIME:syntax error

And we fall back to the C locale at runtime (at least in my environment).
I'll stare at the glibc release notes / bug report a bit more and see if
I can make a more legible explanation.

> 
> > This impacts us since upstream buildroot uses a localdef built against
> > an older eglibc release [0].
> > Update the host-localedef package to build against glibc 2.27.
> > Unfortunately we can't compile just the localedef utility, but even so
> > this does not increase the build time too significantly. eg:
> > 
> > With glibc localedef:
> > real    33m11.278s
> > user    171m51.684s
> > sys     19m46.528s
> > 
> > With eglibc localedef:
> > real    31m12.715s
> > user    155m13.616s
> > sys     16m23.504s
> 
> Meh, two more minutes just to build some not so complicated user-space
> program :-/
> 
> > -HOST_LOCALEDEF_CONF_ENV = CFLAGS="$(HOST_CFLAGS) -fgnu89-inline"
> > +# glibc is part of the toolchain so disable the toolchain dependency
> > +HOST_LOCALEDEF_ADD_TOOLCHAIN_DEPENDENCY = NO
> 
> Not needed for host-localedef.
> 

Thanks for noting this and the others below; I was sure a lot of this was
target-only but I figured someone would be able to notice a few of them
better than me :)

> > -# The makefile does not implement an install target
> > +HOST_LOCALEDEF_SUBDIR = build
> > +
> > +# Thumb build is broken, build in ARM mode
> > +ifeq ($(BR2_ARM_INSTRUCTIONS_THUMB),y)
> 
> This is a target related option...
> 
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -marm
> 
> ... so it is not relevant to observe it when building a host program.
> 
> > +endif
> > +
> > +# MIPS64 defaults to n32 so pass the correct -mabi if
> > +# we are using a different ABI. OABI32 is also used
> > +# in MIPS so we pass -mabi=32 in this case as well
> > +# even though it's not strictly necessary.
> > +ifeq ($(BR2_MIPS_NABI64),y)
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -mabi=64
> > +else ifeq ($(BR2_MIPS_OABI32),y)
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -mabi=32
> > +endif
> 
> Same comment.
> 
> > +ifeq ($(BR2_ENABLE_DEBUG),y)
> > +HOST_LOCALEDEF_EXTRA_CFLAGS += -g
> > +endif
> 
> Same comment.
> 
> > +
> > +# The stubs.h header is not installed by install-headers, but is
> > +# needed for the gcc build. An empty stubs.h will work, as explained
> > +# in http://gcc.gnu.org/ml/gcc/2002-01/msg00900.html. The same trick
> > +# is used by Crosstool-NG.
> > +ifeq ($(BR2_TOOLCHAIN_BUILDROOT_HOST_LOCALEDEF),y)
> > +define HOST_LOCALEDEF_ADD_MISSING_STUB_H
> > +	mkdir -p $(STAGING_DIR)/usr/include/gnu
> > +	touch $(STAGING_DIR)/usr/include/gnu/stubs.h
> 
> STAGING_DIR is observed when building target packages, not host
> packages, so this seems useless.
> 
> > +endef
> > +endif
> > +
> > +# Even though we use the autotools-package infrastructure, we have to
> > +# override the default configure commands for several reasons:
> > +#
> > +#  1. We have to build out-of-tree, but we can't use the same
> > +#     'symbolic link to configure' used with the gcc packages.
> > +#
> > +#  2. We have to execute the configure script with bash and not sh.
> > +#
> > +# Note that as mentionned in
> > +# http://patches.openembedded.org/patch/38849/, glibc must be
> > +# built with -O2, so we pass our own CFLAGS and CXXFLAGS below.
> > +define HOST_LOCALEDEF_CONFIGURE_CMDS
> > +	mkdir -p $(@D)/build
> > +	# Do the configuration
> > +	(cd $(@D)/build; \
> > +		$(HOST_CONFIGURE_OPTS) \
> > +		CFLAGS="-O2 $(HOST_LOCALEDEF_EXTRA_CFLAGS)" CPPFLAGS="" \
> > +		CXXFLAGS="-O2 $(HOST_LOCALEDEF_EXTRA_CFLAGS)" \
> > +		$(SHELL) $(@D)/configure \
> > +		ac_cv_path_BASH_SHELL=/bin/bash \
> > +		libc_cv_forced_unwind=yes \
> > +		libc_cv_ssp=no \
> > +		--target=$(GNU_HOST_NAME) \
> > +		--host=$(GNU_HOST_NAME) \
> > +		--build=$(GNU_HOST_NAME) \
> > +		--prefix=/usr \
> 
> Should be --prefix=$(HOST_DIR)

Oh yes, will fix.

> 
> > +		$(if $(BR2_SOFT_FLOAT),--without-fp,--with-fp) \
> > +		$(if $(BR2_x86_64),--enable-lock-elision) \
> 
> Both BR2_SOFT_FLOAT and BR2_x86_64 are target options, so observing
> them when building a host package is almost always wrong.
> 
> > +		--with-pkgversion="Buildroot" \
> > +		--without-cvs \
> > +		--disable-profile \
> > +		--without-gd \
> > +		--enable-obsolete-rpc)
> > +	$(HOST_LOCALEDEF_ADD_MISSING_STUB_H)
> > +endef
> 
> With all this, aren't we building a full glibc for the host ?
> 
> Can you use the locale/others and locale/install-others make targets
> instead ?
> 
> See how they are doing it in PTXdist:
> https://git.pengutronix.de/cgit/ptxdist/tree/rules/host-localedef.make.
> Most likely using the locale/others make target will help cut the build
> time significantly.

Yes this is the contentious part, we're building a whole glibc. Unfortunately
if we want to build just the locale/others target like PTXdist we also need
their bonus patch to glibc:
https://git.pengutronix.de/cgit/ptxdist/tree/patches/localedef-glibc-2.27/0001-HACK-only-build-and-install-localedef.patch

Otherwise the build fails with, for example:

make[4]: *** No rule to make target '/scratch/builds/locales/build/host-localedef-glibc-2.27-57-g6c99e37f6fb640a50a3113b2dbee5d5389843c1e/build/elf/soinit.os', needed by '/scratch/builds/locales/build/host-localedef-glibc-2.27-57-g6c99e37f6fb640a50a3113b2dbee5d5389843c1e/build/libc.so'.  Stop.
Makefile:215: recipe for target 'locale/others' failed
make[3]: *** [locale/others] Error 2
Makefile:9: recipe for target 'locale/others' failed
make[2]: *** [locale/others] Error 2

So the tradeoff is adding a few minutes of extra build time versus carrying
a bit of a hack to enable just the localedef/others target.

> 
> Thanks!
> 
> Thomas




More information about the buildroot mailing list