speeding up buildroot compile time for multiple users...

[Mathieu Deschamps]

Hello,

How interesting is this subject ! 

On Wednesday 28 June 2006 04:10, Clem Taylor wrote:
> I've been trying to figure out a good way to speed up the buildroot
> compile time in a multiple developer environment by not requiring
> everyone to build everything from scratch. We don't change the
> toolchain and uclibc very often (and many of the tools), so people
> shouldn't have to build everything from scratch just to build from a
> new view.

Avoiding n recompilations and n buildroot trees for n developpers is
a sure way to rationnalization.

Why not use simple unix properties : having a buildroot account (user,group) 
and making user belonging to group buildroot ? While buildroot user
would be buildroot supervisor account (the one whom build prior toolchain),
all user belonging to its group could be developpers  (whose trees derivate 
from supervisor one) or other projects. 

This layout may spread across machines :

 On Wednesday 28 June 2006 11:35, Ayoub wrote:
> Try to use distcc and ccache  :
> http://distcc.samba.org/
> http://ccache.samba.org/


Indeed this is a matter of configuration and a goal could be :

	1. Dedication

A change in terms of configuration, in other words BUILD_DIR (existing),
TARGET_DIR(existing), PACKAGE_DIR (needs to be introduced !)  would 
have to be variable on $USERNAME basis. These could actually come from
a $USERNAME profile don't you think ? Now its fixed on buildroot first 'world 
building'  (in package/Makefile.in or in PACKAGE_DIR/Makefile.in !)

	2. Sharing

A common toolchain and configurations set with a common tool which 
would warn $USERNAME not to modify buildroot .config option having
an impact on common parts. 


For example, actually I've made up myself this config. tool that complies 
partly with 2. Okay that's also half a step.
I use it just as version control system, its based on a date stamping. For 
example, I have theses files to charaterize my buildroot "releases" and
can add new one to precise it:
#       uClibc-config-2005-12-02	#C
#       uboot-config-2005-12-02		#
#       device_table-2005-12-02.txt	#
#       busybox-config-2005-12-02	#
#       buildroot-config-2005-12-02	#C
#	myelement-config-date		#

But for now nothing prevent $USERNAME to modify supervisor common 
parts. Yet if I cleverly just modify busybox-config-# then common
part are preserved. Obviously some rules may assert this, I don't know 
something like : "Don't touch to C marked options files or I shout" or
"Inform supervisor I like to submit a change in this C marked options files".



On Wednesday 28 June 2006 16:27, Mark S. Mathews wrote:
> Use two buildroot trees.
>
> One tree builds _only_ the toolchain and buildhost tools you need and  
> installs them to a fixed location (e.g. /opt/buildroot/<toolchainID>)  
> via the STAGING_DIR et.al. config variables.

> The second tree uses the tools from the first and has a different  
> STAGING_DIR etc. that's local to the given developer.
>

Interesting, but it has to have a _whole_ different BUILD_<arch> dir : if one
developper builds this specific version of foo package whereas his colleague
build that other version of the same package : they can't share same package
and build_arch/package-version ! 

> This way you can rebuild the kernel/apps/rootfs at will without  
> requiring a rebuild of the toolchain (or even an inspection of their  
> source trees).

The way it is now is already able to do so (except prevent inspection
but this is epsilon time share the whole process)


Thanks for your attention


Best Regards,
 
Mathieu Deschamps
Com2gether Design Center
Electronic and Embedded Engineering Services
www.com2gether.net