[uClibc] Buildroot questions.

[Rob Landley]

So I'm playing with buildroot some more, and writing up a HOWTO, and I get
some questions:

I want to break the target list out of the Makefile.  Editing the makefile
should NOT be a necessary part of configuring things except for advanced users.
As such, a few defaults should be set to newbie-friendly values, and the
target list should be broken out.

The default value changes I propose are:

USE_UCLIBC_SNAPSHOT:=false
USE_BUSYBOX_SNAPSHOT:=false

For the simple reason that anybody who knows enough to use that knows enough
to change them (or can figure it out pretty quick).  A newbie's first
experience should probably be with a release version.

Just about all the other configuration you do is either compiler optimizations
(for which hacking the makefile is relatively obvious), or selecting targets.
There is one exception: USE_UCLIBC_TOOLCHAIN.

Question: What's USE_UCLIBC_TOOLCHAIN for?  I mean, what's the alternative?
Can anything after this point build if you DON'T first build the UCLIBC
toolchain?  (Does the rest of buildroot do anything useful if
USE_UCLIBC_TOOLCHAIN is false?  What will it do, build a root filesystem for
glibc instead?  Is this useful?  If so, shouldn't that just be a
glibc_toolchain target in that case that could be swapped out for
uclibc_toolchain?  If it doesn't build ANY toolchain, why not just remove the
uclibc_toolchain target from the targets list and document that?)

Note that the USE_UCLIBC_TOOLCHAIN define is used in Makefile as a
pointless guard around "TARGETS+=uclibc_toolchain" (we're already asking
them to add and remove TARGETS from the makefile as the primary configuration
mechanism; if they want to remove it why not just have them do so?)
And then again it's used as a guard at the top of uclibc_toolchain.mk
(#ifdefing out the whole file if it's not set), which is pointless if you
already #ifdefed out the include...

There are also some checks for it in gcc_target.mk.  (I believe gcc_target
makes the native uclibc compiler inside the root_fs, and the uclibc_toolchain
makes the cross-compiler on the host system, right?  Or is it more complicated
than that?)  The use of USE_UCLIBC_TOOLCHAIN in gcc_target largely seems to
skip redundant steps (like the first gcc build step, and building binutils)
if the toolchain build already did them.  It might be possible to avoid this
with more intelligent dependencies, or if it is really needed then maybe it's
possible to have uclibc_toolchain set it when it's included/run.  (And
again, what is buildroot used for when the toolchain is NOT built?)

Now we get to a new target list selection mechanism.  There are a couple of
options.  One is to just break it out into a separate "config" file that gets
included from the Makefile, still using make systax (so a bunch of
TARGETS+=blah lines).  That's a minimal disruption, and about a ten minute
job.  It also naturally preserves target ordering, which is nice if you
want uclibc_toolchain to build first and ext2root to run last.

A second thought is to pull a fast one on the "make" directory.  Have every
*.mk file add itself to the TARGETS list as it's included.  Rename the existing
make directory "examples", and create a new empty directory for "make".

Downsides:
  - directory renames bork CVS because CVS is dumb.  (So call the new
    directory "config" and rename the include.)
  - Ordering isn't naturally preserved.  (This could be fixed by putting
    proper inter-package dependencies in the makefiles, so openssh knows it
    needs openssl, and everything knows it needs a toolchain, and then make
    does the ordering for us.  That might be a good idea anyway...)

Upsides:
  - People have to look in the "make" directory to see what their options are,
    this gives them an incentive.
  - uclibc_toolchain.mk could set USE_UCLIBC_TOOLCHAIN to true when it's
    included, hence cleaning up gcc_target becomes unnecessary.

In either case, moving the targets to a file or to a directory, it becomes
MUCH easier to slap together an automated tool to configure a build, ala
menuconfig.

if we do this cleanup, buildroot might actually be ready for a numbered release
version.  (Presumably the same version number as the uclibc it uses...)

Rob

(More question comparing uclibc_toolchain with gcc_target...  If there's
a guard against recompiling binutils
-------------- next part --------------
The buildroot package automates the creation of a uClibc-only development
system.  This can be run under an existing linux system via a "user mode
linux" kernel and a loopback mounted root partition, or booted and run as
a full fledged linux system.

To create one of these suckers, follow these steps:

1) Download the buildroot package from

     http://www.uclibc.org/cgi-bin/cvsweb/buildroot/buildroot.tar.gz?tarball=1

   (Buildroot does not have releases yet, the above link will grab a snapshot
   of the current CVS version.)

2) Unpack the tarball and read the README file.

3) Edit the Makefile and adjust any options you feel like.  The Makefile
   has four sections:

   A) configuration options.

      A good first guess here is to set both USE_UCLIBC_SNAPSHOT and
      USE_BUSYBOX_SNAPSHOT to false, and leave everything else alone your
      first time.

   B) TARGETS list.

      This is the list of what to build.  Most of the interesting configuration
      is done by adding packages to the TARGETS list.  Step 4 will talk all
      about this; we'll come back to it.

   C) Root filesystem type.

      A special target at the end of your build creates a filesystem image
      file.  (An image file is a file that contains a virtual disk image,
      I.E. a bunch of virtual sectors all in a row, formatted as a filesystem,
      just like it would look like if it were on a block device like a ramdisk,
      hard drive, or CD-ROM.  You may have used an ISO cd-rom image file to
      burn a CD, or a floppy image file to create a boot diskette.  This is
      similar.  Step 7 of this HOWTO explains how to use this image file
      once the build has created it.)

      Since this image file gets formatted, we have to select the filesystem
      type to format it as.  EXT2 is the standard Linux filesystem type, and
      that's the default.  You can also select cramfs (a compressed read-only
      filesystem, suitable for burning into ROM), or jffs2 (a filesystem
      designed for use with flash memory).

      You can leave this at the default (ext2root) your first time

   D) Everything else.

      There's generally no reason to play with the rest of the Makefile.
      Someday, the first three sections may be broken out into a seperate
      configuration file that's included from the Makefile; in the meantime,
      just ignore the second half of the file unless you're curious how it
      works.


4) Select the packages you want to build.

   The Makefile contains a number of example targets.  You almost always want
   "uclibc_toolchain", which builds three things:

     A) uclibc itself
     B) gcc (configured to link executables against uclibc instead of glibc).
     B) the binutils package (linker, assembler, and other things needed
        to actually make executables and libraries).

   You need the uclibc toolchain to build anything else.

   Beyond the toolchain, an absolutely minimal system will contain the
   targets "busybox" and "tinylogin", to allow you to login and execute
   standard unix commands.  (Busybox is another project in the spirit of
   uclibc, re-implementing tiny versions of most of the standard unix command
   line utilities, and in the process providing a substitute for half a dozen
   traditional packages like fileutils, textutils, sh-utils, sysvinit, bash...)

   Adding targets appends them to the end of the list, and they get built in
   order.  This is important because some packages have prerequisites; for
   example, if you want openssh you must first make zlib and openssl.  Good
   documentation on this sort of thing may be found on the Linux From Scratch
   website: http://www.linuxfromscratch.org

   The make file contains a number of sample targets, including a group of
   packages required to make a development with enough tools to compile
   most other packakges, debugging tools, and network configuration tools
   needed to make a standalone firewall or smart router.

   The complete list of available targets can be found in the "make"
   subdirectory: there are a bunch of makefiles in there that know how to
   build existing packages under the uclibc environment created by buildroot.
   All of makefiles in this directory are automatically included from the
   top level Makefile (the line is "include make/*.mk"), and the target
   to trigger each one is usually found near the end of the makefile in
   question.  (For example, in "autoconf.mk", the target is "autoconf:",
   so you'd add "TARGETS+=autoconf" to the target list section of the top
   level Makefile in order to have buildroot build it for you.)

5) Optionally copy any of the tarballs that buildroot needs into the sources/dl
   directory.  If, for example, you already have the gcc-3.2 tarball or the
   linux-2.4.19 tarball, just copy them into place so you can avoid having
   buildroot download them once again from the net.

   Note that if you would like to download all the source for your selected
   targets immediately (useful on a dialup connection, or taking a laptop
   with you on a plane), the command is "make source".  (Be sure to configure
   your target list before doing this.)  Once the build system has downloaded
   a package, it keeps it around for future builds.  A "make clean" won't
   delete downloaded source packages (although a "make distclean" will).

   By default, the build system will download each package as it finds it
   needs it (using the "wget" command, which most modern Linux distributions
   have).

6) Run 'make' and wait a long time...  There is a lot of stuff that needs to
   be compiled, so be patient.  Go to the kitchen and make yourself a nice
   sandwich.  Since buildroot will download any source files it needs which
   are not already sitting in the sources/dl directory, if you have a slow net
   connection you might consider starting this before you go to bed in the
   hope that it will be done when you wake up. ;-)

7) When buildroot finishes, there will be a new Linux system sitting in
   build_i386/root (or whatever architecture your're building for).  If you're
   an old hand at installing filesystems by hand, you can copy it to a bootable
   partition, and use a boot disk or play with lilo or grub.  (One easy way to
   copy files while preserving permissions and links and such is to use a pipe
   between two instances of tar, ala:

     tar cC build_i386/root . | tar xvC /destinationdir

   To learn how to make a partition bootable, read the Linux Bootdisk HOWTO at
   "http://www.tldp.org/HOWTO/Bootdisk-HOWTO/".

   For pure testing purposes, you can chroot into the new system.  While
   logged in as root, go to the buildroot directory (where the Makefile is)
   and type "chroot build_i386/root /bin/sh --login", then try running stuff.
   Type "exit" when you want to return to your original Linux system.  See
   "man chroot" for more info.

8) If you told the build to create any of the filesystem image targets, you
   can loopback mount them and chroot into them.  As root, try something
   vaguely like:

     mkdir temp
     mount -o loop root_fs temp
     chroot temp /bin/sh --login

   Type "exit" when you're done, and don't forget to "umount temp".

   If you would like to make an initial ram disk, mount the root_fs, create a
   file "initrc" in the root directory of root_fs (which is a shell script that
   will run in instead of init), umount the root_fs, and then compress the
   root_fs file with gzip.  You're done.  Add "initrd=/path/to/root_fs" to
   the linux kernel's command line next time you boot, and it should use
   your new initial ramdisk.  (Note that if your initrc script exits, the boot
   will continue, mounting the normal root filesystem and running init.  For
   more information, see the bootdisk howto.)

   If you used the "user-mode-linux" target, the build process will create a
   special linux kernel that can be run as a normal application program.
   Just run the "UMlinux" executable; it will automatically mount the
   root_fs image and pop up some Xterm windows with which to interact with it.
   For more information, go to "http://user-mode-linux.sourceforge.net".