[Buildroot] [PATCH 9/9] manual: rework Deploying images chapter

[Arnout Vandecappelle]

On 10/05/13 21:49, Samuel Martin wrote:
> Add details about how to deploy images generated by Buildroot on real
> hardware, emulator or VM, over network (TFTP, NFS, PXE).
>
> Signed-off-by: A.R.D. <contact at team-ard.com>
> Signed-off-by: Samuel Martin <s.martin49 at gmail.com>

  Some more spelling corrections:

>
> ---
> changes v2 -> v3:
> - typos and minor rewordings (Thomas De S. and Arnout V.)
> - typos + formating fixes + minor rewordings
> - rename beyond-buildroot.txt -> deploying-images.txt
> - move booting image chapter as chapter #4
> - add missing NFS links
> - enhance NFS boot section
> - add section about TFTP boot
> - add section about preparing custom disk image
>
> changes v1 -> v2 (Samuel):
> - split patch
> - rephrase commit message
> - wrap line at 70-80 chars
> - misc. typo and formating fixes
> - misc. rewordings
> - re-order the "Network boot" section
> - add a word about qemu targets
> - enhance section about disk image generation
> - enhance section about NFS boot + add links
> - keep "Beyond Buildroot" at the end of the manual
> - add cross-refs to "Beyond Buildroot"
>
> Signed-off-by: Samuel Martin <s.martin49 at gmail.com>
> ---
>   docs/manual/beyond-buildroot.txt |  41 ------
>   docs/manual/deploying-images.txt | 290 +++++++++++++++++++++++++++++++++++++++
>   docs/manual/manual.txt           |   4 +-
>   docs/manual/using.txt            |   3 +-
>   4 files changed, 294 insertions(+), 44 deletions(-)
>   delete mode 100644 docs/manual/beyond-buildroot.txt
>   create mode 100644 docs/manual/deploying-images.txt
>
> diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
> deleted file mode 100644
> index a0d4af0..0000000
> --- a/docs/manual/beyond-buildroot.txt
> +++ /dev/null
> @@ -1,41 +0,0 @@
> -// -*- mode:doc; -*-
> -// vim: set syntax=asciidoc:
> -
> -Beyond Buildroot
> -================
> -
> -Boot the generated images
> --------------------------
> -
> -NFS boot
> -~~~~~~~~
> -
> -To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
> -images_ menu.
> -
> -After a complete build, just run the following commands to setup the
> -NFS-root directory:
> -
> --------------------
> -sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
> --------------------
> -
> -Remember to add this path to +/etc/exports+.
> -
> -Then, you can execute a NFS-boot from your target.
> -
> -Chroot
> -------
> -
> -If you want to chroot in a generated image, then there are few thing
> -you should be aware of:
> -
> -* you should setup the new root from the _tar root filesystem_ image;
> -
> -* either the selected target architecture is compatible with your host
> -  machine, or you should use some +qemu-*+ binary and correctly set it
> -  within the +binfmt+ properties to be able to run the binaries built
> -  for the target on your host machine;
> -
> -* Buildroot does not currently provide +host-qemu+ and +binfmt+
> -  correctly built and set for that kind of use.
> diff --git a/docs/manual/deploying-images.txt b/docs/manual/deploying-images.txt
> new file mode 100644
> index 0000000..ad77f09
> --- /dev/null
> +++ b/docs/manual/deploying-images.txt
> @@ -0,0 +1,290 @@
> +// -*- mode:doc; -*-
> +// vim: set syntax=asciidoc:
> +
> +[[deploying-images]]
> +Deploying target images built with Buildroot
> +============================================
> +
> +After having run Buildroot, you will have a brand new filesystem for

  kernel, bootloader and filesystem?

> +your target exported in the 'output/images' directory.
> +The content of this directory depends on the selected options in the
> +Buildroot configuration, especially those from the +Filesystem images+
> +sub-menu.

  +Kernel+, +filesystem images+ and +bootloaders+ menus (they're not 
sub-menus).

> +
> +So what's next? You will probably want to :

  You probably want to do one of the following.

* Copy and install the images on the target device to boot and test it.

* Write the images to removable media that can be booted on the target
   device.

* Boot and test the images...

> +
> +* deploy and/or install the freshly built images on the target to boot
> +  and test it;
> +
> +* boot and test the images in emulators (http://wiki.qemu.org/Main_Page[Qemu],
> +  http://www.linaro.org/engineering/engineering-projects/armv8[Foundation_v8]
> +  --- a AArch64 emulator, ...);

  an AArch64 emulator

> +
> +* generate a virtual disk to dump to real system or to use in
> +  virtualization systems (http://wiki.qemu.org/Main_Page[Qemu],
> +  https://www.virtualbox.org/[VirtualBox], ...).
> +  This is mostly useful for 'x86' and 'x86_64' targets.

  'i386' and 'x86_64' target architectures.

> +
> +This part of the work is really depending on each project and
> +hardware, so we cannot describe every solutions here. This is where
> +Buildroot's work ends.
> +
> +For some of the off-the-shelf boards that Buildroot supports, you can
> +find a specific explanation in a readme file in the _board_ directory.
> +
> +This chapter aims at guiding new users on what to do next to avoid
> +staying in the dark.
> +
> +This is *not* an exact guide on how to precisely do what is described.
> +Please take the time to have a look to referred projets to get those
> +details.

  These six sentences should be one paragraph rather than four. There's 
also a bit of redundancy. Together with a little rephrasing and spelling 
correction that becomes:

This part of the work is really depending on each project and hardware,
so we cannot describe every solution here. This is where Buildroot's
work ends. The rest of this chapter gives some general examples and
hints about how the images can be deployed to the target device. This is
_not_ an exact guide and doesn't provide full details. Please take the
time to have a look to referred projects to get those details.


> +
> +
> +Deploying images on the target hardware
> +---------------------------------------
> +
> +Buildroot comes with a set of minimal configurations for allowing to

  that allow

> +boot various existing targets, from different vendors.
> +
> +All these configurations are stored in the 'configs/' directory.
> +Most of these configurations also come with a directory:

  Remove the :

> +'board/<vendor name>/<platform name>', for additional resources.
> +Some of them also contain a +readme.txt+ file with information about

  'readme.txt'

> +flashing and/or booting the given platform.
> +
> +Although, deploying the generated images on the actual hardware
> +(flashing and booting the hardware) is very tight to the target
> +hardware and can differ a lot from one target to another.
> +Buildroot cannot provide information on this part.
> +
> +For deploying images on the actual target, refer to the hardware
> +vendor instructions, or the readme file in the _board_ directory if
> +any.

  I don't like how these two paragraphs are formulated, but I can't 
readily think of an improvement.

> +
> +[WARNING]
> +Be careful, *the _board's_ readme files coming within Buildroot are
> +provided without warranty of any kind*.
> +They are contributions from Buildroot users, but Buildroot developers
> +do not ensure their correctness, nor maintain them.

  Is this warning really necessary? Most readme.txt files use removable 
media (or emulation), and the few that do write to flash have warnings 
already. The only exception is the Telit EVK-PRO3, so maybe we should 
just add a warning there.

> +
> +
> +Booting images in emulators
> +---------------------------
> +
> +Buildroot comes with a set of configurations for various
> +architectures running in http://wiki.qemu.org/Main_Page['Qemu'], or
> +http://www.linaro.org/engineering/engineering-projects/armv8['Foundation_v8']
> +(an AArch64 emulator).
> +
> +These configurations are stored under the 'board/qemu/<target name>'
> +directory.

  The armv8 one is under board/arm/foundation-v8.

> +
> +Each of these configurations has a 'defconfig' and comes with a
> ++readme.txt+ file providing details to use the built images with
> +'Qemu'.
> +
> +They are regularly tested and maintained by the Buildroot core
> +developers.
> +
> +If you built one of these configurations and have 'Qemu' or
> +'Foundation_v8' installed on your host machine, booting the images
> +should be straight forward.
> +
> +
> +Preparing a raw disk image for customized target layout

  Preparing bootable removable media

> +-------------------------------------------------------
> +
> +This case is close to booting the actual hardware, so Buildroot
> +cannot provides any information how to achieve it for every single
> +target.

How the target device is close to the actual hardware, so Buildroot
cannot explain how to achieve it for every single target. For boards
that use the same processor as one of the boards supported by Buildroot,
the 'readme.txt' can be a good starting point.

> +
> +However, Buildroot provides _host-tools_ that may help generating
> +customized disk images, see the 'Host utilities' menu.

In addition, Buildroot provides tools that run on your host that can help
generating bootable media or booting a device over USB, serial or the
network.

> +
> +[NOTE]
> +Some of these tools are not really well documented, you may need to
> +browse the source trees to find out how to use them.
> +
> +
> +Preparing a bootable raw disk file for virtualization
> +-----------------------------------------------------
> +
> +[NOTE]
> +This section is mostly x86 target specific.
> +
> +If you plan to use virtual machines, or to copy a binary bootable
> +image to your target, you may need to create a _disk image_.
> +
> +To create a bootable raw _disk image_ file, you will need to:
> +
> +* create an empty file with the +dd+ command;
> +
> +* edit the partition table of this _disk image_ file using some tools
> +  like +fdisk+ (be carefull when using +fdisk+; mis-usage can damage

  careful

> +  the host machine. Look at its manpage before using it;
> +
> +* install the MBR;
> +
> +* create nodes in +/dev+ pointing to the _disk image_ and its
> +  partitions (as you will have with +/dev/sda+, +/dev/sda1+,
> +  +/dev/sda2+, etc) with
> +  http://robert.penz.name/73/kpartx-a-tool-for-mounting-partitions-within-an-image-file/[kpartx];
> +
> +* mount one or several partitions of the _disk image_ with the +mount+
> +  command;
> +
> +* populate the root partition by extracting into it the
> +  root-filesystem tarball generated by Buildroot.
> +
> +
> +Booting the generated image over the network
> +--------------------------------------------
> +
> +It is common to boot the target over the network when dealing with
> +embedded devices, whatever the reasons could be:
> +
> +* to speed-up the in-progress images test during the development
> +  phase;
> +* to avoid premature wear or the flash memory devices;
> +* to deploy the actual production images on the hardawre;

  hardware

> +* or because the target is not equiped with memory storage;

  equipped

> +* ...
> +
> +To achieve network-boot, there are several methods that depend on both
> +the facility network infrastructure and the targets. Among these, the
> +most common are:
> +
> +* TFTP boot
> +* NFS boot
> +* PXE boot (x86-specific)
> +
> +
> +TFTP boot
> +~~~~~~~~~
> +
> +Depending on the bootloader installed on the target, it may be
> +possible to download the kernel image through the newtork, using TFTP

  network

> +protocole, loading it in RAM, then jump into it.

  the TFTP protocol

> +
> +For further information, refer to the bootloader documentation.
> +
> +Here is some related documentation to TFTP server setup and TFTP boot:
> +
> +* http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch16_:_Telnet,_TFTP,_and_xinetd#TFTP
> +* https://linuxlink.timesys.com/docs/linux_tftp
> +* http://www.webune.com/forums/how-to-install-tftp-server-in-linux.html
> +
> +
> +NFS boot
> +~~~~~~~~
> +
> +Loading the kernel over NFS
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +Depending on the bootloader installed on the target, it may be
> +possible to download the kernel image through the newtork, using NFS

  network

> +protocole, loading it in RAM, then jump into it.

  the NFS protocol

> +
> +For further information, refer to the bootloader documentation.
> +
> +[[deploying-images-nfs-links]]
> +Here is some related documentation to NFS server setup and NFS boot:
> +
> +* http://tldp.org/HOWTO/NFS-HOWTO/index.html
> +* https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt
> +* https://wiki.archlinux.org/index.php/NFS
> +* http://www.solid-run.com/mw/index.php/Setup_NFS_boot
> +* http://wiki.openelec.tv/index.php?title=Network_Boot_-_NFS
> +* http://www.armadeus.com/wiki/index.php?title=NFS
> +
> +NFS root filesystem mounted on +/+
> +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> +
> +The idea is to mount +/+ using a network shared folder from an
> +http://tldp.org/HOWTO/NFS-HOWTO/index.html[NFS] server
> +(usually on the host development machine).
> +
> +To enable the NFS boot, you should select the _tar root filesystem_
> +option in the _Filesystem images_ submenu.

  menu

> +
> +The embedded kernel needs at least the following options:

  The target kernel

> +
> +* NFS filesystem support (+CONFIG_NFS_FS+);
> +
> +* Root file system on NFS (+CONFIG_ROOT_NFS+);

  filesystem

> +
> +* Ethernet (+CONFIG_NET_ETHERNET+);
> +
> +* The ethernet driver for the embedded network card;

  for the target's network interface;

> +
> +* IP: kernel level autoconfiguration. This includes:
> +
> +  * +CONFIG_IP_PNP+;
> +  * +CONFIG_IP_PNP_BOOTTP+;
> +  * +CONFIG_IP_PNP_DHCP+.

  AFAIK these are automatic when you select ROOT_NFS, no?

> +
> +After a complete build, just run the following command to setup the
> +NFS root directory on the server:
> +
> +-------------------
> +sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
> +-------------------
> +
> +Make sure the NFS root location appears in the +/etc/exports+ file,
> +and an _NFS_ daemon is running on the server.

  No need to put NFS in italics.

> +
> +After editing +/etc/exports+, you should run:
> +
> +-------------------
> +sudo exportfs -ra
> +-------------------
> +
> +To boot on a NFS root-filesystem, adjust the kernel command line

  root filesystem

> +parameters (see https://www.kernel.org/doc/Documentation/kernel-parameters.txt
> +and the xref:deploying-images-nfs-links[above links]).
> +
> +For further information, refer to the
> +xref:deploying-images-nfs-links[above links].
> +
> +
> +Network PXE bootloader
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +[NOTE]
> +This section is mostly x86 target specific.
> +
> +To fully boot on the network you need a network bootloader. This is
> +optional and you could use your classic bootloader to mount an NFS
> +rootfs.
> +
> +http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf[PXE]
> +is a specification that has been implemented at least by
> +http://www.syslinux.org/wiki/index.php/PXELINUX[PXELINUX] and
> +http://ipxe.org/[iPXE].
> +
> +The main idea is to have a DHCP server that provide a link to a generic

  provides


  Regards,
  Arnout


> +boot ROM that is accessible from a TFTP server.
> +Then your target boots with it and comes back to the TFTP server to get
> +the specific stuff (for instance its boot menu).
> +
> +Here are some hints on how to setup this:
> +
> +* http://www.digitalpeer.com/id/linuxnfs
> +
> +
> +Chroot into generated image
> +---------------------------
> +
> +If you want to 'chroot' in a generated image, then there are few things
> +you should be aware of:
> +
> +* you should setup the new root from the _tar root filesystem_ image;
> +
> +* either the selected target architecture is compatible with your host
> +  machine, or you should use some +qemu-*+ binary and correctly set it
> +  within the +binfmt+ properties to be able to run the binaries built
> +  for the target on your host machine;
> +
> +* Buildroot does not currently provide +host-qemu+ nor +binfmt+
> +  correctly built and set for that kind of use. This usage is beyond
> +  Buildroot scope.
> diff --git a/docs/manual/manual.txt b/docs/manual/manual.txt
> index 9685667..8fcb2c2 100644
> --- a/docs/manual/manual.txt
> +++ b/docs/manual/manual.txt
> @@ -21,6 +21,8 @@ include::starting-up.txt[]
>
>   include::working-with.txt[]
>
> +include::deploying-images.txt[]
> +
>   include::faq-troubleshooting.txt[]
>
>   include::going-further.txt[]
> @@ -29,8 +31,6 @@ include::developer-guide.txt[]
>
>   include::legal-notice.txt[]
>
> -include::beyond-buildroot.txt[]
> -
>   include::get-involved.txt[]
>
>   include::contribute.txt[]
> diff --git a/docs/manual/using.txt b/docs/manual/using.txt
> index de29ad6..783370d 100644
> --- a/docs/manual/using.txt
> +++ b/docs/manual/using.txt
> @@ -67,7 +67,8 @@ Buildroot output is stored in a single directory, +output/+.
>   This directory contains several subdirectories:
>
>   * +images/+ where all the images (kernel image, bootloader and root
> -  filesystem images) are stored.
> +  filesystem images) are stored. For further details for using/booting
> +  the images, refer to xref:deploying-images[].
>
>   * +build/+ where all the components except for the cross-compilation
>     toolchain are built (this includes tools needed to run Buildroot on
>


-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F