[Buildroot] [PATCH 1/4] fs/custom: generate complete, partition-based device images
Yann E. MORIN
yann.morin.1998 at free.fr
Fri Dec 20 22:32:54 UTC 2013
From: "Yann E. MORIN" <yann.morin.1998 at free.fr>
Contrary to the existing fs/ schemes, which each generate only a single
filesystem image for the root filesystem, this new scheme allows the
user to generate more complex images.
The basis behind this is a .ini-like description of the layout of the
final target storage:
- the list of device(s)
- per-device, the list of partition(s)
- per-partition, the content
It is possible to create MBR- or GPT-based partitoining schemes. Adding
new ones should be relatively easy (but would need adequate host tools).
For now, the only content possible for partitions is a filesystem. It
should be pretty easy to add new types (eg. un-formated, or raw blob).
Also, only two filesystems are supported: ext{2,3,4} and vfat. Adding
more will be relatively easy, provided we have the necessary host
packages to deal with those filesystems.
The existing Buildroot filesystem generators are re-used as much as
possible when it makes sense; when it does not (eg. for vfat), a specific
generator is used.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998 at free.fr>
Cc: Arnout Vandecappelle <arnout at mind.be>
Cc: Ryan Barnett <rjbarnet at rockwellcollins.com>
---
docs/manual/appendix.txt | 1 +
docs/manual/customize-filesystems.txt | 35 ++++
docs/manual/customize.txt | 2 +
docs/manual/partition-layout.txt | 314 ++++++++++++++++++++++++++++++++++
fs/Config.in | 1 +
fs/custom/Config.in | 16 ++
fs/custom/boot/gpt | 126 ++++++++++++++
fs/custom/boot/mbr | 57 ++++++
fs/custom/boot/pre-post | 8 +
fs/custom/custom.mk | 38 ++++
fs/custom/fs/ext | 22 +++
fs/custom/fs/pre-post | 67 ++++++++
fs/custom/fs/vfat | 17 ++
fs/custom/functions | 47 +++++
fs/custom/genimages | 242 ++++++++++++++++++++++++++
15 files changed, 993 insertions(+)
create mode 100644 docs/manual/customize-filesystems.txt
create mode 100644 docs/manual/partition-layout.txt
create mode 100644 fs/custom/Config.in
create mode 100644 fs/custom/boot/gpt
create mode 100644 fs/custom/boot/mbr
create mode 100644 fs/custom/boot/pre-post
create mode 100644 fs/custom/custom.mk
create mode 100644 fs/custom/fs/ext
create mode 100644 fs/custom/fs/pre-post
create mode 100644 fs/custom/fs/vfat
create mode 100644 fs/custom/functions
create mode 100755 fs/custom/genimages
diff --git a/docs/manual/appendix.txt b/docs/manual/appendix.txt
index 74ee8fd..53f4205 100644
--- a/docs/manual/appendix.txt
+++ b/docs/manual/appendix.txt
@@ -6,6 +6,7 @@ Appendix
include::makedev-syntax.txt[]
include::makeusers-syntax.txt[]
+include::partition-layout.txt[]
// Automatically generated lists:
diff --git a/docs/manual/customize-filesystems.txt b/docs/manual/customize-filesystems.txt
new file mode 100644
index 0000000..cddee42
--- /dev/null
+++ b/docs/manual/customize-filesystems.txt
@@ -0,0 +1,35 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[filesystem-custom]]
+Customizing the generated filesystem images
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
++Buildroot+ knows by default how to generate a few different kind of
+filesystems, such as +squashfs+, +ext2/3/4+, +cramfs+... But those
+filesystems are all generated to contain the complete target directory
+hierarchy in a single filesystem, mounted as the root filesystem +/+.
+That is, even if you select both an +ext2+ and a +squashfs+ filesystems,
+the content of the two generated images will be the exact same, only the
+types of the filesystems will be different.
+
+Most devices require a more complex setup, with different parts of the
+directory structure split across different filesystems, each stored on
+different partitions of one or more storage devices.
+
++Buildroot+ can generate such complex setups, using a +partition table layout
+description+. This is a simple text file, not unlike the +.ini+ style of
+configuration files, that describes how the target directory hierarchy has
+to be split across the target storage devices. It is a bit like a flattened
+tree of the storage layout.
+
+Set the variable +BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE+ to the path of
+the file containing your +partition table layout description+.
+
+See xref:part-layout-desc-syntax[] for the complete documentation of the
++partition table layout description+ syntax.
+
+[underline]*Note:* Although more versatile than the single filesystem image
+mechanism, the +partition table layout description+ might be unable to
+describe very complex setups. For example, it is not capable of handling
++initramfs+ based systems, or NFS-mounted filesystems.
diff --git a/docs/manual/customize.txt b/docs/manual/customize.txt
index 7e46fd8..6d062ea 100644
--- a/docs/manual/customize.txt
+++ b/docs/manual/customize.txt
@@ -14,6 +14,8 @@ include::customize-kernel-config.txt[]
include::customize-toolchain.txt[]
+include::customize-filesystems.txt[]
+
include::customize-store.txt[]
include::customize-packages.txt[]
diff --git a/docs/manual/partition-layout.txt b/docs/manual/partition-layout.txt
new file mode 100644
index 0000000..553f862
--- /dev/null
+++ b/docs/manual/partition-layout.txt
@@ -0,0 +1,314 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+[[part-layout-desc-syntax]]
+
+Partition table layout description syntax
+-----------------------------------------
+
+The +partition table layout description+ syntax is not unlike the standard
+https://en.wikipedia.org/wiki/.ini[+.ini+] syntax. There are two types of
+entries: +sections+, that may each contain zero or more +properties+.
+
++Sections+ are specified between square brackets +[]+, _eg._: +[name]+.
+
++Properties+ are specified as key-value pairs, _eg._: +key=value+, and
+are documented as:
+
+* +key-name+ (optional or mandatory): description
+** +value1+: description
+** +value2+: description
+** ...
+
+[underline]*Note:* Unlike the standard +.ini+ syntax, the +partition table
+layout description+ _is_ case-sensitive.
+
+The order of +sections+ is irrelevant. However, for readability, we recomend
+the +partition table layout description+ starts with the +global+ section.
+
+The global section
+~~~~~~~~~~~~~~~~~~
+
+The +[global]+ section defines some global settings, and the list of devices.
+
+Properties for the global section
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +extract+ (mandatory): the type of base image to extract
+** +tar+: extract the +rootfs.tar+ base image generated by +Buildroot+
+
+* +devices+ (mandatory): the comma-separated list of storage devices to
+ use on the device. Each device is the filename of the device node
+ present in +/dev+
+
+* +keep_parts+ (optional): also copy the individual partition images
+ of all devices to +$(BINARIES_DIR)+. Settings from the device sections
+ or the partition sections take precedence over this one.
+** +yes+: copy the individual partition images
+** +no+ (the default): do not copy individual partition images
+
+The devices and partitions sections
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The devices and partitions sections define, for each device or partition,
+the content of that device or partition.
+
+For each device listed in the +[global]+ section, there must be a
+corresponding section named after that device.
+
+For each partition listed in a device section, there must be a corresponding
+section named after that partition.
+
+Properties for the device section
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +type+ (mandatory): the type of content for that device or partition
+** +boot+: the device contains one or more partitions, and
+ _may_ serve as a boot device
+
+* +keep_parts+ (optional): also copy the individual partition images
+ for this device to +$(BINARIES_DIR)+. Settings from the partition
+ sections take precedence over this one.
+** +yes+: copy the individual partition images
+** +no+ (the default): do not copy individual partition images
+
+Properties for the partition section
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +type+ (mandatory): the type of content for that device or partition
+** +fs+: the partition contains a filesystem
+
+* +size+: the size of that partition, in bytes
+
+* +keep+ (optional): copy this partition image to +$(BINARIES_DIR)+
+** +yes+: copy this partition image
+** +no+ (the default): do not copy this partition image
+
+Properties for +type=boot+
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +boot_type+ (mandatory): the partitioning scheme to use on this device
+** +mbr+: use an https://en.wikipedia.org/wiki/Master_boot_record[MBR]
+ partitioning scheme
+** +gpt+: use a https://en.wikipedia.org/wiki/GUID_Partition_Table[GPT]
+ partitioning scheme
+
+* +partitions+ (mandatory): the comma-separated list of partition(s) on
+ this device; no two partitions may have the same name, even if they
+ reside on different devices; partitions names shall match this regexp:
+ `^[[:alpha:]][[:alnum:]-_]*$` (_ie._ starts with a letter, followed by
+ zero or more alpha-numeric character or a dash or an underscore)
+
+* +partalign+ (optional): the alignment of partitions, in bytes; defaults
+ to an alignment of one, which means no alignment; depending on the
+ +boot_type+, some restrictions may apply, and are documented for each
+ +boot_type+
+
+Properties for +boot_type=mbr+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +mbr_bootcode+ (optional): the bootcode to use, as a path to the file
+ containing the bootcode image, relative to the +$(BINARIES_DIR)+
+ directory; defaults to no bootcode (eg. filled with zeroes)
+
+* +partalign+: must be a multiple of 512
+
+Properties for +boot_type=gpt+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +partalign+: must be a multiple of 512
+
+********
+Currently, only 512-byte sectors are supported. 4k sectors are not.
+********
+
+Properties for partitions whose containing device is +boot_type=mbr+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +mbr_type+ (mandatory): the partition
+ https://en.wikipedia.org/wiki/Partition_type#List_of_partition_IDs[type]
+
+Properties for +type=fs+
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +fs_type+ (mandatory): the type of filesystem to generate
+** +ext+: generate an extended filesystem (ext2, ext3, ext4)
+** +vfat+: generate a VFAT filesystem (FAT16, FAT32)
+
+* +fs_label+ (optional): the label to assign to this filesystem, if that
+ filesystem supports a label
+
+* +fs_files_0+, +fs_files_1+, +fs_files_N+ (optional): the list of files,
+ relative to $(BINARIES_DIR), to store in the filesystem. These entries
+ must be indexed starting from 0, and must be sequential: the first
+ missing entry ends the list
+
+* +fs_root+ (optional): the mountpoint of the filesystem
+
+* +fs_vfstype+ (optional): the type of filesystem to use when calling
+ +mount+, if different from +fs_type+
+
+* +fs_mntopts+ (optional): the mount options; defaults to +defaults+
+
+_Note_: valid use-cases for +fs_root+ and +fs_files_N+:
+
+* if only +fs_root+ is specified (and no +fs_files_N+): the filesystem
+ content is made exclusively from +$(TARGET_DIR)/$(fs_root)+, and the
+ filesystem is mounted at runtime
+
+* if both +fs_root+ and at least one +fs_files_N+ are specified: the
+ filesystem content is made exclusively from the +fs_files_N+ entries,
+ and mounted at runtime. +$(TARGET_DIR)/$(fs_root)+ must be empty
+
+* if at least one +fs_files_N+ is specified, and +fs_root+ is not: the
+ filesystem content is made exclusively from the +fs_files_N+ entries,
+ and the filesystem is not mounted at runtime
+
+* if neither +fs_root+ nor +fs_files_N+ is specified: this is an error
+
+Properties for +fs_type=ext+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +ext_gen+ (mandatory): the generation of extended filesystem to generate
+** +2+, +3+, +4+: for an ext2, ext3 or ext4 filesystem
+
+* +ext_rev+ (mandatory): the revision of the extended filesystem
+** +0+ (ext2 only): generate a revision 0 extended filesystem filesystem
+** +1+ (mandatory for ext3 or ext4): generate a revision 1 extended
+ filesystem
+
+Properties for +fs_type=vfat+
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* +vfat_size+ (optional): the VFAT-size of the filesystem
+** +12+, +16+, +32+: generate a FAT12, FAT16, or FAT32
+
+Generation of the filesystems
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The filesystems are generated in an order such that a filesystem that is
+mounted as a sub-directory of another filesystem is generated first.
+
+A filesystem is filled with the content of the directory corresponding to
+its mountpoint, and then that directory is emptied before continuing to the
+next filesystem. That way, the mountpoints are empty before the filesystem
+that contains them are generated.
+
+Finally, an entry in added in +/etc/fstab+ for each generated filesystem, so
+they are mounted at boot time.
+
+Requirements on host packages
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Depending on what types of partitioning and filesystems are used by your
++partition table layout description+, you may have to enable some
+host-packages in the +Host utilities+ sub-menu.
+
+Since the content of a +partition table layout description+ is very
+specific to a board and/or a project, there is no way for Buildroot to
+automatically select those host packages, and thus it is your
+responsibility to select the appropriate ones.
+
+For example, for an MBR partitioning, you will have to enable the +host
+genpart+ package. For FAT filesystems, you will have to enable both of
++host dosfstools+ and +host mtools+.
+
+Examples
+~~~~~~~~
+
+.Simplest partition table layout description
+====
+----
+[global]
+extract=tar
+devices=sda
+
+[sda]
+type=boot
+boot_type=mbr
+partitions=root
+partalign=1048576
+
+[root]
+type=fs
+fs_type=ext
+fs_vfstype=ext4
+fs_root=/
+ext_gen=4
+ext_rev=1
+----
+
+The +partition table layout description+ above defines a single device
++sda+. That device contains a single partition, +root+, with an ext4
+filesystem, which is filled with the whole content of the +rootfs.tar+,
+and is mounted on +/+.
+====
+
+.More copmplex table layout description
+====
+----
+[global]
+extract=tar
+devices=mmcblk0,sda
+
+[mmcblk0]
+type=boot
+boot_type=mbr
+partitions=boot,root
+partalign=$((1024*1024))
+
+[sda]
+type=boot
+boot_type=mbr
+partitions=data
+partalign=4096
+
+[boot]
+type=fs
+mbr_type=$((0xC))
+size=$((16*1048576))
+fs_type=vfat
+fs_mntopts=ro
+fs_label=BOOT
+fs_root=/boot
+vfat_size=32
+
+[root]
+type=fs
+mbr_type=$((0x83))
+size=268435456
+fs_type=ext
+fs_vfstype=ext4
+fs_mntopts=discard,delalloc
+fs_root=/
+fs_label=ROOT
+ext_gen=4
+ext_rev=1
+
+[data]
+type=fs
+mbr_type=$((0x83))
+size=$((4*1024*1048576))
+fs_type=ext
+fs_vfstype=ext2
+fs_root=/data
+fs_label=DATA
+ext_gen=2
+ext_rev=1
+----
+====
+
+The example above defines two devices, +mmcblk0+ and +sda+.
+
+The +mmcblk0+ device contains two partitions, +boot+ and +root+; partitions
+are aligned on a 1MiB boundary. The +sda+ device contains a single partition,
++data+, aligned on a 4KiB boundary.
+
+The +boot+ partition is a 16MiB FAT32 filesystem filled with the content
+of, and mounted on, +/boot+, and with label +BOOT+.
+
+The +data+ partition is a 4GiB ext2r1 filesystem filled with the content
+of, and mounted on, +/data+, and with label +DATA+.
+
+The +root+ partition is a 256MiB ext4 filesystem filled the the rest of,
+and mounted on, +/+, and with label +ROOT+.
diff --git a/fs/Config.in b/fs/Config.in
index da4c5ff..44e04f7 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -3,6 +3,7 @@ menu "Filesystem images"
source "fs/cloop/Config.in"
source "fs/cpio/Config.in"
source "fs/cramfs/Config.in"
+source "fs/custom/Config.in"
source "fs/ext2/Config.in"
source "fs/initramfs/Config.in"
source "fs/iso9660/Config.in"
diff --git a/fs/custom/Config.in b/fs/custom/Config.in
new file mode 100644
index 0000000..e5a8ee7
--- /dev/null
+++ b/fs/custom/Config.in
@@ -0,0 +1,16 @@
+config BR2_TARGET_ROOTFS_CUSTOM
+ bool "Custom partition table layout"
+ select BR2_TARGET_ROOTFS_TAR
+
+config BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE
+ string "path to the custom partition table layout description"
+ depends on BR2_TARGET_ROOTFS_CUSTOM
+ help
+ Enter the path to a partition-table for your device.
+
+ This will allow Buildroot to generate a more complex target
+ image, which may consist of more than one filesystem on more
+ than one partition.
+
+ See docs/manual/bla-bla on how to construct such a partition
+ table.
diff --git a/fs/custom/boot/gpt b/fs/custom/boot/gpt
new file mode 100644
index 0000000..8592cbf
--- /dev/null
+++ b/fs/custom/boot/gpt
@@ -0,0 +1,126 @@
+# Build a complete GPT-based image
+
+# For a GPT-based partitionning, we need to compute the complete
+# image size before we can attempt to generate the partition table.
+# Then, we need to add the size of the GPT itself, plus that of its
+# backup copy, plus the protective MBR.
+# The size of the GPT itself depends on the sector size, and the
+# number of partitions in the GPT. Sectors can be either 512-byte
+# or 4096-byte large; The numbers of partitions is unlimited, but
+# it is suggested there is space to store at least 128 of them; a
+# partition description is 128-byte large.
+#
+# https://en.wikipedia.org/wiki/GUID_Partition_Table
+#
+# So, here's what we do:
+# - consider 512-byte sectors (since GPT on 4k sectors is not well
+# documented)
+# - consider at least 128 partitions; if the layout defines more than
+# that, we need to round that number up to the smallest multiple of
+# 4 (since there are 4 partition descriptions in a 512-byte sector)
+# - generate an empty, sparse file that is big enough to store the MBR,
+# the two GPT copies, and the aligned partitions.
+# - dump each partition in turn in their final location in that file
+# - generate a parted script that creates the partition table in that
+# file
+#
+# Simg = 512 + 2*(Sgpt) + Σ( aligned(Spart,512) )
+# Sgpt = 512 + Nent*128
+#
+# Where:
+# Simg : size of the image
+# Sgpt : size of one GPT
+# Spart : size of each partition
+# Nent : number of partition entries
+# aligned() : the alignment function
+#
+# Sicne 4k-large sectors are not really explained on Wikipedia, we can
+# add this later on.
+
+do_image() {
+ # ${1} is fs_root, irrelevant here
+ local img="${2}"
+
+ # How many partitions do we have?
+ nb_parts=0
+ for part in ${partitions[${dev}]//,/ }; do
+ nb_parts=$((nb_parts+1))
+ done
+
+ # How many partition entries do we need?
+ nb_entries=$((4*((nb_parts+3)/4)))
+ nb_entries=$((nb_entries<128?128:nb_entries))
+
+ # The size of a single GPT
+ gpt_size=$((512+(128*nb_entries)))
+
+ # Offset of the first partition
+ begin=$(align_val $((512+$(align_val ${gpt_size} 512))) ${partalign} )
+
+ # Initialise our image file
+ dd if=/dev/zero of="${img}" \
+ bs=1 seek=${begin} count=0 \
+ conv=sparse 2>/dev/null
+
+ # Compute the space required to store all partitions
+ # and store them in the image file
+ size_parts=0
+ _offset=${begin}
+ i=1
+ debug "adding partions descriptions\n"
+ for part in ${partitions[${dev}]//,/ }; do
+ debug " %s\n" "${part}"
+ _part_img="${tmp_dir}/${dev}.${part}.img"
+ _size=$( align_val $( stat -c '%s' "${_part_img}" ) 512 )
+ part_offset+=( ${_offset} )
+ _attr="${values["${part}:gpt_attr"]}"
+ _label="${values["${part}:gpt_label"]}"
+
+ # If the partition has no label, use the filesystem label
+ if [ -z "${_label}" ]; then
+ _label="${values["${part}:fs_label"]}"
+ fi
+ if [ -z "${_label}" ]; then
+ _label="data"
+ fi
+
+ debug " start=%s\n" "${_offset}"
+ debug " size =%s\n" "${_size}"
+ debug " end =%s\n" "$((_offset+_size-1))"
+
+ dd if="${_part_img}" of="${img}" \
+ bs=512 seek=$((_offset/512)) \
+ conv=notrunc,sparse 2>/dev/null
+
+ parted_script+=( mkpart "${_label}" \
+ ${_offset} \
+ $((_offset+_size-1)) \
+ )
+ if [ -n "${_attr}" ]; then
+ for attr in "${_attr//,/ }"; do
+ parted_script+=( set ${i} ${attr} on )
+ done
+ fi
+
+ size_parts=$((size_parts+_size))
+ _offset=$((_offset+_size))
+ i=$((i+1))
+ done
+
+ # Terminate our image file
+ img_size=$(align_val $(( begin + size_parts + gpt_size )) 512)
+ debug "begin =%s\n" ${begin}
+ debug "nb_entry=%s\n" ${nb_entries}
+ debug "gpt_size=%s\n" ${gpt_size}
+ debug "img_size=%s\n" ${img_size}
+ dd if=/dev/zero of="${img}" \
+ bs=1 seek=${img_size} \
+ count=0 conv=sparse 2>/dev/null
+
+ for i in parted -s "${img}" mklabel gpt unit B "${parted_script[@]}"; do
+ debug "--> '%s'\n" "${i}"
+ done
+ parted -s "${img}" mklabel gpt unit B "${parted_script[@]}"
+}
+
+# vim: ft=sh
diff --git a/fs/custom/boot/mbr b/fs/custom/boot/mbr
new file mode 100644
index 0000000..af7c6cb
--- /dev/null
+++ b/fs/custom/boot/mbr
@@ -0,0 +1,57 @@
+# Build a complete MBR-based image
+
+do_image() {
+ # ${1} is fs_root, irrelevant here
+ local img="${2}"
+ local i begin part part_img size type _begin _size
+ local -a part_offset part_file
+
+ # Fill-in the boot record with zeroes
+ # Ideally, we should dump the bootloader code here, but since
+ # we don't have any so far, that will have to be done in a
+ # later step.
+ debug "adding (fake) bootcode\n"
+ dd if=/dev/zero of="${img}" bs=$((0x1be)) count=0 seek=1 2>/dev/null
+
+ # Generate partition entries
+ i=0
+ begin=${partalign}
+ debug "adding partitions descriptors\n"
+ for part in ${partitions[${dev}]//,/ }; do
+ debug " %s\n" "${part}"
+ part_offset+=( ${begin} )
+ part_img="${tmp_dir}/${dev}.${part}.img"
+ part_file+=( "${part_img}" )
+ size=$( align_val $( stat -c '%s' "${part_img}" ) 512 )
+ type="${values["${part}:mbr_type"]}"
+ # LBA is exressed in a number of 512-byte blocks
+ # and genparts only deals with LBA
+ _begin=$((begin/512)) # begin is already 512-byte aligned
+ _size=$((size/512)) # size is already 512-byte aligned
+ debug " start=%s (LBA %s)\n" "${begin}" "${_begin}"
+ debug " size =%s (LBA %s)\n" "${size}" "${_size}"
+ debug " type =%s\n" "${type}"
+ genpart -b ${_begin} -s ${_size} -t ${type} >>"${img}"
+ begin=$( align_val $((begin+size)) ${partalign} )
+ i=$((i+1))
+ done
+ nb_parts=${i}
+ # Generate entries for empty partitions
+ for(( ; i<4; i++ )); do
+ debug " (empty)\n"
+ genpart -t 0 >>"${img}"
+ done
+ # Dump the boot signature
+ printf "\x55\xaa" >>"${img}"
+
+ for(( i=0; i<nb_parts; i++ )); do
+ part_img="${part_file[${i}]}"
+ offset=${part_offset[${i}]}
+ _offset=$(( offset/512 )) # offset is already 512-byte aligned
+ dd if="${part_img}" of="${img}" \
+ bs=512 seek=${_offset} \
+ conv=notrunc,sparse 2>/dev/null
+ done
+}
+
+# vim: ft=sh
diff --git a/fs/custom/boot/pre-post b/fs/custom/boot/pre-post
new file mode 100644
index 0000000..507d17e
--- /dev/null
+++ b/fs/custom/boot/pre-post
@@ -0,0 +1,8 @@
+do_image_pre() {
+ :
+}
+do_image_post() {
+ :
+}
+
+#vim: set ft=sh
diff --git a/fs/custom/custom.mk b/fs/custom/custom.mk
new file mode 100644
index 0000000..63c1f23
--- /dev/null
+++ b/fs/custom/custom.mk
@@ -0,0 +1,38 @@
+################################################################################
+#
+# custom partitioning
+#
+################################################################################
+
+define ROOTFS_CUSTOM_CMD
+ BUILD_DIR=$(BUILD_DIR) fs/custom/genimages \
+ '$(call qstrip,$(BR2_TARGET_ROOTFS_CUSTOM_PARTITION_TABLE))'
+endef
+
+# rootfs-custom uses rootfs.tar as the source to generate
+# the resulting image(s), so we need to build it first.
+ROOTFS_CUSTOM_DEPENDENCIES += rootfs-tar
+
+# All of the following filesystem generators, or partition managers, are
+# optional, but if they are selected, we may need them, so we need to
+# depend on them
+ifeq ($(BR2_PACKAGE_HOST_DOSFSTOOLS),y)
+ROOTFS_CUSTOM_DEPENDENCIES += host-dosfstools
+endif
+ifeq ($(BR2_PACKAGE_HOST_E2FSPROGS),y)
+ROOTFS_CUSTOM_DEPENDENCIES += host-e2fsprogs
+endif
+ifeq ($(BR2_PACKAGE_HOST_GENEXT2FS),y)
+ROOTFS_CUSTOM_DEPENDENCIES += host-genext2fs
+endif
+ifeq ($(BR2_PACKAGE_HOST_GENPART),y)
+ROOTFS_CUSTOM_DEPENDENCIES += host-genpart
+endif
+ifeq ($(BR2_PACKAGE_HOST_MTOOLS),y)
+ROOTFS_CUSTOM_DEPENDENCIES += host-mtools
+endif
+ifeq ($(BR2_PACKAGE_HOST_PARTED),y)
+ROOTFS_CUSTOM_DEPENDENCIES += host-parted
+endif
+
+$(eval $(call ROOTFS_TARGET,custom))
diff --git a/fs/custom/fs/ext b/fs/custom/fs/ext
new file mode 100644
index 0000000..5f9b4e7
--- /dev/null
+++ b/fs/custom/fs/ext
@@ -0,0 +1,22 @@
+# Create an extended file system
+
+do_image() {
+ local root_dir="${1}"
+ local img="${2}"
+ local -a fs_opts
+ local gen rev
+
+ fs_opts+=( -z )
+ fs_opts+=( -d "${root_dir}" )
+ [ -z "${size}" ] || fs_opts+=( -b $((size/1024)) )
+ [ -n "${ext_gen}" ] || ext_gen=2
+ [ -n "${ext_rev}" ] || ext_rev=1
+
+ # Remember, we're running from Buildroot's TOP_DIR
+ GEN=${ext_gen} REV=${ext_rev} \
+ ./fs/ext2/genext2fs.sh "${fs_opts[@]}" "${img}" >/dev/null
+
+ [ -z "${fs_label}" ] || tune2fs -L "${fs_label}" "${img}" >/dev/null
+}
+
+# vim: ft=sh
diff --git a/fs/custom/fs/pre-post b/fs/custom/fs/pre-post
new file mode 100644
index 0000000..40ec047
--- /dev/null
+++ b/fs/custom/fs/pre-post
@@ -0,0 +1,67 @@
+#-----------------------------------------------------------------------------
+do_image_pre() {
+ local i file
+
+ # if fs_root_dir is not specified, we have to create one
+ # It *does* override the caller's fs_root_dir value, but
+ # that's on purpose
+ # If fs_root_dir is specified, and we have at least fs_files_0,
+ # then fs_root_dir/ must be enpty
+ if [ -z "${fs_root_dir}" ]; then
+ if [ -n "${values["${part}:fs_files_0"]}" ]; then
+ error "%s: no fs_root specified, and no fs_files_0\n" "${part}"
+ fi
+ fs_root_dir="$( mktemp -d "${tmp_dir}/XXXXXX" )"
+ else
+ if [ -n "${values["${part}:fs_files_0"]}" \
+ -a $( ls -1A "${fs_root_dir}" |wc -l ) -ne 0 ]; then
+ error "%s: %s is not empty, but fs_files_0 is specified\n" \
+ "${part}" "${fs_root_dir#${fs_root}}"
+ fi
+ fi
+
+ i=0
+ while true; do
+ file="${values["${part}:fs_files_${i}"]}"
+ [ -n "${file}" ] || break
+ debug "%s: adding fs_files_%d %s\n" "${part}" ${i} "${file}"
+ install -D "${BINARIES_DIR}/${file}" "${fs_root_dir}/${file##*/}"
+ i=$((i+1))
+ done
+}
+
+#-----------------------------------------------------------------------------
+do_image_post() {
+ local rootfs_dir="${1}"
+ local fs_root="${2}"
+ local img_file="${3}"
+ local part="${4}"
+ local dev mntops vfstype fs_root_esc
+
+ subname+="[post-image]"
+
+ # Empty the partition's mountpoint
+ find "${fs_root_dir}" -maxdepth 1 \! -path "${fs_root_dir}" -exec rm -rf {} +
+
+ # Add entry in fstab, but not if this is '/'
+ # Don't add either if rootfs was not extracted
+ if [ "${fs_root}" = "/" -o -z "${fs_root}" \
+ -o -z "${values["global:extract"]}" ]; then
+ return 0
+ fi
+ fs_root_esc="$( sed -r -e 's:/:\\/:g;' <<<"${fs_root}" )"
+ sed -r -i -e "/[^[:space:]]+[[:space:]]+${fs_root_esc}[[:space:]]/d" \
+ "${rootfs_dir}/etc/fstab"
+ dev="$( get_part_dev_node "${part}" )"
+ vfstype="${fs_vfstype:-${fs_type}}"
+ mntops="${fs_mntops:-defaults}"
+ printf "/dev/%s %s %s %s 0 0\n" \
+ "${dev}" "${fs_root}" \
+ "${vfstype}" "${mntops}" \
+ >>"${rootfs_dir}/etc/fstab"
+
+ subname="${subname%\[post-image\]}"
+}
+
+#-----------------------------------------------------------------------------
+# vim: ft=sh
diff --git a/fs/custom/fs/vfat b/fs/custom/fs/vfat
new file mode 100644
index 0000000..aa5545f
--- /dev/null
+++ b/fs/custom/fs/vfat
@@ -0,0 +1,17 @@
+# Create a VFAT file system
+
+do_image() {
+ local root_dir="${1}"
+ local img="${2}"
+ local -a fs_opts
+
+ dd if=/dev/zero of="${img}" bs=${size} count=0 seek=1 2>/dev/null
+
+ [ -z "${vfat_size}" ] || fs_opts+=( -F ${vfat_size} )
+ [ -z "${fs_label}" ] || fs_opts+=( -n "${fs_label}" )
+ mkfs.vfat "${fs_opts[@]}" "${img}" >/dev/null
+
+ mcopy -i "${img}" "${root_dir}/"* '::'
+}
+
+# vim: ft=sh
diff --git a/fs/custom/functions b/fs/custom/functions
new file mode 100644
index 0000000..4b41021
--- /dev/null
+++ b/fs/custom/functions
@@ -0,0 +1,47 @@
+# Common functions
+
+#------------------------------------------------------------------------------
+align_val() {
+ local val="${1}"
+ local align="${2}"
+ local aligned
+
+ aligned=$(( ( (val+align-1) / align ) * align ))
+
+ printf "%d" ${aligned}
+}
+
+#------------------------------------------------------------------------------
+# Some trace functions
+trace() {
+ local fmt="${1}"
+ shift
+
+ printf "%s" "${myname}"
+ if [ -n "${subname}" ]; then
+ printf "(%s)" "${subname}"
+ fi
+ printf ": ${fmt}" "${@}"
+}
+
+debug() { :; }
+if [ -n "${DEBUG}" ]; then
+ debug() {
+ trace "${@}" >&2
+ }
+fi
+
+error() {
+ trace "${@}" >&2
+ exit 1
+}
+
+on_error() {
+ local ret=${?}
+
+ error "unexpected error caught: %d\n" ${ret}
+}
+trap on_error ERR
+set -E -e
+
+# vim: ft=sh
diff --git a/fs/custom/genimages b/fs/custom/genimages
new file mode 100755
index 0000000..00820fc
--- /dev/null
+++ b/fs/custom/genimages
@@ -0,0 +1,242 @@
+#!/bin/bash
+set -e
+set -E
+
+#-----------------------------------------------------------------------------
+main() {
+ local part_table="${1}"
+ local tmp_dir
+ local rootfs_dir
+ local -a devices
+ local extract
+ local cur_section
+ local -a sections devices partitions
+ local -A variables values partdevs
+ local sec dev part var val
+ local secs devs parts vars vals
+ local has_global_section
+
+ # We need bash 4 or above for asociative arrays
+ if [ ${BASH_VERSINFO[0]} -lt 4 ]; then
+ error "bash 4 or above is needed\n"
+ fi
+
+ if [ ! -f "${part_table}" ]; then
+ error "%s: no such file\n" "${part_table}"
+ exit 1
+ fi
+
+ export PATH="${HOST_DIR}/usr/bin:${HOST_DIR}/usr/sbin:${PATH}"
+
+ # Parse all the sections in one go, we'll sort
+ # all the mess afterwards...
+ debug "parsing partitions descriptions file '%s'\n" \
+ "${part_table}"
+ has_global_section=0
+ while read line; do
+ line="$( sed -r -e 's/[[:space:]]*#.*$//;' <<<"${line}" )"
+
+ # Detect start of global section, skip anything else
+ case "${line}" in
+ "") continue ;;
+ '[global]') has_global_section=1 ;;&
+ '['*']')
+ cur_section="$( sed -r -e 's/[][]//g;' <<<"${line}" )"
+ debug " entering section '%s'\n" "${cur_section}"
+ sections+=( "${cur_section}" )
+ continue
+ ;;
+ ?*=*) ;;
+ *) error "malformed entry '%s'\n" "${line}" ;;
+ esac
+
+ var="${line%%=*}"
+ eval val="${line#*=}"
+ debug " adding '%s'='%s'\n" "${var}" "${val}"
+ variables+=( ["${cur_section}"]=",${var}" )
+ values+=( ["${cur_section}:${var}"]="${val}" )
+ done <"${part_table}"
+
+ if [ ${has_global_section} -eq 0 ]; then
+ error "no global section defined\n"
+ fi
+
+ # Create lists of devices, partitions, and partition:device pairs.
+ debug "creating intermediate lists\n"
+ devices=( ${values["global:devices"]//,/ } )
+ for dev in "${devices[@]}"; do
+ # Sanity check first: all devices must have a corresponding
+ # section, which means they should have a type
+ if [ -z "${values["${dev}:type"]}" ]; then
+ error "device '%s' has no type (no section?)\n" "${dev}"
+ fi
+ partitions+=( ${values["${dev}:partitions"]//,/ } )
+ for part in ${values["${dev}:partitions"]//,/ }; do
+ # Sanity check first: all partitions must have a corresponding
+ # section, which means they should have a type
+ if [ -z "${values["${part}:type"]}" ]; then
+ error "partition '%s' has no type (no section?)\n" "${dev}"
+ fi
+ partdevs+=( ["${part}"]="${dev}" )
+ done
+ done
+
+ # Now, we must order the partitions so that their mountpoint
+ # is empty by the time we build the upper-level partition.
+ # For example, given this layout of mountpoints:
+ # /
+ # /usr
+ # /usr/var
+ # We must ensure /usr/var is empty at the time we create the /usr
+ # filesystem image; and similarly, we must ensure /usr is empty by
+ # the time we create the / filesystem image
+ # So, a simple reverse alphabetical sort will do the trick
+ debug "sorting partitions\n"
+ sorted_parts=( $(
+ for part in "${partitions[@]}"; do
+ # Partitions that are not mounted can be generated
+ # in any order
+ if [ -n "${values["${part}:fs_root"]}" ]; then
+ printf "%s:%s\n" "${part}" "${values["${part}:fs_root"]}"
+ else
+ printf "%s\n" "${part}"
+ fi
+ done \
+ |sort -t: -k2 -r \
+ |sed -r -e 's/:[^:]+$//;'
+ ) )
+
+ tmp_dir="${BUILD_DIR}/genimages.tmp"
+ rootfs_dir="${tmp_dir}/rootfs"
+ # Since we don't remove it in case of error (to be able to inspect its
+ # content), we must remove it now (a previous run may have left it).
+ rm -rf "${tmp_dir}"
+ mkdir -p "${rootfs_dir}"
+
+ case "${values["global:extract"]}" in
+ tar)
+ trace "extracting rootfs.tar\n"
+ tar xf "${BINARIES_DIR}/rootfs.tar" -C "${rootfs_dir}"
+ ;;
+ *) error "unknown extract method '%s'\n" "${extract:-(none)}"
+ ;;
+ esac
+
+ # Render all partition images
+ for part in "${sorted_parts[@]}"; do
+ trace "preparing filesystem for partition '%s'\n" "${part}"
+ render_img "${rootfs_dir}" "${part}" \
+ "${tmp_dir}/${partdevs["${part}"]}.${part}.img"
+ done
+
+ # Aggregate all devices images
+ for dev in "${devices[@]}"; do
+ trace "assembling partitions in device '%s'\n" "${dev}"
+ render_img "${rootfs_dir}" "${dev}" "${tmp_dir}/${dev}.img"
+ done
+
+ # Copy all partitions and devices images to the image dir
+ if [ "${values["global:keep_partitions"]}" = "yes" ]; then
+ for part in "${sorted_parts[@]}"; do
+ debug "copying partition '%s' to image dir\n" "${part}"
+ dd if="${tmp_dir}/${partdevs["${part}"]}.${part}.img" \
+ of="${BINARIES_DIR}/$( get_part_dev_node "${part}" ).img" \
+ bs=4096 conv=sparse 2>/dev/null
+ done
+ fi
+ for dev in "${devices[@]}"; do
+ debug "copying device '%s' to image dir\n" "${dev}"
+ dd if="${tmp_dir}/${dev}.img" \
+ of="${BINARIES_DIR}/${dev}.img" \
+ bs=4096 conv=sparse 2>/dev/null
+ done
+
+ [ -n "${DEBUG}" ] || rm -rf "${tmp_dir}"
+}
+
+#-----------------------------------------------------------------------------
+render_img() {
+ local rootfs_dir="${1}"
+ local img="${2}"
+ local img_file="${3}"
+ local type sub_type fs_root_dir
+
+ type="${values["${img}:type"]}"
+ sub_type="${values["${img}:${type}_type"]}"
+
+ # Sanity checks
+ [ -n "${type}" ] || error "'%s': unspecified type\n" "${img}"
+ if [ ! -d "fs/custom/${type}" ]; then
+ error "'%s': unsupported type '%s'\n" "${img}" "${type}"
+ fi
+ [ -n "${sub_type}" ] || error "'%s': unspecified %s_type\n" "${img}" "${type}"
+ if [ ! -f "fs/custom/${type}/${sub_type}" ]; then
+ error "'%s': unknown %s_type '%s'\n" "${img}" "${type}" "${sub_type}"
+ fi
+
+ # Need to call the renderer in a subshell so that its definitions
+ # do not pollute our environment
+ subname="${sub_type}"
+ (
+ trap 'exit $?' ERR
+ for var in ${variables["${img}"]//,/ }; do
+ eval "${var}=\"${values["${img}:${var}"]}\""
+ done
+ fs_root_dir="${rootfs_dir}${fs_root}"
+ . "fs/custom/${type}/pre-post"
+ . "fs/custom/${type}/${sub_type}"
+ do_image_pre "${rootfs_dir}" "${fs_root}" "${img_file}" "${img}"
+ do_image "${fs_root_dir}" "${img_file}"
+ do_image_post "${rootfs_dir}" "${fs_root}" "${img_file}" "${img}"
+ )
+ ret=${?}
+ [ ${ret} -eq 0 ] || exit ${ret}
+ subname=""
+}
+
+#-----------------------------------------------------------------------------
+get_part_dev_node() {
+ local part="${1}"
+ local dev
+ local i c p
+
+ dev="${partdevs["${part}"]}"
+ i="${values["${dev}:partstart"]:-1}"
+
+ # If device node ends with a number, partitions are denoted
+ # with a 'p' before the partition number, eg.:
+ # /dev/mmcblk0 --> /dev/mmcblk0p1
+ # /dev/sda --> /dev/sda1
+ case "${dev#${dev%?}}" in
+ [0-9]) c="p";;
+ *) c="";;
+ esac
+
+ for p in ${values["${dev}:partitions"]//,/ }; do
+ if [ "${p}" = "${part}" ]; then
+ printf "%s%s%d" "${dev}" "${c}" ${i}
+ return 0
+ fi
+ i=$((i+1))
+ done
+
+ error "'%s': partition not found. WTF?\n" "${part}"
+}
+
+#-----------------------------------------------------------------------------
+myname="${0##*/}"
+mydir="${0%/*}"
+
+TOP_DIR="$( pwd )"
+export myname mydir TOP_DIR
+
+. "fs/custom/functions"
+
+# This script can deal with extracting the rootfs tarball, but we need to
+# be root for that.
+if [ $(id -u) -ne 0 ]; then
+ printf "error: not root\n"
+ exit 1
+else
+ main "${@}"
+fi
--
1.8.1.2
More information about the buildroot
mailing list