The design of mdev (mini-udev for busybox).

Rob Landley rob at landley.net
Mon Dec 5 19:08:42 UTC 2005


On Monday 05 December 2005 11:50, Paul Fox wrote:
>  > Rummage, rummage, rummage...
>  >
>  > Having looked about your code, I believe I can say with some authority
>  > that you really, truly, _deeply_ need to look into initramfs.  You can
>  > have all that crud in the gzipped cpio archive built into the kernel
>  > image, and the kernel will extract it for you on boot.
>
> can someone tell me why cpio was chosen for this?  wouldn't tar be
> just as useful, and lower cost in "oh-no-not-another-required-tool" terms?

The actual flame war (in december, 2001) started here:
http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1538.html

And spawned a second thread (specifically on tar vs cpio) here:
http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1587.html

I'd definitely recommend reading all of both threads.

The choice of cpio happened because A) Al Viro vetoed tar in the kernel, 
calling it "ugly as hell", B) Al Viro wrote the code implementing cpio.

(LWN did its usual excellent job summarizing:
http://lwn.net/Articles/14776/
But didn't focus on tar vs cpio specifically.)

The things to realise specifically on tar vs cpio are:

1) The cpio format is simpler for the kernel to parse.  Internally, tar is a 
bit of a hack with gnu extensions and such.

Basically, this is a complete spec for the cpio format the kernel uses, in 112 
(mostly english) lines, from the kernel source browser:
http://sosdg.org/~coywolf/lxr/source/Documentation/early-userspace/buffer-format.txt

And you can't do that with tar. :(

2) cpio actually predates tar, it's been part of unix from the AT&T days, and 
it's the archive format used internally by Red Hat's Package Manager (RPM).  
We already have extract support for it in busybox (or we couldn't do RPM), we 
just never bothered to add create support.  (And we don't need to because  
the kernel made a tool to do it, although we probably should.)

So saying "oh no, not another tool!" is kinda silly since this predates System 
V and any system that uses RPM is likely to have cpio installed on general 
principles.  You just never noticed.  (By the way, Red Hat also uses cpio for 
its device driver disks.  I first encountered cpio when I was assigned to 
create a red hat device driver disk for 7.0 and I ran "file" on the file in 
there and it went "gzipped cpio archive" and I went "what the...?"  This was 
over five years ago...)

> we're using the excellent inittar patches from:
>     http://www.escape.de/users/outback/linux/index_en.html
> for this purpose.  (and we have 2.6 patches if anyone's interested.)
> having no more need for creating an initrd is nice at build time
> (no need to be root), and at runtime: instead of setting up an 
> initrd, and then needing tmpfs in addition, the tarball goes
> right into tmpfs, which gets mounted directly as root.

Did you read my initramfs documentation?
http://sosdg.org/~coywolf/lxr/source/Documentation/filesystems/ramfs-rootfs-initramfs.txt

You don't need to be root for the kernel to build a cpio archive for you 
during the kernel build.  In fact you don't even need cpio installed on the 
system.

The kernel's usr/gen_init_cpio.c is entirely self-contained.  The kernel's 
built-in boot time extractor is also (obviously) entirely self-contained, so 
what the actual format is would be more or less irrelevant.  They could have 
invented their own, but chose to go with the simplest existing standard.  
(And the "technically best choice" was not the same as "most popular".)

So your external patch's entire reason for existing is because you want to use 
a different format for the kernel's internal data storage.  (Out of 
curiosity, is there a patch for using "zip"?)

> i realize initramfs solves a slightly different problem than this,

Not really.  You're re-inventing the wheel.  The text file format 
gen_init_cpio accepts was created exactly so you _wouldn't_ have to be root 
to create a cpio archive, yet could still specify ownership and permissions 
on all the files and directories and device nodes and so on the archive 
should contain.

My initramfs documentation gives an example of how to use it...

> but i'm baffled by the cpio choice.

It's smaller and simpler, it's a standard that's actually older than tar, and 
Al Viro put his foot down when tar was suggested, calling it "ugly as hell":

http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1540.html
http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1550.html

So what if it's not what the GNU guys standardized on?  Linux ain't part of 
the gnu project.  I admit that the command line syntax of the standard cpio 
utility sucks _mightily_.  For example, the syntax to extract the cpio 
archive created by the linux kernel build is:

cpio -i -d -H newc -F initramfs_data.cpio --no-absolute-filenames

Which is just too intuitive for words.  But the kernel's built-in tools have a 
_much_ better user interface.  The text file format gen_init_cpio takes is 
pretty straightforward.  Here's the actual one I'm using for my initramfs:

dir /dev 755 0 0
nod /dev/console 644 0 0 c 5 1
nod /dev/loop0 600 0 0 b 7 0
dir /bin 755 1000 1000
slink /bin/sh busybox 777 0 0
dir /proc 755 0 0
dir /sys 755 0 0
dir /mnt 755 0 0
file /init initramfs/init.sh 755 0 0
file /bin/busybox initramfs/busybox 755 0 0

And glancing at it, I've got the ownership on my bin directory wrong. :)

If you run "usr/gen_init_cpio" after the kernel build it'll dump a usage 
message telling you the format of this file, and the kernel also has a 
"scripts/gen_initramfs_list.sh" to do it for you.  (Which is how my ownership 
got wrong in the above example script, I suspect.  Hasn't caused an actual 
problem, but I need to fix it...)

Rob
-- 
Steve Ballmer: Innovation!  Inigo Montoya: You keep using that word.
I do not think it means what you think it means.



More information about the busybox mailing list