The busybox "printf" command is always 32 bit even when the shell isn't.

[Rob Landley]

On Sunday 04 January 2009 15:04:50 Denys Vlasenko wrote:
> On Sunday 04 January 2009 20:11, Rob Landley wrote:
> > > > > > $ printf %x 2251797561885434; echo
> > > > > > 7ffff79c842fa
> > > > > > $ ./busybox printf %x 2251797561885434; echo
> > > > > > 79c842fa
> > > > > >
> > > > > > I need 64 bit math to convert timeconst.pl to timeconst.sh in the
> > > > > > Linux kernel build...
> > > > >
> > > > > By your command
> > > >
> > > > Thanks, and I'm happy to have this patch now to help get the perl
> > > > removal stuff into the kernel, but going forward I think there should
> > > > probably be a big config switch, "use 64 bit math on 32 bit
> > > > platforms".
> > > >
> > > > Generally if you want 64 bit math, you probably want it everywhere,
> > > > but 32 bit systems will be with us in the embedded world for a while
> > > > yet (especially arm and mips and such), and doing unnecessary 64 bit
> > > > math there is especially painful.
> > > >
> > > > Also, what's the deal with ash "built in printf"?  Does this relate
> > > > to nofork, or is it a separate implementation, or...?
> > >
> > > Shells have "builtins". Which are sort of NOFORK, but are enabled
> > > independently of NOFORK. The rationale is that even if you do not
> > > use NOEXEC and NOFORK _applets_ (which is understandable,
> > > they are not as stable as I would like),
> > > you sure want "echo", "test" etc in shells to not be slow as hell.
> >
> > The only reason nofork applets would be unstable is because the applet in
> > question doesn't clean up after itself or uses the wrong error handling
> > functions (exiting on error isn't nice if it kills your shell).
>
> Or shell user running "rm -rfi" and pressing Ctrl-Z at rm's prompt.

This is an existing problem.  Type "read i" and then hit ctrl-z.

And if you think _that_ one's bad, using a builtin command in a pipeline is 
even more fun.  ("printf hello | sleep 100" pretty much forces the shell to 
fork not one but two child processes to maintain its sanity.)

(I spent most of a year looking at strange corners of shell implementation 
back when unifying the busybox shells was my problem.)

> Think what shell needs to do to properly create a child, and clean up
> open fds (opened by rm by virtue of doing recursive directory removal),
> rm's allocaled memory etc etc etc. This is not simple.

I know.  But I also know that shell builtins and nofork applets are mostly the 
same problem domain, and that the _easy_ thing to do is treat cd and exit and 
such as "hidden" nofork commands (commands that don't have their names 
published externally by busybox) and then put 'em in the same process lookup 
table.

In which case, inserting all builtin commands at the beginning or the end of 
the $PATH is the config option, and whether or not to fork becomes an 
implementation detail, and having individual config options for each command 
seems a bit silly.

> Perhaps we can declare that NOFORKs are not background'able from shell,
> and accept the fact that this either severely restricts the set of applets
> that can be NOFORKs (for example, cat can't be: "cat 14gig_file >/dev/disk"
> takes a long time), or creates an unexpected behavior for them
> when they are used from shell.
>
> What do you think about it?

Well, ask yourself what the _advantage_ of nofork is.

Fork is a fairly cheap operation in Linux, Ingo molnar managed a couple 
million fork/exit cycles per second (described about halfway through 
http://kerneltrap.org/node/517 ) when he cleaned up the fork/exit code for 
scalability a few years back.

What you're really trying to avoid is an _exec_ (and the corresponding need 
for /proc/self/exe or similar), and there are a lot more applets that the 
shell can fork and then run as a function call without doing an exec than it 
can run in the same process context.

So what you need is a NOEXEC category, and that's more like a shell config 
option rather than an individual applet thing.  It's fairly generic: name an 
applet that _wouldn't_ work if you did a fork() and then called its main() as 
a function with the right arguments.  (There might be one, but tagging those 
as NOT working is more interesting than tagging the 99% that do.  You can even 
reuse your shared global struct, since the parent's copy won't get stomped, so 
it's not that bad from a resource perspective.)

Now the funky bit there is that vfork() won't work that way, only real fork().  
So you'd need a config option to disable this behavior (vfork() with exec, vs 
real fork() without exec).

Yes, you'd might want to take care to avoid leaking resources like file 
handles and large malloc() memory chunks into the new process (although copy 
on write remains your friend in the malloc() case), but if you only do noexec 
calls from the _shell_ rather than from arbitrary other applets that need to 
spawn stuff, and if you only do it in cases where you'd do a $PATH lookup 
anyway (I.E. no / in the name), then that's a constrained set of cases to get 
right.

> > But if you _have_ a printf implementation that you know is safe to use in
> > the shell, why would you have a second applet implementation?  And if
> > they are the same implementation, what's the second config entry for?
>
> They are the same implementation. What is "the second config entry"?
> I see only one entry which controls whether printf is a builtin or not.

There's a config entry for printf under coreutils, and there's a config entry 
for printf acting as a builtin under the ash menu.  That's two config entries 
for the same command, and the difference in behavior seems a bit small to 
warrant it.

Good to know it's one implementation, though.  I see ash has been greatly 
cleaned up since I was last paying attention to it. :)

Rob