Should BusyBox and uClibc also make a "flag version" like Embedded Linux?

Wed Nov 10 22:04:06 UTC 2010

No, they shouldn't.

On Friday 05 November 2010 11:33:36 Bradley M. Kuhn wrote:
> LWN.net  wrote at 18:30 (EDT) on Thursday:
> > As a result of discussions held at two recent embedded Linux summits
> > (and reported back to the recent Kernel Summit), the [Linux] community
> > has decided to identify specific kernel versions as "flag versions" to
> > try to reduce "version fragmentation". On the linux-embedded mailing
> > list, Tim Bird ... has announced that 2.6.35 will be the first
> > embedded flag version, and it will be supported by (at least) Sony,
> > Google, MeeGo, and Linaro. "First, it should be explained what having
> > a flag version means. It means that suppliers and vendors throughout
> > the embedded industry will be encouraged to use a particular version
> > of the kernel for software development, integration and testing. Also,
> > industry and community developers agree to work together to maintain a
> > long-term stable branch of the flag version of the kernel (until the
> > next flag version is declared), in an effort to share costs and
> > improve stability and quality."
>
> Tim Bird's email post that LWN is quoting from above can be seen at:
> http://lwn.net/Articles/413341/rss
>
> (There's also a brief summary at
> http://lwn.net/SubscriberLink/413036/a954ac0a143a99d9/ of the discussion
> that took place at the embedded kernel summit on this).

Neither uClibc nor busybox has even 1% of the churn of the linux kernel.

Look: Linux has to deal with binary only modules, which the developers hate.  
To compensate for this, they go out of their way to avoid having a stable 
internal API (which could argualy be used as a copyright barrier and thus 
prevent the modules from being derived works of the Linux kernel to which 
GPLv2 must apply).  To avoid this, they break internal compatability 
essentially every release.  They go out of their way _not_ to make forward-
porting modules easy (otherwise they'd have a checklist each release saying do 
this this and this, and you'll be up to date).

Instead they promote as much churn as possible, every single release, and make 
keeping up with it a black art.  This is to force a stark binary choice with 
no middle ground: either get your code merged in-tree where they forward port 
it, or your code remains stuck on older versions and can only be forward-
ported by a labor-intensive combination of research, guesswork, and elbow 
grease, and luck.

BusyBox does not have this problem.  A patch against an 18 month old version 
of the linux kernel is 6 releases out of date, and has almost no chance of 
applying or working on a current kernel.  A patch against an 18 month old 
version of busybox will probably apply to current busybox unmodified.

> I read this and began wondering if uClibc and BusyBox had an interest in
> doing "flag versions" in tandem with Linux.

uClibc has gone entire years without a release on more than one occasion.  
(And I don't mean like 2009 where there was no new _development_ release and 
only the 0.9.30.1 bugfix-only release.  No, I mean there wasn't any new release 
of any kind from uClibc in the calendar year 2006, and there was a year and a 
half gap between 0.9.29 and the following 0.9.30-rc with nothing in between.)

Even under the new management, the project's last release was seven months 
ago, and my Aboriginal Linux project is carrying eight uClibc fix patches since 
then.  I just posted to uclibc about something that got a fix posted in 
February but still isn't fixed in 0.9.31, and of course there have been no 
releases since so it's still broken in current.  I've forwarded patches to the 
uclibc list from 2007 that hadn't been applied to the development branch yet, 
let alone made it into a release.  I reply to six month old messages more 
often than I reply to current ones.

As for the development branch: the OLS paper on NPTL for uClibc was published 
in 2006, but NPTL support still hasn't made it onto a uClibc release yet.  A 
human this constipated would be dead by now.

In that context, the idea of marking specific releases of uClibc "special" is 
black comedy.  There aren't enough release of this project for any release 
_not_ to be special.

> As most of you know, I do a
> lot of GPL enforcement work for BusyBox, and I find frequently companies
> stuck on ridiculously old versions of BusyBox and uClibc.  I constantly
> encourage companies, as part of compliance efforts, to use more recent
> versions but I have been mostly unsuccessful in that part of the effort.

Flag versions will do nothing for this.  Embedded developers aren't stuck on 
ridiculously old versions of busybox because newer versions of busybox don't 
_work_.  Or because newer versions of busybox aren't a drop-in replacement for 
the old ones.  They're stuck because they don't replace software that works 
for them.  Last I checked, linksys routers were still using a kernel from 
2003, and since then Cisco dissolved its linksys engineering team.

The kernel is creating workarounds for its own problems.  I really don't see 
how they apply to uClibc or busybox.

> Thus, I thought perhaps this "flag version" of Linux is an opportunity
> for the BusyBox and uClibc community to also pick versions that the
> embedding companies will standardize around,

Ha!

Ok, you actually got a very quiet giggle from me, which qualifies as a "laugh 
out loud".  The depth of the epic fail to understand the embedded world 
embodied in that sentence is just awe inspiring.  Really it is.

> and thus allow the BusyBox
> and uClibc community to better dictate what versions end up being
> de-facto long-term-support versions.

I still get emails about busybox versions from Erik's tenure.  I haven't been 
busybox maintainer since 2006, and I still get emails about it.  Somebody 
emailed me about a busybox GPL violation over the weekend, and I haven't done 
_THAT_ in over a year either.  (The ongoing french thing notwithstanding.)

By the way, from a development standpoing, why on earth would we say "don't 
bother trying out our new releases"?  Why would we send that message to 
anybody?

> BTW, both Erik and I have known Tim Bird for some time.  I can't speak
> for Erik, but I am willing to spend some time reaching out to Tim to
> coordinate this on behalf BusyBox/uClibc if there's interest from the
> BusyBox and uClibc community.

For years, Linus's position was "if you want long term support on a specific 
version, that's what distributions are for.  If you want _us_ to fix bugs, 
upgrade to the newest version and tell us if it's still broken."

Some people weren't happy with that, and decided to set up an intermediate 
staging area to help distros coordinate their long term support efforts.  
Adrian Bunk backported bugfixes to 2.6.16 for several years:

  http://kerneltrap.org/node/6930
  http://kerneltrap.org/node/7164
  http://lwn.net/Articles/266707/

And when that became hopelessly out of date he moved to 2.6.27:

  http://lkml.org/lkml/2008/10/12/2

Tim is basically taking over this job from Adrian.  It's not news.  Honest and 
truly it isn't.  (Good grief, the wikipedia article on Linux_kernel covers 
this.)

The fact that it went on for many years and you never even heard about it 
shows how UTTERLY USELESS this effort is.  If distros choose to coordinate, 
they'll do so.  If they choose to differentiate, they'll do so.

Bradley?  Go to http://kernel.org/doc/video.html and watch the last link.  
It's most of an hour long, but really helps to explain modern open source 
software release management better than any other single resource I've found.

Rob
-- 
GPLv3: as worthy a successor as The Phantom Menace, as timely as Duke Nukem 
Forever, and as welcome as New Coke.