[PATCH] implement futimes

Sun Nov 22 22:21:03 UTC 2009

On Sunday 22 November 2009 08:14:55 Sergio M. Ammirata, Ph.D. wrote:
> Carmelo,
>
> Not to be disrespectful, but I agree with Rob's assessment. I too have been
> using the uclibc library for more than 5 years and it's releases are few
> and far between compared to many other open source projects.
>
> I have seen at least 3 other branches of uclibc being maintained by other
> people outside the official repositories because of the slow or
> non-existent response from the uclibc developers.
>
> The fact that the nptl branch is still not in the main branch is in my
> opinion a lack of vision from the maintainers of the project. The nptl code
> is needed in order to compile any recent version of VLC for example.
>
> Recently, I spent over 60 hours of work using multiple machines to perform
> simultaneous compilations trying to get the nptl branch working on x386.
> Unfortunately, I had to stop because the solution to the problem was beyond
> my skillset. I am able to do some coding and prepare patches for simple
> things and offered my time to test anything that needed to be tested to no
> avail.

Peter Mazinger once explained to me the difference between the various uClibc 
NPTL implementations and what pieces of each needed to be put together to 
unify it.  Unfortunately, I don't remember the details, and Manuel Nova drove 
Peter away from uClibc development back in 2006 by complaining that the large 
number of commits Peter was making to the mainline -dev repository were 
breaking Manuel's private out-of-tree patches which he refused to commit to 
mainline.

That was the point at which the uClibc development process officially graduated 
to "diseased", if you ask me...

> I also reported an application crash that happens with the 0.9.30.1 and
> newer libc malloc code and that was not there in 0.9.28. I could not find a
> solution for it.

Which target, what .config, what application, how do you reproduce the 
crash...?

If you can reproduce it under my build system (or build with my toolchain), 
I'll debug it and fix it (in my de-facto fork, grumble):

  http://impactlinux.com/code/firmware/dowloads

I've already had to dig into the malloc code and fix stuff that nobody EVER 
TESTED:

  http://landley.net/notes-2008.html#27-10-2008

> The developers of the app swear that it is a bug in the
> ulibc libraries and the uclibc developers say those types of changes of
> behavior and normal and expected and that it is the application doing
> something wrong (even though it is exactly the same application source code
> in both cases).
>
> My point is, has the target user of the uclibc changed since its
> conception?
>
> Are you guys only focusing on big companies that develop embedded devices?

I'm not.  I'm focusing on any developers who wants to play with non-x86 
targets under qemu, whether it's hobbyists learning or professionals doing 
emulated build clusters on cheap x86 hardware.

> It would explain why there is still no support for 386 on the nptl branch.

There are three different NPTL targets, written by three different people who 
didn't get to see each other's code because SJ Hill refused to check his 
original NPTL code until he got paid for it:

  http://ibot.rikers.org/%23uclibc/20060325.html.gz

>04:11.09	ds	so I'll just slog through and debug it
>04:11.16	sjhill	debug what?
>04:11.24	ds	why I'm getting segfaults
>04:11.29	sjhill	*sigh*
>04:11.34	sjhill	you're wasting your time
>04:11.47	sjhill	i have the 2000+ lines of patches needed to make the dynamic
> loader work 
>04:12.05	sjhill	or are you talking just normal binaries?
>04:12.18	ds	both, actually
>04:12.22	sjhill	forget NPTL
>04:12.33	sjhill	the normal stuff, that is a problem
>04:13.07	ds	are you saying that you have stuff that's not checked into the
> nptl branch? 
>04:13.14	sjhill	yes.
>04:13.17	ds	ah, ok
>04:13.21	ds	i missed that part
>04:13.24	ds	sorry
>04:13.37	sjhill	np
>04:13.44	sjhill	i thought i was being clear, but guess not
>04:14.57	ds	I can't say that I tested static binaries too thoroughly
>04:15.19	ds	so I may have just done something blatantly stupid
>04:15.28	sjhill	static NPTL binaries will absolutely not work
>04:15.34	sjhill	i haven't even debugged them
>04:15.40	sjhill	they segfault all over the place
>04:16.10	dalias	it looks like i'm going to have to write my own pthread
> implementation.. 
>04:16.14	dalias	maybe uclibc can use it once i'm done
>04:16.22	dalias	imo nptl is hopelessly broken
>04:16.27	sjhill	you are an idiot
>04:16.42	dalias	um, why do you say that?
>04:16.54	sjhill	the reason the code is not checked in is because i did it
> under contract 
>04:16.59	sjhill	once i get paid, the code gets released
>04:17.02	sjhill	which is in two months
>04:17.12	sjhill	impatient aren't we?

So there you have it, S.J. Hill and Manuel Nova both kept development out of 
tree for a long time, refusing to check it in until they got paid, and the 
result was that mainline development slowed to a crawl, independent 
("competing") attempts to implement the same functionality were kept out of 
the tree, and the longer they stayed separate the harder it got to merge them.

And to this day, there are people who still insist this wasn't really a 
problem, and continue to do the same thing.

> Is there someone within the uclibc developer organization maintaining the
> priorities of the limited developer resources?

In theory, the maintainer.  In practice, not that I've noticed.

> If so, can those be
> communicated to the user community through the web site so that we can all
> make an informed decision whether to continue using the library? Embedded
> storage has become larger and larger recently and the standard glibc
> library is no longer an outrageous choice.

The main reasons I still use uClibc (other than inertia) are:

1) glibc went lgplv3, which I don't want to get on me.
2) glibc requires perl to build.

> If it is a matter of resources, why don't you post a message on your web
> site asking for help. I am sure there are hundreds of qualified individuals
> that can help with the project.

The problem isn't a lack of developer time.  Lots of developers would be happy 
to help out.  I've done it, and here's BusyBox maintainer Denys Vlasenko 
repeatedly taking an interest:

  http://lists.uclibc.org/pipermail/uclibc/2008-July/040588.html
  http://lists.busybox.net/pipermail/uclibc/2009-February/042022.html

The problem is coordination.

First of all, they decided to do the merge somewhere other than the -dev 
branch, which was either an enormous mistake in retrospect or they _meant_ it 
to take 3 years and still be stalled:

  http://lists.uclibc.org/pipermail/uclibc/2006-August/037013.html

Secondly, the project is set up so that there are bottlenecks that only 
specific people can resolve, and those people then sit on those bottlenecks for 
months until somebody else is made the new bottleneck.  (Oddly, this somebody 
is never the project's maintainer...)  Meanwhile other people post random 
unrelated patches here in a steady trickle, wihch can never go into a release 
due to said bottlenecks.

The fun part is the musical rotating bottlenecks.  As noted above, sjhill was 
the NPTL bottleneck back in 2006.  His merge estimates tended to be things 
like "two months":

  http://lists.uclibc.org/pipermail/uclibc/2006-March/035793.html

Five months later, Peter Mazinger beat some activity out of him:

  http://lists.uclibc.org/pipermail/uclibc/2006-August/016213.html
  http://lists.uclibc.org/pipermail/uclibc/2006-August/036995.html
  http://lists.uclibc.org/pipermail/uclibc/2006-August/037042.html
  http://lists.uclibc.org/pipermail/uclibc/2006-August/037121.html

And so it continued on in 2007, where he was "a little behind doing customer 
releases":

  http://lists.uclibc.org/pipermail/uclibc/2007-May/038742.html
  http://lists.uclibc.org/pipermail/uclibc/2007-August/039210.html

Of course I started complaining about lack of releases and questioning the 
nptl branch, yet again.  Of course there were big long threads...

  http://lists.uclibc.org/pipermail/uclibc/2007-September/039215.html
  http://lists.uclibc.org/pipermail/uclibc/2007-September/039236.html

Just like a year later: big long threads:
  http://lists.uclibc.org/pipermail/uclibc/2008-September/019976.html

Anyway, at some point sjhill either threw in the towel or stopped responding, 
and Carmelo stepped up to do the merge, so he became the one and only person 
trying to merge the three branches, and thus the bottleneck:

  http://lists.uclibc.org/pipermail/uclibc/2008-April/019260.html

And he remained the bottleneck for a while:

  http://lists.uclibc.org/pipermail/uclibc/2008-April/040090.html
  http://lists.busybox.net/pipermail/uclibc/2009-February/041949.html
  http://lists.uclibc.org/pipermail/uclibc/2009-June/042553.html

This message is particularly amusing in retrospect:

  http://lists.uclibc.org/pipermail/uclibc/2008-April/019260.html

He was the guy who just declared this problem no longer interesting.

I've never understood why it's a separate branch:

  http://lists.uclibc.org/pipermail/uclibc/2008-December/041670.html

Why we can't just check in the three implementations into -dev and merge them 
in situ.  Put the code into the tree and let the uClibc development community 
have at.

This is especially crazy since NPTL support for uClibc has been implemented 
from scratch at least three different tiimes.  Sjhill did one for mips:

  http://kernel.org/doc/ols/2006/ols2006v1-pages-409-420.pdf
  http://kernel.org/doc/ols/2006/slides/sjh-ols-2006-presentation.odp

Carmelo did one for sh4:

  http://lists.uclibc.org/pipermail/uclibc/2007-August/039207.html

And of course codesourcery did its own from scratch for arm, and maintained it 
while while trying to get it merged:
  http://lists.uclibc.org/pipermail/uclibc/2006-September/037393.html
  http://lists.uclibc.org/pipermail/uclibc/2009-January/041779.html

In the amount of time that's been wasted on the bottleneck-based merge 
approach in a separate -dev tree, NPTL could have been implemented for every 
single supported target about three times over.

Meanwhile, the "don't break -dev by dumping nptl into it" approach has meant 
that we nobody's interested in testing -dev and nobody's interested in cutting 
releases from -dev, so stability of -dev and development speed in -dev are 
both much much slower than if the nptl patches had just been merged back in 
2006.

Instead, like the proverbial cutting wedge whose edge splits into two points, 
foreward progress goes to zero.

> Regards,

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds