[PATCH] libc: Add support for NUMA system calls.

[Will Newton]

On Fri, Sep 17, 2010 at 5:58 AM, Paul Mundt <lethal at linux-sh.org> wrote:
> On Wed, Sep 15, 2010 at 04:15:28PM +0100, Will Newton wrote:
>> Add support for get_mempolicy(2), mbind(2), migrate_pages(2),
>> move_pages(2) and set_mempolicy(2).
>>
>> These system calls can be used to manipulate the CPU and memory
>> allocation strategies of a process when used with a NUMA enabled
>> kernel. NUMA is used by a number of embedded SoCs to allow
>> processes to make use of low-latency on-chip memories.
>>
>> The numaif.h header has been taken from the numactl/libnuma
>> package.
>>
>> Signed-off-by: Will Newton <will.newton at imgtec.com>
>> ---
>>
>> libnuma doesn't work well with uClibc because it uses symbol versioning
>> which isn't supported by uClibc ldso. It's also a rather heavyweight solution
>> for the embedded space where typically that level of complexity is not
>> required.
>>
>> This patch adds the system calls and header from libnuma as a lightweight
>> alternative.
>>

Hi Paul,

Thanks for the feedback.

> While I don't see anything wrong with wiring up the system calls, they
> were not really intended for applications to be calling in to them
> directly. While providing an API-complete reimplementation of libnuma is
> more than we want to do in uClibc, it would probably be worthwhile
> hacking up a lightweight numactl for busybox that backs directly in to
> them.

Is there any documentation that explains how these system calls are
meant to be used? For my purposes they seem a much more useful API
than libnuma, which seems to be more designed for "you are running on
a big system with lots of CPUs that may change at runtime" whereas I'm
looking at small systems with fixed numbers of fixed function nodes
(e.g. like the ones you were talking about in your 2007 ELC paper).

> These probably should also not be in the main libc, uClibc would be
> better off just providing its own tiny libnuma. You can easily follow the
> case of what we're already doing for librt and so on at the top level,
> for example. At present anything that's using <numaif.h> will be linking
> against libnuma by default, so this will at least keep things slightly
> consistent, even if the full libnuma API isn't provided.

It would certainly be possible to split the system call wrappers out
into a separate lib, but I do have some reasons for not doing that:

1. It does not implement the libnuma API, so -lnuma will not work as a
user might expect and would conflict with an actual libnuma.
2. The system call wrappers are very small.

The system calls in libnuma are defined as weaks, which I took to
imply "these should really be in libc but we didn't want to wait for a
glibc release cycle", although that may be a misreading of the
situation. Either way, it means that this patch and libnuma are not
incompatible - the numaif.h header is identical, and the weak
definitions of the system calls in libnuma mean that linking should
work fine too.