Thread: Portability report: ninja

Meson depends on ninja, so it behooves us to look into portability
of ninja if we're considering migrating to meson as the build system.
I tried it out quickly on some ancient platforms, and here's what
I found.

1. The documentation is kinda sucky, eg it fails to explain how
to run ninja's own regression tests.  I ended up with this
reverse-engineered recipe:

git clone git://github.com/ninja-build/ninja.git
cd ninja
git checkout release
./configure.py --bootstrap
./ninja ninja_test
./ninja_test
sudo cp ninja /usr/local/bin          # or location of choice

We'll probably have to include that in our own documentation.

2. You need Python to build it this way.  There is also a process
for bootstrapping using cmake, but that doesn't seem like a more
attractive dependency for old platforms.  Fortunately for my
purposes here, it seems to work with fairly old Python --- I built
successfully with python 2.6.2, though not with 2.4.1.

3. You also need a C++ compiler, but really old ones will work.
I did not see any problems with g++ as old as 4.0.1, other than
some compiler warnings about non-virtual destructors.

4. It built and passed self-test on macOS Leopard (10.5.8), which is
pretty old ... but not old enough for prairiedog, which is stuck on
10.4.x.  There, the build fails miserably for lack of <spawn.h>.
It looks like that was added to POSIX in 2001, so one could have
wished for it in a 2005-vintage OS; but nope, Apple took another
couple of years to get around to that.  I'm not even going to bother
trying on gaur's pre-turn-of-the-century OS.

5. It built and passed self-test on Solaris 11, but failed self-test
on Solaris 10.

6. While configure.py thinks it knows what to do on AIX, it fails
on AIX 7.1 and 7.2:

Traceback (most recent call last):
  File "./configure.py", line 544, in <module>
    if platform.is_aix() and not platform.is_os400_pase():
  File "./configure.py", line 103, in is_os400_pase
    return self._platform == 'os400' or os.uname().sysname.startswith('OS400')
AttributeError: 'tuple' object has no attribute 'sysname'

Possibly the ninja guys would take a patch for that (or maybe
this is a you-need-python-3 case?).  I do see /usr/include/spawn.h
on that platform, so there's room to hope it'd work.


So it's pretty clear that if we go this way, it'll be the end of
the line for support for some very old OS versions.  I can't,
however, argue with the idea that it's reasonable to require
POSIX 2001 support now.  Based on these results, I doubt that
ninja will give us trouble on any platform that isn't old
enough to get its drivers license.

            regards, tom lane

Hi,

On 11/1/21 15:25, Tom Lane wrote:
> So it's pretty clear that if we go this way, it'll be the end of
> the line for support for some very old OS versions.  I can't,
> however, argue with the idea that it's reasonable to require
> POSIX 2001 support now.  Based on these results, I doubt that
> ninja will give us trouble on any platform that isn't old
> enough to get its drivers license.


You can also look at it as:


If PostgreSQL choose a newer build system then it is up to the companies 
owning the "non-supported" operating systems to add support for the 
build system in question; not the PostgreSQL community.


+1 for POSIX.2001 and meson/ninja.


Best regards,

  Jesper

On Tue, Nov 2, 2021 at 8:25 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 6. While configure.py thinks it knows what to do on AIX, it fails
> on AIX 7.1 and 7.2:
>
> Traceback (most recent call last):
>   File "./configure.py", line 544, in <module>
>     if platform.is_aix() and not platform.is_os400_pase():
>   File "./configure.py", line 103, in is_os400_pase
>     return self._platform == 'os400' or os.uname().sysname.startswith('OS400')
> AttributeError: 'tuple' object has no attribute 'sysname'

Yeah, the result type changed: os.uname()[0] works on python2 AND python3.

On Tue, Nov 2, 2021 at 8:25 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 5. It built and passed self-test on Solaris 11, but failed self-test
> on Solaris 10.

I haven't had time to actually do anything with it yet, but I can
report that meson and ninja are, as of quite recently, available in
the package repository of OpenIndiana.

That's the distribution of illumos I found very convenient to work
with last time I turned the Solaris-family machines red in the build
farm.  The reason I like it is that there's a rolling "vagrant" image
(libvirt and virtualbox flavours), so it's possible to spin up a
working VM with minimum effort.  My current way of keeping track of
what commands are needed to install the various packages PostgreSQL
needs for niche OSes corresponding to our build farm is to record it
all in Vagrant files.

https://github.com/macdice/postgresql-dev-vagrant/blob/master/openindiana/Vagrantfile

Hi,

On 2021-11-01 15:25:08 -0400, Tom Lane wrote:
> Meson depends on ninja, so it behooves us to look into portability
> of ninja if we're considering migrating to meson as the build system.
> I tried it out quickly on some ancient platforms, and here's what
> I found.

Thanks, that's helpful!


> 1. The documentation is kinda sucky, eg it fails to explain how
> to run ninja's own regression tests.  I ended up with this
> reverse-engineered recipe:
> 
> git clone git://github.com/ninja-build/ninja.git
> cd ninja
> git checkout release
> ./configure.py --bootstrap
> ./ninja ninja_test
> ./ninja_test
> sudo cp ninja /usr/local/bin          # or location of choice
> 
> We'll probably have to include that in our own documentation.

Or perhaps a helper script? I was wondering whether we'd eventually want a
wrapper ./configure / makefile. Perhaps not worth it though - there's enough
projects using meson and ninja that it might add more confusion than being
helpful.


> There is also a process for bootstrapping using cmake, but that doesn't seem
> like a more attractive dependency for old platforms.

Yea. Especially given that meson itself needs python anyway.


> Fortunately for my
> purposes here, it seems to work with fairly old Python --- I built
> successfully with python 2.6.2, though not with 2.4.1.

meson will need a newer python though...


> 4. It built and passed self-test on macOS Leopard (10.5.8), which is
> pretty old ... but not old enough for prairiedog, which is stuck on
> 10.4.x.  There, the build fails miserably for lack of <spawn.h>.
> It looks like that was added to POSIX in 2001, so one could have
> wished for it in a 2005-vintage OS; but nope, Apple took another
> couple of years to get around to that.  I'm not even going to bother
> trying on gaur's pre-turn-of-the-century OS.

> 5. It built and passed self-test on Solaris 11, but failed self-test
> on Solaris 10.

I think we can live with those...


> 6. While configure.py thinks it knows what to do on AIX, it fails
> on AIX 7.1 and 7.2:
> 
> Traceback (most recent call last):
>   File "./configure.py", line 544, in <module>
>     if platform.is_aix() and not platform.is_os400_pase():
>   File "./configure.py", line 103, in is_os400_pase
>     return self._platform == 'os400' or os.uname().sysname.startswith('OS400')
> AttributeError: 'tuple' object has no attribute 'sysname'
> 
> Possibly the ninja guys would take a patch for that (or maybe
> this is a you-need-python-3 case?).  I do see /usr/include/spawn.h
> on that platform, so there's room to hope it'd work.

That does seem like it'd be a issue. Briefly trawling the ninja git log it
does look like there's regular-ish maintenance stuff for AIX, so I'd hope we
could get it fixed. I do suspect it's just a python3 issue, as Thomas noted.


> Based on these results, I doubt that ninja will give us trouble on any
> platform that isn't old enough to get its drivers license.

Agreed.

There's also alternative compatible ninja implementation in C99 ([1]). But I
think it's minimum requirements aren't actually lower than ninja's (it says it
requires posix 2008).

Greetings,

Andres Freund


[1] https://github.com/michaelforney/samurai

Andres Freund <andres@anarazel.de> writes:
> On 2021-11-01 15:25:08 -0400, Tom Lane wrote:
>> Fortunately for my
>> purposes here, it seems to work with fairly old Python --- I built
>> successfully with python 2.6.2, though not with 2.4.1.

> meson will need a newer python though...

Yup.  I was just trying to establish what the lower bound was for
ninja, since they (ahem) don't document it.

The meson docs say it needs Python >= 3.6, which I think is going
to be an issue, but I've not researched it yet.  What I was mainly
curious about today was whether anything using ninja as the backend
could meet our portability goals.  My mind's more or less at ease
on that, but I am afraid that requiring 3.6 is going to be a problem
on LTS distributions.  That's a topic for another thread though.

            regards, tom lane

On 2021-11-02 09:27:17 +1300, Thomas Munro wrote:
> On Tue, Nov 2, 2021 at 8:25 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > 6. While configure.py thinks it knows what to do on AIX, it fails
> > on AIX 7.1 and 7.2:
> >
> > Traceback (most recent call last):
> >   File "./configure.py", line 544, in <module>
> >     if platform.is_aix() and not platform.is_os400_pase():
> >   File "./configure.py", line 103, in is_os400_pase
> >     return self._platform == 'os400' or os.uname().sysname.startswith('OS400')
> > AttributeError: 'tuple' object has no attribute 'sysname'
> 
> Yeah, the result type changed: os.uname()[0] works on python2 AND python3.

FWIW, with python3 ninja builds and tests successfully on AIX.