Thread: A small note on the portability of cmake

A small note on the portability of cmake

From
Tom Lane
Date:
I tried to build cmake on OpenBSD running on my old HPPA hardware,
not because I cared about cmake per se but because it's a dependency
of something I did care about.  Failed miserably.  It turns out cmake has
a hard dependency on libuv which (a) has a hard dependency on atomic ops
and (b) according to its own docs, doesn't really care about any
platforms other than Linux/macOS/Windows and maybe FreeBSD.  It looks
to me like switching to that toolchain would greatly constrict our
ability to build on non-mainstream platforms, simply because the tools
themselves fail to build.

Just sayin' ...

            regards, tom lane


Re: A small note on the portability of cmake

From
Jesse Zhang
Date:
On Sat, Jan 19, 2019 at 8:50 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Failed miserably.  It turns out cmake has a hard dependency on libuv
> which (a) has a hard dependency on atomic ops and (b) according to its
> own docs, doesn't really care about any platforms other than
> Linux/macOS/Windows and maybe FreeBSD.>

I beg to disagree with point b above: the libuv project continually
receives (and accepts) patches fixing bugs specifically for OpenBSD.
Atomic ops (compare-and-exchange) might be a harder dependency to shed
for libuv. Does the fallback onto compiler intrinsics
(__sync_val_compare_and_swap, or on GCC 4.7+,
__atomic_compare_exchange_n) not work here?

Jesse


Re: A small note on the portability of cmake

From
Andres Freund
Date:
Hi,

On 2019-01-19 23:39:37 -0800, Jesse Zhang wrote:
> On Sat, Jan 19, 2019 at 8:50 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Failed miserably.  It turns out cmake has a hard dependency on libuv
> > which (a) has a hard dependency on atomic ops and (b) according to its
> > own docs, doesn't really care about any platforms other than
> > Linux/macOS/Windows and maybe FreeBSD.>
> 
> I beg to disagree with point b above: the libuv project continually
> receives (and accepts) patches fixing bugs specifically for OpenBSD.
> Atomic ops (compare-and-exchange) might be a harder dependency to shed
> for libuv. Does the fallback onto compiler intrinsics
> (__sync_val_compare_and_swap, or on GCC 4.7+,
> __atomic_compare_exchange_n) not work here?

HPPA doesn't hardware instructions for atomic ops other than
test-and-set IIRC.

Greetings,

Andres Freund


Re: A small note on the portability of cmake

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> On 2019-01-19 23:39:37 -0800, Jesse Zhang wrote:
>> Atomic ops (compare-and-exchange) might be a harder dependency to shed
>> for libuv. Does the fallback onto compiler intrinsics
>> (__sync_val_compare_and_swap, or on GCC 4.7+,
>> __atomic_compare_exchange_n) not work here?

Nope, the failure manifests as

/usr/local/lib/libuv.so.1.0: undefined reference to `__sync_val_compare_and_swap_4'

when some dependent package tries to use the library.  So the build
failed to notice that the compiler intrinsics don't exist.  (I was
using OpenBSD's packaging of libuv, which I guess doesn't bother
running any test cases during build.)

> HPPA doesn't hardware instructions for atomic ops other than
> test-and-set IIRC.

Indeed, the main reason why I'm interested in keeping this old dinosaur
going at all is that it is so different from other platforms in terms
of what we can assume about spinlocks and atomic ops.  Keeps us honest.

            regards, tom lane


Re: A small note on the portability of cmake

From
Andres Freund
Date:
Hi,

On 2019-01-20 10:15:53 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > HPPA doesn't hardware instructions for atomic ops other than
> > test-and-set IIRC.
> 
> Indeed, the main reason why I'm interested in keeping this old dinosaur
> going at all is that it is so different from other platforms in terms
> of what we can assume about spinlocks and atomic ops.  Keeps us honest.

FWIW, while that clearly is the policy right now, I quite doubt that
it's beneficial. It's not like there's going to be new hardware
platforms without at least cmpxchg / ll/sc support. So I'm not seeing
what not requiring them keeps us honest about.

Greetings,

Andres Freund


Re: A small note on the portability of cmake

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> On 2019-01-20 10:15:53 -0500, Tom Lane wrote:
>> Indeed, the main reason why I'm interested in keeping this old dinosaur
>> going at all is that it is so different from other platforms in terms
>> of what we can assume about spinlocks and atomic ops.  Keeps us honest.

> FWIW, while that clearly is the policy right now, I quite doubt that
> it's beneficial. It's not like there's going to be new hardware
> platforms without at least cmpxchg / ll/sc support. So I'm not seeing
> what not requiring them keeps us honest about.

I think you're being short-sighted.  I agree that any reasonable new
hardware platform would have that functionality in some form, but
it won't necessarily be exactly like x86_64 does it.  The particular
things I think HPPA is keeping us honest about have to do with the
size of spinlocks and whether they initialize to zero or not.
See e.g. 6b93fcd14 for an actual bug caught by that.

            regards, tom lane


Re: A small note on the portability of cmake

From
Andres Freund
Date:
Hi,

On 2019-01-20 14:37:43 -0500, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2019-01-20 10:15:53 -0500, Tom Lane wrote:
> >> Indeed, the main reason why I'm interested in keeping this old dinosaur
> >> going at all is that it is so different from other platforms in terms
> >> of what we can assume about spinlocks and atomic ops.  Keeps us honest.
> 
> > FWIW, while that clearly is the policy right now, I quite doubt that
> > it's beneficial. It's not like there's going to be new hardware
> > platforms without at least cmpxchg / ll/sc support. So I'm not seeing
> > what not requiring them keeps us honest about.
> 
> I think you're being short-sighted.  I agree that any reasonable new
> hardware platform would have that functionality in some form, but
> it won't necessarily be exactly like x86_64 does it.

I agree on that part.  I can't see them doing something as weird as HPPA
however - yes there'll be some ll/sc and some cmpxchg style archs, and
perhaps their alignment requirements will be different, but not much
more than that. And that's largely what we already have covered.


> The particular things I think HPPA is keeping us honest about have to
> do with the size of spinlocks and whether they initialize to zero or
> not.  See e.g. 6b93fcd14 for an actual bug caught by that.

Sure.  But it also increases the test matrix, requires adding code for
fallbacks, makes dependencies more painful as evidenced here.

Greetings,

Andres Freund