Thread: HP-UX port

Hi everyone.

I work at the High Performance Systems division of Hewlett Packard, and
have been asked to evaluate postgresql for possible use in one of our
internal tools (software life cycle related).

I have encountered the following issues so far:

problems with configure:
    I have tried building on both HP K-Class and multi-node V-Class (2 node
    V2200 => 2 16 CPU boxes wired together to make 1 32 CPU machine).
    Unfortunately, configure does not recognize the uname output from these
    machines.
        # uname -a
        HP-UX v-a B.11.10 A 9000/800 71301 two-user license
    (yes its an unreleased OS.. I realize that :))

    I changed in configure.guess the following:
    # diff config.guess.orig config.guess
    371c371
    <           9000/6?? | 9000/7?? | 9000/80[24] | 9000/8?[13679] |
    9000/892 )
    ---
    >           9000/6?? | 9000/7?? | 9000/80[024] | 9000/8?[13679] |
    >           9000/892 )

    ANd that results in configure at least getting far enough so that it
    tells me that it needs a template.  I configured it with:
    ./configure --with-template=hpux_cc --with-CXX=aCC --with-perl

    Then ran gmake.

Next problem: lex appears to be broken on HPUX 11.10.  I got all sorts
    of errors when it runs "lex scan.l".  So I grabbed flex-2.5.4a and
    installed that.  This got past the scan.l part, but still spits out all
    sorts of errors on pl_gram.c:
    # gmake
    flex -i -l scan.l
    sed -e 's/yy/plpgsql_yy/g' -e 's/YY/PLPGSQL_YY/g' <lex.yy.c >pl_scan.c
    rm -f lex.yy.c
    cc -I../../../include -I../../../backend   -Wl,-E -Ae
    -I../../../interfaces/libpq -I../../../include -I../../../backend +z -c
    -o pl_parse.o pl_gram.c
    cc: "scan.l", line 85: error 1588: "K_ASSIGN" undefined.
    cc: "scan.l", line 86: error 1588: "K_ASSIGN" undefined.
    cc: "scan.l", line 87: error 1588: "K_DOTDOT" undefined.
    cc: "scan.l", line 88: error 1588: "K_ALIAS" undefined.
    cc: "scan.l", line 89: error 1588: "K_BEGIN" undefined.
    cc: "scan.l", line 90: error 1588: "T_BPCHAR" undefined.
    cc: "scan.l", line 91: error 1588: "T_CHAR" undefined.
    cc: "scan.l", line 92: error 1588: "K_CONSTANT" undefined.
    cc: "scan.l", line 93: error 1588: "K_DEBUG" undefined.
    cc: "scan.l", line 94: error 1588: "K_DECLARE" undefined.
    cc: "scan.l", line 95: error 1588: "K_DEFAULT" undefined.
    cc: "scan.l", line 96: error 1588: "K_ELSE" undefined.
    cc: "scan.l", line 97: error 1588: "K_END" undefined.
    cc: "scan.l", line 98: error 1588: "K_EXCEPTION" undefined.
    cc: "scan.l", line 99: error 1588: "K_EXIT" undefined.
    cc: "scan.l", line 100: error 1588: "K_FOR" undefined.
    cc: "scan.l", line 101: error 1588: "K_FROM" undefined.
    cc: "scan.l", line 102: error 1588: "K_IF" undefined.
    cc: "scan.l", line 103: error 1588: "K_IN" undefined.
    cc: "scan.l", line 104: error 1588: "K_INTO" undefined.
    cc: "scan.l", line 105: error 1588: "K_LOOP" undefined.
    cc: "scan.l", line 106: error 1588: "K_NOT" undefined.
    cc: "scan.l", line 107: error 1588: "K_NOTICE" undefined.
    cc: "scan.l", line 108: error 1588: "K_NULL" undefined.
    cc: "scan.l", line 109: error 1588: "K_PERFORM" undefined.
    cc: "scan.l", line 110: error 1588: "K_RAISE" undefined.
    cc: "scan.l", line 111: error 1588: "K_RECORD" undefined.
    cc: "scan.l", line 112: error 1588: "K_RENAME" undefined.
    cc: "scan.l", line 113: error 1588: "K_RETURN" undefined.
    cc: "scan.l", line 114: error 1588: "K_REVERSE" undefined.
    cc: "scan.l", line 115: error 1588: "K_SELECT" undefined.
    cc: "scan.l", line 116: error 1588: "K_THEN" undefined.
    cc: "scan.l", line 117: error 1588: "K_TO" undefined.
    cc: "scan.l", line 118: error 1588: "K_TYPE" undefined.
    cc: "scan.l", line 119: error 1588: "T_VARCHAR" undefined.
    cc: "scan.l", line 120: error 1588: "K_WHEN" undefined.
    cc: "scan.l", line 121: error 1588: "K_WHILE" undefined.
    cc: "scan.l", line 123: error 1588: "O_OPTION" undefined.
    cc: "scan.l", line 124: error 1588: "O_DUMP" undefined.
    cc: "scan.l", line 139: error 1588: "T_NUMBER" undefined.
    cc: "scan.l", line 174: error 1588: "T_STRING" undefined.
    gmake: *** [pl_parse.o] Error 1

    After looking at pl_gram.c, it includes pl_scan.c near the top of the
    file, but all of the items that are listed above as being undefined
    come later in the file.  I moved the #include "pl_sca.c" line down
    below all the #defines, and everything builds correctly.  I dont know a
    lot about lex/yacc, so I am not sure if this is a flex issue, or if
    there is something else at work here.

Next problem: testing.
    I ran the regression tests and the bench tests (WISC).  Some
    regression tests failed, but that is to be expected I supposed.
    However, when I ran the bench tests. only queries 1-15 run. After that,
    postgresql dies with a FATAL error due to a stuck spinlock:

    FATAL: s_lock(400c122c) at bufmgr.c:665, stuck spinlock. Aborting.

    If anyone wants more information on this bug let me know and I will see
    what I can do :).

Those are all of the problems I have had so far.  I have it up and running,
but it took a little tinkering.  Hopefully we can fix the configure issues
with different HP hardware platforms for the next release.   Im not sure,
but my guess is that configure will choke on the new HP N-Class machines as
well.  I dont have access to one to test it though :(.

Regards,
Mike

> Those are all of the problems I have had so far.  I have it up and running,
> but it took a little tinkering.  Hopefully we can fix the configure issues
> with different HP hardware platforms for the next release.   Im not sure,
> but my guess is that configure will choke on the new HP N-Class machines as
> well.  I dont have access to one to test it though :(.

We should be shipping the postgres source tar files with flex/bison
already run on the parsing files. It sounds like at least one slipped
through the cracks. Tom, is this up your alley?

As an aside, HPs are not known for their smooth compatibility between
releases, at least in my household. Any suggestions on general
solutions to build problems on new releases would be appreciated; I
recall that every release has required at least some tweaks to get
working again. I'm not sure that any of your particular problems are
in this category, but I thought I'd take the opportunity to whine
about it ;)

Thanks for the feedback.

                        - Thomas

--
Thomas Lockhart                lockhart@alumni.caltech.edu
South Pasadena, California

Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
> We should be shipping the postgres source tar files with flex/bison
> already run on the parsing files. It sounds like at least one slipped
> through the cracks. Tom, is this up your alley?

It looks to me like he tried to build with the system yacc/lex.

We already knew that HP's lex just Won't Work in HPUX 9 and 10
(doc/FAQ_HPUX points this out), and it seems HPUX 11 is no better.
No big surprise though, since a lot of vendor lexes seem to fail.

The compile failure in pl_gram.c looks to be a plain old portability
mistake in src/pl/plpgsql/src/gram.y: it includes pl_scan.c in the
grammar header section, which works OK in bison output because bison
emits the #define constants before copying the header.  I expect that
HP's yacc does it in some other order.  Jan, may I suggest moving the
#include "pl_scan.c" down to the file trailer?  Or compiling the lexer
as a separate source file, like everyone else does?

>     I changed in configure.guess the following:
>     # diff config.guess.orig config.guess
>     371c371
>     <           9000/6?? | 9000/7?? | 9000/80[24] | 9000/8?[13679] |
>     9000/892 )
>     ---
>     >           9000/6?? | 9000/7?? | 9000/80[024] | 9000/8?[13679] |
>     >           9000/892 )

I wonder why the GNU guys aren't just using 9000/8?? ... actually they
might be by now.  I think our copy of config.guess is already a few
revisions out of date.

>     I ran the regression tests and the bench tests (WISC).  Some
>     regression tests failed, but that is to be expected I supposed.

The "failures" enumerated in FAQ_HPUX are to be expected.  Did you
see any others?  (I can send you a copy of what I consider the normal
regression diffs on HPUX 9/10; it'd be interesting to see if 11 is any
different...)

>     However, when I ran the bench tests. only queries 1-15 run. After that,
>     postgresql dies with a FATAL error due to a stuck spinlock:
>
>     FATAL: s_lock(400c122c) at bufmgr.c:665, stuck spinlock. Aborting.
>
>     If anyone wants more information on this bug let me know and I will see
>     what I can do :).

This appears to be worth digging into.  I've been running Postgres for
quite some time on HPUX 9 and 10 and not seen any such problem.

BTW, are you building in a 32- or 64-bit environment?  We are gradually
flushing 64-bit-long portability bugs out of the code, but I am sure
some remain.  Check the pgsql mailing list archives for discussions
mentioning Alpha platforms to see what's been found before.

            regards, tom lane

> Thomas Lockhart <lockhart@alumni.caltech.edu> writes:
> > We should be shipping the postgres source tar files with flex/bison
> > already run on the parsing files. It sounds like at least one slipped
> > through the cracks. Tom, is this up your alley?
>
> It looks to me like he tried to build with the system yacc/lex.
>
> We already knew that HP's lex just Won't Work in HPUX 9 and 10
> (doc/FAQ_HPUX points this out), and it seems HPUX 11 is no better.
> No big surprise though, since a lot of vendor lexes seem to fail.

This is true.  I first tried with the vendor supplied lex, which failed.

> >     I ran the regression tests and the bench tests (WISC).  Some
> >     regression tests failed, but that is to be expected I supposed.
>
> The "failures" enumerated in FAQ_HPUX are to be expected.  Did you
> see any others?  (I can send you a copy of what I consider the normal
> regression diffs on HPUX 9/10; it'd be interesting to see if 11 is any
> different...)

I found the HPUX faq after I posted the message.  The only different
failure I saw was that the random regression failed.  All of the others are
listed int eh FAQ. Here are the failures I got:

# grep failed regress.out
int2 ..  failed
int4 ..  failed
float8 ..  failed
geometry ..  failed
horology ..  failed
random ..  failed

# diff results/random.out expected/random.out
22,23c22
<    123
< (1 row)
---
> (0 rows)

The expected part shows:
QUERY: SELECT random FROM RANDOM_TBL
  WHERE random NOT BETWEEN 80 AND 120;
random
------
(0 rows)

and the actual results are:
QUERY: SELECT random FROM RANDOM_TBL
  WHERE random NOT BETWEEN 80 AND 120;
random
------
   123
(1 row)


Which looks like a genuine bug to me :).

> >     However, when I ran the bench tests. only queries 1-15 run. After that,
> >     postgresql dies with a FATAL error due to a stuck spinlock:
> >
> >     FATAL: s_lock(400c122c) at bufmgr.c:665, stuck spinlock. Aborting.
> >
> >     If anyone wants more information on this bug let me know and I will see
> >     what I can do :).
>
> This appears to be worth digging into.  I've been running Postgres for
> quite some time on HPUX 9 and 10 and not seen any such problem.
>
> BTW, are you building in a 32- or 64-bit environment?  We are gradually
> flushing 64-bit-long portability bugs out of the code, but I am sure
> some remain.  Check the pgsql mailing list archives for discussions
> mentioning Alpha platforms to see what's been found before.

I am on a V2250 machine which is a 16 cpu PA-RISC 2.0 environment. Which is
a pure 64 bit environment.  HP-UX will no longer support 32 bit environments
beginning with 11.01.  Interestingly, this bug is not 100% repeatable.
When I run the test, about half otf the time the bug happens.  The other
half, the test runs to completion.

Ill scan the alpha discussions and see if I can dig into this over the next
few days :).  Maybe this is an excuse to try to get my hands dirty with the
source ;).

Regards,
Mike

Michael J Schout <mschout@mail.gkg-com.com> writes:
> The expected part shows:
> QUERY: SELECT random FROM RANDOM_TBL
>   WHERE random NOT BETWEEN 80 AND 120;
> random
> ------
> (0 rows)

> and the actual results are:
> QUERY: SELECT random FROM RANDOM_TBL
>   WHERE random NOT BETWEEN 80 AND 120;
> random
> ------
>    123
> (1 row)

> Which looks like a genuine bug to me :).

No, that's expected behavior, actually.  The random test wouldn't be
random if it didn't fail occasionally :-).  The parameters are set so
that the "expected" results are produced most of the time.  If you get
a non-expected result from random more than about one time in five,
then it's worth investigating... but if you *never* got an unexpected
result, it would suggest the random function wasn't working...

> I am on a V2250 machine which is a 16 cpu PA-RISC 2.0 environment. Which is
> a pure 64 bit environment.  HP-UX will no longer support 32 bit environments
> beginning with 11.01.  Interestingly, this bug is not 100% repeatable.
> When I run the test, about half otf the time the bug happens.  The other
> half, the test runs to completion.

Interesting.  May or may not be a 64-bit issue; it might be a problem
with the machine-dependent spinlock code.  Check out
src/include/storage/s_lock.h and src/backend/storage/buffer/s_lock.c.

            regards, tom lane

> Michael J Schout <mschout@mail.gkg-com.com> writes:
> > and the actual results are:
> > QUERY: SELECT random FROM RANDOM_TBL
> >   WHERE random NOT BETWEEN 80 AND 120;
> > random
> > ------
> >    123
> > (1 row)
>
> > Which looks like a genuine bug to me :).
>
> No, that's expected behavior, actually.  The random test wouldn't be
> random if it didn't fail occasionally :-).  The parameters are set so

I agree :).  When I qlanced at it the first time, I missed the "NOT" in the
query, so I didnt think that 123 should have been returned at all, which is
obviously false :).  It looks like the regressions are okay then, which is
a relief :).

...
> Interesting.  May or may not be a 64-bit issue; it might be a problem
> with the machine-dependent spinlock code.  Check out
> src/include/storage/s_lock.h and src/backend/storage/buffer/s_lock.c.

Ill hopefully get some time next week to look at this.  I'll let you know
if I can figure it out :)

Mike