Thread: Cannot Restart PostgreSQL-8.1.4

    I had a problem with SQL-Ledger running on the local httpd that traced
back to some crufty old libpg.so* from 2003 and 2004 in /usr/local/lib. I
removed those (saved them, actually), ran ldconfig, then restarted both
httpd and postgresql. Unfortunately, the latter really has not started
despite indicating on the console that it has.

    Postgres-8.1.4 installed.

    Here're the libaries in /usr/lib/:

[rshepard@salmo ~]$ ll /usr/lib/libpq*
-rw-r--r-- 1 root root 149728 2006-05-24 15:06 /usr/lib/libpq.a
lrwxrwxrwx 1 root root     12 2006-07-06 17:19 /usr/lib/libpq.so ->
libpq.so.4.1*
lrwxrwxrwx 1 root root     12 2006-01-27 10:22 /usr/lib/libpq.so.3 ->
libpq.so.3.1*
-rwxr-xr-x 1 root root 110586 2006-01-26 09:49 /usr/lib/libpq.so.3.1*
lrwxrwxrwx 1 root root     12 2006-07-06 17:19 /usr/lib/libpq.so.4 ->
libpq.so.4.1*
-rwxr-xr-x 1 root root 111532 2006-05-24 15:06 /usr/lib/libpq.so.4.1*

    And, in case postgres is looking in /usr/local/lib/ it has:

[rshepard@salmo ~]$ ll /usr/local/lib/libpq*
lrwxrwxrwx 1 root root 21 2007-01-24 10:38 /usr/local/lib/libpq.so ->
/usr/lib/libpq.so.4.1*
lrwxrwxrwx 1 root root 21 2007-01-24 10:35 /usr/local/lib/libpq.so.3 ->
/usr/lib/libpq.so.3.1*
lrwxrwxrwx 1 root root 21 2007-01-24 10:38 /usr/local/lib/libpq.so.4 ->
/usr/lib/libpq.so.4.1*

    When I run '/etc/rc.d/rc.postgresql start' it returns 'Starting
PostgreSQL: ok', but '/etc/rc.d/rc.postgresql status' returns 'pg_ctl:
neither postmaster nor postgres running' which is true.

    I don't know what I did to break the installation, but I would greatly
appreciate help getting it running again ASAP.

TIA,

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Rich Shepard <rshepard@appl-ecosys.com> writes:
>     When I run '/etc/rc.d/rc.postgresql start' it returns 'Starting
> PostgreSQL: ok', but '/etc/rc.d/rc.postgresql status' returns 'pg_ctl:
> neither postmaster nor postgres running' which is true.

Have you looked in the postmaster log?  Under almost all circumstances,
a failing postmaster should emit some sort of bleat on stderr, or
perhaps to syslog if you've configured it that way (and it managed to
get as far as absorbing the contents of postgresql.conf).

Some PG start scripts redirect the postmaster's stderr to /dev/null;
if yours does, change it, at least till you've resolved the problem.

            regards, tom lane

On Wed, 24 Jan 2007, Tom Lane wrote:

> Have you looked in the postmaster log?

Tom,

   I went looking for it before writing, but did not find it. Now I have.
The ownership and permissions of /var/lib/pgsql and /var/lib/postgresql were
FUBAR.

   I changed the permissions and it's now running.

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

On Wed, 24 Jan 2007, Tom Lane wrote:

> Have you looked in the postmaster log?

Tom, et al.:

   I got postmaster running now, but have been blocked by Bug #2701 when I
try to run SQL-Ledger. Google has 1,830 hits for it, so I'm far from the
first. :-)

   I'm running -8.1.4 here. In /usr/lib/ I have:

lrwxrwxrwx 1 root root     12 2006-07-06 17:19 libpq.so -> libpq.so.4.1*
lrwxrwxrwx 1 root root     12 2006-01-27 10:22 libpq.so.3 -> libpq.so.3.1*
-rwxr-xr-x 1 root root 110586 2006-01-26 09:49 libpq.so.3.1*
lrwxrwxrwx 1 root root     12 2006-07-06 17:19 libpq.so.4 -> libpq.so.4.1*
-rwxr-xr-x 1 root root 111532 2006-05-24 15:06 libpq.so.4.1*

   What version of libpq.so do I need?

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

On Wed, Jan 24, 2007 at 12:02:30PM -0800, Rich Shepard wrote:
>   I got postmaster running now, but have been blocked by Bug #2701 when I
> try to run SQL-Ledger. Google has 1,830 hits for it, so I'm far from the
> first. :-)
>
>   I'm running -8.1.4 here. In /usr/lib/ I have:

<snip>

>   What version of libpq.so do I need?

The server doesn't care, what matter is what version the *client* (in
this case SQL-Ledger) expects. ldd should tell you that.

(It's ok to have multiple versions of the client library installed...)

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Wed, 24 Jan 2007, Martijn van Oosterhout wrote:

>>   What version of libpq.so do I need?
>
> The server doesn't care, what matter is what version the *client* (in this
> case SQL-Ledger) expects. ldd should tell you that.

Martijn,

   OK. I'll go back there (which is where I started this journey). I don't
know how to run ldd on a perl script.

> (It's ok to have multiple versions of the client library installed...)

   Sure. I do for many libraries.

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

On Wednesday 24 January 2007 13:08, Rich Shepard
<rshepard@appl-ecosys.com> wrote:
>    OK. I'll go back there (which is where I started this journey). I
> don't know how to run ldd on a perl script.

It'll be whatever version the DBD::Pg module is compiled to use.  Just
rebuilding and installing an updated version of that package will
likely fix your problem.

--
"Pulling together is the aim of despotism and tyranny. Free men pull in
all kinds of directions." -- Terry Pratchett

On Wed, Jan 24, 2007 at 01:08:04PM -0800, Rich Shepard wrote:
> >The server doesn't care, what matter is what version the *client* (in this
> >case SQL-Ledger) expects. ldd should tell you that.
>
> Martijn,
>
>   OK. I'll go back there (which is where I started this journey). I don't
> know how to run ldd on a perl script.

Method 1: Find the .so module that perl uses (try locate Pg.so and ldd
that).

Method 2: strace -e open <program>

And look at which version it tries to open.

How exactly is it failing?

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

On Wed, 24 Jan 2007, Martijn van Oosterhout wrote:

> Method 1: Find the .so module that perl uses (try locate Pg.so and ldd
> that).

Martijn,

   Thank you for teaching me a new trick today. The results of method 1 are:

     /lib/libsafe.so.2 (0xb7f76000)
         linux-gate.so.1 =>  (0xffffe000)
         libpq.so.3 => /usr/lib/libpq.so.3 (0xb7f5f000)
         libc.so.6 => /lib/tls/libc.so.6 (0xb7e1a000)
         libdl.so.2 => /lib/tls/libdl.so.2 (0xb7e16000)
         libcrypt.so.1 => /lib/tls/libcrypt.so.1 (0xb7de8000)
         libresolv.so.2 => /lib/tls/libresolv.so.2 (0xb7dd4000)
         libnsl.so.1 => /lib/tls/libnsl.so.1 (0xb7dbd000)
         /lib/ld-linux.so.2 (0x80000000)

So it's finding all the libraries it wants, and it's using the older
libpq.so. Interesting.

> Method 2: strace -e open <program>

   It's a perl script -- actually, a large family of scripts -- that act as
middleware between a web browser and an httpd. I don't see anything in the
login.pl output that looks suspicious.

> How exactly is it failing?

   I open a new firefox tab and open http://localhost/sql-ledger/login.pl.
After entering my username and password and pressing [Enter] I get an
internal server error:

The server encountered an internal error or misconfiguration and was unable
to complete your request.

Please contact the server administrator, rshepard@appl-ecosys.com and inform
them of the time the error occurred, and anything you might have done that
may have caused the error.

More information about this error may be available in the server error log.

   The log file (/var/log/apache/error_log) shows:

[Wed Jan 24 15:01:20 2007] [error] [client 127.0.0.1] Premature end of
script headers: /usr/local/sql-ledger/login.pl
/usr/local/sql-ledger/login.pl: symbol lookup error:
/usr/lib/perl5/site_perl/5.8.8/i486-linux/auto/DBD/Pg/Pg.so: undefined symbol: PQserverVersion

   The version of DBD::Pg is 1.4.9 which is the most recent one available.

Thanks,

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

On 1/24/07, Rich Shepard <rshepard@appl-ecosys.com> wrote:
The reason is when DBD::Pg probed the PostgreSQL configuration using pg_config --version it received a response of at least 8.0.0 or later.  Yet when it attempted to link it could only find libpq.so.3.  I would try rebuilding DBD::Pg, and run "make test" to verify whether it works before installing.

I don't know what the rules are for ld to locate libraries and link them, but perhaps if ldconfig was not run and libpq.so.4 didn't link to the right version it used libpq.so.3.

--
Chad
http://www.postgresqlforums.com/

On Wed, 24 Jan 2007, Alan Hodgson wrote:

> It'll be whatever version the DBD::Pg module is compiled to use.  Just
> rebuilding and installing an updated version of that package will likely
> fix your problem.

Alan,

   I'm not following you. DBD::Pg is the most current version (1.49 from May
2006). That I built and installed this morning. It's calling
/usr/lib/libpg.so.3.1, but I also have /usr/lib/ligpg.so.4.1. The perl
script, login.pl, hasn't changed in a couple of years.

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863

Rich Shepard <rshepard@appl-ecosys.com> writes:
>    I'm not following you. DBD::Pg is the most current version (1.49 from May
> 2006). That I built and installed this morning. It's calling
> /usr/lib/libpg.so.3.1, but I also have /usr/lib/ligpg.so.4.1.

Somehow DBD::Pg picked up the wrong (older) version of libpq.so;
you need to look into why.  Wrong search path while building maybe?

            regards, tom lane

On Wed, 24 Jan 2007, Tom Lane wrote:

> Somehow DBD::Pg picked up the wrong (older) version of libpq.so; you need
> to look into why.  Wrong search path while building maybe?

Tom,

   I _thought_ that might be what happened, but I wanted to make sure before
I dug myself into a hole.

   You are absolutely correct. I'll bet it looked first at /usr/local/lib and
found the really old versions of libpq.so. I had cleaned those out, but
after making and installing Pg.so.

   I looked through the DBD::Pg source files and the paths looked OK to my
naive eyes. So I just re-ran the 'perl Makefile.PL, make, make test, and
make install' series. All tests passed, and SQL-Ledger now comes up. Whew!!

   I'll bet those old libraries were left over from Red Hat 7.3; it was in
September of 2003 that I migrated to Slackware.

   You have my sincere thanks for clearing up a mess that occupied me since
yesterday afternoon.

Rich

--
Richard B. Shepard, Ph.D.               |    The Environmental Permitting
Applied Ecosystem Services, Inc.        |          Accelerator(TM)
<http://www.appl-ecosys.com>     Voice: 503-667-4517      Fax: 503-667-8863