Thread: Broken postgres links need to find callers

Broken postgres links need to find callers

From
Rich Shepard
Date:
   I managed to mess up postgresql-10.3 on this Slackware-14.2 desktop
server/workstation. It worked OK until I tried adding access to an another
application.

   For a reason I don't know, adding that listening address revealed that
many sym links are looking for 10.2 directories. I've found and fixed many
of these and need help finding the rest (perhaps only one more).

   Running pg_ctl start fails:
$ pg_ctl start -D /var/lib/pgsql/10.3/data/
waiting for server to start....2018-10-31 10:02:01.312 PDT [1285] FATAL:
could not open directory "/usr/share/postgresql-10.2/timezonesets": No such
file or directory 2018-10-31 10:02:01.312 PDT [1285] HINT: This may indicate
an incomplete PostgreSQL installation, or that the file
"/usr/lib/postgresql/10.3/bin/postgres" has been moved away from its proper
location.
  stopped waiting

   What file is looking for the timezonesets directory? It's using a symlink
but the error message is not telling me where it is so I can remove it and
replace it with a symlink to the proper postgresql-10.3/timezonesets
directory.

TIA,

Rich



Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 10:18 AM, Rich Shepard wrote:
>    I managed to mess up postgresql-10.3 on this Slackware-14.2 desktop
> server/workstation. It worked OK until I tried adding access to an another
> application.
> 
>    For a reason I don't know, adding that listening address revealed that
> many sym links are looking for 10.2 directories. I've found and fixed many
> of these and need help finding the rest (perhaps only one more).

What was the listening address you added?

What happens if you remove the listening address?

Did you recently upgrade from 10.2 --> 10.3?

> 
>    Running pg_ctl start fails:
> $ pg_ctl start -D /var/lib/pgsql/10.3/data/
> waiting for server to start....2018-10-31 10:02:01.312 PDT [1285] FATAL:
> could not open directory "/usr/share/postgresql-10.2/timezonesets": No such
> file or directory 2018-10-31 10:02:01.312 PDT [1285] HINT: This may 
> indicate
> an incomplete PostgreSQL installation, or that the file
> "/usr/lib/postgresql/10.3/bin/postgres" has been moved away from its proper
> location.
>   stopped waiting
> 
>    What file is looking for the timezonesets directory? It's using a 
> symlink
> but the error message is not telling me where it is so I can remove it and
> replace it with a symlink to the proper postgresql-10.3/timezonesets
> directory.
> 
> TIA,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> What was the listening address you added?

Adrian,

   I added the host name.

> What happens if you remove the listening address?

   I don't think this makes a difference. pg_ctl is calling a program that
looks for timezonesets in the wrong directory

> Did you recently upgrade from 10.2 --> 10.3?

   On March 1st of this year.

Regards,

Rich


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 10:55 AM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> What was the listening address you added?
> 
> Adrian,
> 
>    I added the host name.
> 
>> What happens if you remove the listening address?
> 
>    I don't think this makes a difference. pg_ctl is calling a program that
> looks for timezonesets in the wrong directory

You said it made a difference when you added it, just trying to figure 
out if removing it also makes a difference. If not then we need to look 
elsewhere for an explanation.

> 
>> Did you recently upgrade from 10.2 --> 10.3?
> 
>    On March 1st of this year.
> 
> Regards,
> 
> Rich
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> You said it made a difference when you added it, just trying to figure out if 
> removing it also makes a difference. If not then we need to look elsewhere 
> for an explanation.

Adrian,

   Each time I hit a broken symlink pg_ctl told me which link was broken. For
example, the libpgtypes.so* files were sought in
/usr/lib/postgresql/10.2/lib/ when they're actually now in
/usr/lib/postgresql/10.3/lib/. Removing broken links and creating new,
valid ones fixes them.

   So, I'm seeking the library or executable file that's finding timezonesets
in the no-longer existing ../10.2/ directory.

Regards,

Rich



Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 11:15 AM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> You said it made a difference when you added it, just trying to figure 
>> out if removing it also makes a difference. If not then we need to 
>> look elsewhere for an explanation.
> 
> Adrian,
> 
>    Each time I hit a broken symlink pg_ctl told me which link was 
> broken. For
> example, the libpgtypes.so* files were sought in
> /usr/lib/postgresql/10.2/lib/ when they're actually now in
> /usr/lib/postgresql/10.3/lib/. Removing broken links and creating new,
> valid ones fixes them.
> 
>    So, I'm seeking the library or executable file that's finding 
> timezonesets
> in the no-longer existing ../10.2/ directory.
> 

If you refuse to implement the suggestions I asked for then I cannot 
help you, as you are now off on a different tangent. One that on the 
face of it is dangerous.

> Regards,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> If you refuse to implement the suggestions I asked for then I cannot help 
> you, as you are now off on a different tangent. One that on the face of it is 
> dangerous.

   In var/lib/pgsql/10.3/data/postgresql.conf:

# - Connection Settings -

_addresses = 'localhost'

   No listener_addresses present.

$ pg_ctl start -D /var/lib/pgsql/10.3/data/
waiting for server to start....2018-10-31 18:33:03.323 GMT [3352] LOG:  unrecognized configuration parameter
"_addresses"in file "/var/lib/pgsql/10.3/data/postgresql.conf" line 59
 
2018-10-31 18:33:03.323 GMT [3352] FATAL:  configuration file "/var/lib/pgsql/10.3/data/postgresql.conf" contains
errors
  stopped waiting
pg_ctl: could not start server

Rich



Re: Broken postgres links need to find callers

From
Andrew Gierth
Date:
>>>>> "Rich" == Rich Shepard <rshepard@appl-ecosys.com> writes:

 Rich>   I managed to mess up postgresql-10.3 on this Slackware-14.2
 Rich> desktop server/workstation. It worked OK until I tried adding
 Rich> access to an another application.

 Rich> waiting for server to start....2018-10-31 10:02:01.312 PDT [1285] FATAL:
 Rich> could not open directory "/usr/share/postgresql-10.2/timezonesets": No such
 Rich> file or directory 2018-10-31 10:02:01.312 PDT [1285] HINT: This may indicate
 Rich> an incomplete PostgreSQL installation, or that the file
 Rich> "/usr/lib/postgresql/10.3/bin/postgres" has been moved away from its proper
 Rich> location.

Is there a pg_config binary in /usr/lib/postgresql/10.3/bin/ and if so,
what is the output of  /usr/lib/postgresql/10.3/bin/pg_config --sharedir

Also what is the output of  /usr/lib/postgresql/10.3/bin/postgres -V

The most plausible explanation I can see for what you're seeing there is
that what you have as /usr/lib/postgresql/10.3/bin/postgres is not
actually the 10.3 binary but rather the 10.2 one. There should be no
symlinks involved there - the path that is reported in the error message
is the one that the postgres binary actually did try to open.

-- 
Andrew (irc:RhodiumToad)


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Andrew Gierth wrote:

> Is there a pg_config binary in /usr/lib/postgresql/10.3/bin/ and if so,
> what is the output of /usr/lib/postgresql/10.3/bin/pg_config --sharedir

Andrew,

   Yes, pg_config is present but pointing to the wrong directory:
# /usr/lib/postgresql/10.3/bin/pg_config --sharedir
/usr/share/postgresql-10.2

   However, the file dates are that of the upgrade from 10.2 to 10.3:

-rwxr-xr-x 1 root root 7096448 Mar  1  2018 postgres*
lrwxrwxrwx 1 root root       8 Mar  1  2018 postmaster -> postgres*
-rwxr-xr-x 1 root root  514732 Mar  1  2018 psql*

and the postgres version is 10.3:

> Also what is the output of  /usr/lib/postgresql/10.3/bin/postgres -V

# /usr/lib/postgresql/10.3/bin/postgres -V
postgres (PostgreSQL) 10.3

> The most plausible explanation I can see for what you're seeing there is
> that what you have as /usr/lib/postgresql/10.3/bin/postgres is not
> actually the 10.3 binary but rather the 10.2 one. There should be no
> symlinks involved there - the path that is reported in the error message
> is the one that the postgres binary actually did try to open.

   Can pg_config be corrected independent of anything else. That seems to be
where the blockage is found.

Thanks,

Rich


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 11:33 AM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> If you refuse to implement the suggestions I asked for then I cannot 
>> help you, as you are now off on a different tangent. One that on the 
>> face of it is dangerous.
> 
>    In var/lib/pgsql/10.3/data/postgresql.conf:
> 
> # - Connection Settings -
> 
> _addresses = 'localhost'
> 
>    No listener_addresses present.

As the below shows that is not a valid setting.

Try:

listen_addresses = ''

> 
> $ pg_ctl start -D /var/lib/pgsql/10.3/data/
> waiting for server to start....2018-10-31 18:33:03.323 GMT [3352] LOG:  
> unrecognized configuration parameter "_addresses" in file 
> "/var/lib/pgsql/10.3/data/postgresql.conf" line 59
> 2018-10-31 18:33:03.323 GMT [3352] FATAL:  configuration file 
> "/var/lib/pgsql/10.3/data/postgresql.conf" contains errors
>   stopped waiting
> pg_ctl: could not start server
> 
> Rich
> 
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 11:48 AM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Andrew Gierth wrote:
> 
>> Is there a pg_config binary in /usr/lib/postgresql/10.3/bin/ and if so,
>> what is the output of /usr/lib/postgresql/10.3/bin/pg_config --sharedir
> 
> Andrew,
> 
>    Yes, pg_config is present but pointing to the wrong directory:
> # /usr/lib/postgresql/10.3/bin/pg_config --sharedir
> /usr/share/postgresql-10.2

What does:

/usr/lib/postgresql/10.3/bin/pg_config --version

show?

What does:

ps ax | grep post

show?


> 
>    However, the file dates are that of the upgrade from 10.2 to 10.3:
> 
> -rwxr-xr-x 1 root root 7096448 Mar  1  2018 postgres*
> lrwxrwxrwx 1 root root       8 Mar  1  2018 postmaster -> postgres*
> -rwxr-xr-x 1 root root  514732 Mar  1  2018 psql*
> 
> and the postgres version is 10.3:
> 
>> Also what is the output of  /usr/lib/postgresql/10.3/bin/postgres -V
> 
> # /usr/lib/postgresql/10.3/bin/postgres -V
> postgres (PostgreSQL) 10.3
> 
>> The most plausible explanation I can see for what you're seeing there is
>> that what you have as /usr/lib/postgresql/10.3/bin/postgres is not
>> actually the 10.3 binary but rather the 10.2 one. There should be no
>> symlinks involved there - the path that is reported in the error message
>> is the one that the postgres binary actually did try to open.
> 
>    Can pg_config be corrected independent of anything else. That seems 
> to be
> where the blockage is found.
> 
> Thanks,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> listen_addresses = ''

Adrian,

#listen_addresses = ''

$ pg_ctl start -D /var/lib/pgsql/10.3/data/
waiting for server to start....2018-10-31 12:12:39.530 PDT [4398] FATAL:  could not open directory
"/usr/share/postgresql-10.2/timezonesets":No such file or directory
 
2018-10-31 12:12:39.530 PDT [4398] HINT:  This may indicate an incomplete PostgreSQL installation, or that the file
"/usr/lib/postgresql/10.3/bin/postgres"has been moved away from its proper location.
 
  stopped waiting
pg_ctl: could not start server

listen_addresses = ''

$ pg_ctl start -D /var/lib/pgsql/10.3/data/
waiting for server to start....2018-10-31 12:13:28.141 PDT [4413] FATAL:  could not open directory
"/usr/share/postgresql-10.2/timezonesets":No such file or directory
 
2018-10-31 12:13:28.141 PDT [4413] HINT:  This may indicate an incomplete PostgreSQL installation, or that the file
"/usr/lib/postgresql/10.3/bin/postgres"has been moved away from its proper location.
 
  stopped waiting
pg_ctl: could not start server

Thanks,

Rich


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> What does:
>
> /usr/lib/postgresql/10.3/bin/pg_config --version
>
> show?

# /usr/lib/postgresql/10.3/bin/pg_config --version
PostgreSQL 10.3

> What does:
>
> ps ax | grep post
>
> show?

# ps ax | grep post
  1307 ?        Ss     1:29 /usr/libexec/postfix/master -w
  4445 ?        S      0:00 spawn -n policy -t unix user=nobody argv=/usr/bin/perl
/usr/lib/postfix/postfix-policyd-spf-perl
  4446 ?        Ss     0:00 /usr/bin/perl /usr/lib/postfix/postfix-policyd-spf-perl
  4456 ?        S      0:00 spawn -n policy -t unix user=nobody argv=/usr/bin/perl
/usr/lib/postfix/postfix-policyd-spf-perl
  4457 ?        Ss     0:00 /usr/bin/perl /usr/lib/postfix/postfix-policyd-spf-perl
  4483 pts/2    S+     0:00 grep post

Rich


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 12:14 PM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> listen_addresses = ''
> 
> Adrian,
> 
> #listen_addresses = ''
> 
> $ pg_ctl start -D /var/lib/pgsql/10.3/data/
> waiting for server to start....2018-10-31 12:12:39.530 PDT [4398] 
> FATAL:  could not open directory 
> "/usr/share/postgresql-10.2/timezonesets": No such file or directory
> 2018-10-31 12:12:39.530 PDT [4398] HINT:  This may indicate an 
> incomplete PostgreSQL installation, or that the file 
> "/usr/lib/postgresql/10.3/bin/postgres" has been moved away from its 
> proper location.
>   stopped waiting
> pg_ctl: could not start server

What does:

pg_ctl --version

show?

> 
> listen_addresses = ''
> 
> $ pg_ctl start -D /var/lib/pgsql/10.3/data/
> waiting for server to start....2018-10-31 12:13:28.141 PDT [4413] 
> FATAL:  could not open directory 
> "/usr/share/postgresql-10.2/timezonesets": No such file or directory
> 2018-10-31 12:13:28.141 PDT [4413] HINT:  This may indicate an 
> incomplete PostgreSQL installation, or that the file 
> "/usr/lib/postgresql/10.3/bin/postgres" has been moved away from its 
> proper location.
>   stopped waiting
> pg_ctl: could not start server


So it seems listen_addresses is not the issue at this point.

So when you added the new application did you make any other changes?

At this point you need to get back to two discreet Postgres installs 
10.2 and 10.3.

1) Remove all the symlinks you made.

2) In the 10.3/ verify that the programs in bin/ are actually the 10.3 
ones using prgm_name --version.

3) Try pg_ctl start.



> 
> Thanks,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> What does:
> pg_ctl --version
> show?

# pg_ctl --version
pg_ctl (PostgreSQL) 10.3

> So when you added the new application did you make any other changes?

   I did not add another application; grass has been installed here for
decades. Because I could not connect to the postgres database for a spatial
project it was suggested that I expand the listen_addresses to include the
server name, too.

> At this point you need to get back to two discreet Postgres installs 10.2 and 
> 10.3.

    I did not have two distinct installations of postgres, only the 10.3
version. It was some of the directories labeled 10.2/ that seem to be the
issue.

    When I run pg_config --configure this is the result:
# ./pg_config --configure
'--prefix=/usr/lib/postgresql/10.2' '--sysconfdir=/etc/postgresql/10.2'
'--includedir=/usr/include' '--datarootdir=/usr/share' '--mandir=/usr/man'
'--docdir=/usr/doc/postgresql-10.3' '--datadir=/usr/share/postgresql-10.2'
'--with-openssl' '--with-tcl' '--with-perl' '--with-python' '--with-libxml'
'--with-libxslt' '--enable-thread-safety'
'--with-system-tzdata=/usr/share/zoneinfo' '--disable-nls'
'--build=i586-slackware-linux' 'build_alias=i586-slackware-linux'
'CFLAGS=-O2 -march=i586 -mtune=i686' 'PKG_CONFIG_PATH=/usr/lib/pkgconfig

   Notice that the prefix, sysconfdir, and datadir specify 10.2. But, the
build script shows the version as 10.3 and uses that to configure the build
(entire script will be provided on request):

PRGNAM=postgresql
VERSION=${VERSION:-10.3}
BUILD=${BUILD:-1}
TAG=${TAG:-_SBo}

PG_VERSION=${PG_VERSION:-10.3}
PG_PORT=${PG_PORT:-5432}
PG_UID=${PG_UID:-209}
PG_GID=${PG_GID:-209}
...
./configure \
   --prefix=/usr/lib${LIBDIRSUFFIX}/$PRGNAM/$PG_VERSION \
   --sysconfdir=/etc/$PRGNAM/$PG_VERSION \
   --includedir=/usr/include \
   --datarootdir=/usr/share \
   --mandir=/usr/man \
   --docdir=/usr/doc/$PRGNAM-$VERSION \
   --datadir=/usr/share/$PRGNAM-$PG_VERSION \

   The build script sets the prefix, sysconfdir, and datadir to $PG_VERSION
which is defined as 10.3 so why 10.2 shows up in pg_config makes no sense to
me. And why psql worked without issue from the upgrade date of March 1 to
today with this inconsistency also puzzles me.

   If it matters, there's no /etc/postgresql/ and none in the backups since
the beginning of August.

Regards,

Rich


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 1:09 PM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> What does:
>> pg_ctl --version
>> show?
> 
> # pg_ctl --version
> pg_ctl (PostgreSQL) 10.3
> 
>> So when you added the new application did you make any other changes?
> 
>    I did not add another application; grass has been installed here for
> decades. Because I could not connect to the postgres database for a spatial
> project it was suggested that I expand the listen_addresses to include the
> server name, too.
> 
>> At this point you need to get back to two discreet Postgres installs 
>> 10.2 and 10.3.
> 
>     I did not have two distinct installations of postgres, only the 10.3
> version. It was some of the directories labeled 10.2/ that seem to be the
> issue.

Are there actually 10.2/ directories or is that just what you are seeing 
in the error messages and the pg_config output?

> 
>     When I run pg_config --configure this is the result:

Previously you used:

/usr/lib/postgresql/10.3/bin/pg_config

is that the same as below?:

> # ./pg_config --configure

In other words what does:

./pg_config --version

show?


> me. And why psql worked without issue from the upgrade date of March 1 to
> today with this inconsistency also puzzles me.

Did the server been running continuously from the upgrade to the time 
you made the listen_addresses change?

> 
>    If it matters, there's no /etc/postgresql/ and none in the backups since
> the beginning of August.
> 
> Regards,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> Are there actually 10.2/ directories or is that just what you are seeing in 
> the error messages and the pg_config output?

Adrian,

   No 10.2/ directories, only what is shown in the error messages and
pg_config output.

> Previously you used:
> /usr/lib/postgresql/10.3/bin/pg_config
> is that the same as below?:
>> # ./pg_config --configure

   Yes. I was in that directory when I ran pg_config.[1]

> In other words what does:
> ./pg_config --version
> show?

   This shows 10.3

> Did the server been running continuously from the upgrade to the time you
> made the listen_addresses change?

   Yes, other than a few kernel upgrades.

[1] This prompted me to look for more pg_config files, and I found a symlink
in /usr/bin/ that pointed to /usr/lib/postgresql/10.2/bin/pg_config which
does not exist. I changed that symlink to point to the 10.3/ pg_config
version but there's still a broken link somewhere because pg_ctl start
cannot find the correct directory for timezonesets.

Regards,

Rich


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Rich Shepard wrote:

> [1] This prompted me to look for more pg_config files, and I found a symlink
> in /usr/bin/ that pointed to /usr/lib/postgresql/10.2/bin/pg_config which
> does not exist. I changed that symlink to point to the 10.3/ pg_config
> version but there's still a broken link somewhere because pg_ctl start
> cannot find the correct directory for timezonesets.

   Thanks for pointing me to the links in /usr/bin/, Adrian. All of them are
pointing to the mythical 10.2/ directory. I'll fix those links and report
the results of running pg_ctl start.

Regards,

Rich


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Rich Shepard wrote:

> I'll fix those links and report the results of running pg_ctl start.

   Still bad links remaining. Some of those symlinks in /usr/bin/ dated back
to versons 9.4 and 9.6. Why they were not removed during upgrades remains a
mystery.

Rich


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Rich Shepard wrote:

> Still bad links remaining.

   Every pg_* not in /usr/lib/postgresql/10.3/bin/ now points to its namesake
there.

   Question: if pg_dump, pg_dumpall, pg_restore, pg_ctl, and pg_controldata
have symlinks in /usr/bin/ do they also need symlinks in /bin/? The ones
found there dated back to 2010 and I don't know that they're needed now.

   pg_ctl start still cannot find the proper timezonesets/ directory. If you
have suggestions what pg_ctl is calling that looks in the non-existent 10.2/
directory instead of the existing 10.3/ directory please let me know.

Rich


Re: Broken postgres links need to find callers

From
Andrew Gierth
Date:
>>>>> "Rich" == Rich Shepard <rshepard@appl-ecosys.com> writes:

 Rich>   Yes, pg_config is present but pointing to the wrong directory:
 Rich> # /usr/lib/postgresql/10.3/bin/pg_config --sharedir
 Rich> /usr/share/postgresql-10.2

What this says is that you somehow have a pg 10.3 binary which has been
compiled with ./configure --datadir=/usr/share/postgresql-10.2

which seems, to say the least, somewhat odd.

pg_config isn't used by the postgres binary to find paths, so "fixing"
it wouldn't help. The same paths that were compiled into pg_config are
compiled into the postgres binary, and pg_config and postgres contain
the same relocation logic.

-- 
Andrew (irc:RhodiumToad)


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 2:01 PM, Rich Shepard wrote:

>> Did the server been running continuously from the upgrade to the time you
>> made the listen_addresses change?
> 
>    Yes, other than a few kernel upgrades.

So no, as I presume you rebooted on the kernel upgrade which caused the 
Postgres server to stop/start.

> 
> [1] This prompted me to look for more pg_config files, and I found a 
> symlink
> in /usr/bin/ that pointed to /usr/lib/postgresql/10.2/bin/pg_config which
> does not exist. I changed that symlink to point to the 10.3/ pg_config
> version but there's still a broken link somewhere because pg_ctl start
> cannot find the correct directory for timezonesets.
> 
> Regards,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> So no, as I presume you rebooted on the kernel upgrade which caused the 
> Postgres server to stop/start.

   True. It stopped for the time it took the server to reboot.

Rich


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Andrew Gierth wrote:

> What this says is that you somehow have a pg 10.3 binary which has been
> compiled with ./configure --datadir=/usr/share/postgresql-10.2
>
> which seems, to say the least, somewhat odd.

Andrew,

   Quite odd rather than somewhat odd because the configure options in the
build script point to version 10.3, not 10.2.

> pg_config isn't used by the postgres binary to find paths, so "fixing" it
> wouldn't help. The same paths that were compiled into pg_config are
> compiled into the postgres binary, and pg_config and postgres contain the
> same relocation logic.

   Okay.

   I'll check with the SlackBuilds.org postgresql package maintainer and
confirm that re-building and re-installing 10.3 will not adversely affect
the data/ directory. Then I'll re-build and re-install it.

   Why it worked flawlessly until today when I modified the postgresql.conf
file to add access by another host name and then broke is also quite odd.

Thanks,

Rich


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 2:40 PM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Rich Shepard wrote:
> 
>> Still bad links remaining.
> 
>    Every pg_* not in /usr/lib/postgresql/10.3/bin/ now points to its 
> namesake
> there.

Hmm. Grasping at straws. In a previous post you mentioned:

"If it matters, there's no /etc/postgresql/ and none in the backups 
since the beginning of August. "

What is in the backups at the beginning of August.

As Andrew pointed out the binary you are using thinks its files are in 
10.2/.

The question is what was running Postgres 'correctly' before this most 
recent change?

Is there information in the system logs from your last successful start 
that you would help with this?



> 
>    Question: if pg_dump, pg_dumpall, pg_restore, pg_ctl, and pg_controldata
> have symlinks in /usr/bin/ do they also need symlinks in /bin/? The ones
> found there dated back to 2010 and I don't know that they're needed now.
> 
>    pg_ctl start still cannot find the proper timezonesets/ directory. If 
> you
> have suggestions what pg_ctl is calling that looks in the non-existent 
> 10.2/
> directory instead of the existing 10.3/ directory please let me know.
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 3:03 PM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Andrew Gierth wrote:
> 
>> What this says is that you somehow have a pg 10.3 binary which has been
>> compiled with ./configure --datadir=/usr/share/postgresql-10.2
>>
>> which seems, to say the least, somewhat odd.
> 
> Andrew,
> 
>    Quite odd rather than somewhat odd because the configure options in the
> build script point to version 10.3, not 10.2.

Well there is something strange going. From a previous post:

"
When I run pg_config --configure this is the result:
# ./pg_config --configure
'--prefix=/usr/lib/postgresql/10.2' '--sysconfdir=/etc/postgresql/10.2'
'--includedir=/usr/include' '--datarootdir=/usr/share' '--mandir=/usr/man'
'--docdir=/usr/doc/postgresql-10.3' '--datadir=/usr/share/postgresql-10.2'
'--with-openssl' '--with-tcl' '--with-perl' '--with-python' '--with-libxml'
'--with-libxslt' '--enable-thread-safety'
'--with-system-tzdata=/usr/share/zoneinfo' '--disable-nls'
'--build=i586-slackware-linux' 'build_alias=i586-slackware-linux'
'CFLAGS=-O2 -march=i586 -mtune=i686' 'PKG_CONFIG_PATH=/usr/lib/pkgconfig

"

Note '--docdir=/usr/doc/postgresql-10.3' where everything else is 
pointing at 10.2/

> 
>> pg_config isn't used by the postgres binary to find paths, so "fixing" it
>> wouldn't help. The same paths that were compiled into pg_config are
>> compiled into the postgres binary, and pg_config and postgres contain the
>> same relocation logic.
> 
>    Okay.
> 
>    I'll check with the SlackBuilds.org postgresql package maintainer and
> confirm that re-building and re-installing 10.3 will not adversely affect
> the data/ directory. Then I'll re-build and re-install it.
> 
>    Why it worked flawlessly until today when I modified the postgresql.conf
> file to add access by another host name and then broke is also quite odd.
> 
> Thanks,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 3:08 PM, Adrian Klaver wrote:
> On 10/31/18 3:03 PM, Rich Shepard wrote:
>> On Wed, 31 Oct 2018, Andrew Gierth wrote:
>>
>>> What this says is that you somehow have a pg 10.3 binary which has been
>>> compiled with ./configure --datadir=/usr/share/postgresql-10.2
>>>
>>> which seems, to say the least, somewhat odd.
>>
>> Andrew,
>>
>>    Quite odd rather than somewhat odd because the configure options in 
>> the
>> build script point to version 10.3, not 10.2.
> 
> Well there is something strange going. From a previous post:
> 
> "
> When I run pg_config --configure this is the result:
> # ./pg_config --configure
> '--prefix=/usr/lib/postgresql/10.2' '--sysconfdir=/etc/postgresql/10.2'
> '--includedir=/usr/include' '--datarootdir=/usr/share' '--mandir=/usr/man'
> '--docdir=/usr/doc/postgresql-10.3' '--datadir=/usr/share/postgresql-10.2'
> '--with-openssl' '--with-tcl' '--with-perl' '--with-python' '--with-libxml'
> '--with-libxslt' '--enable-thread-safety'
> '--with-system-tzdata=/usr/share/zoneinfo' '--disable-nls'
> '--build=i586-slackware-linux' 'build_alias=i586-slackware-linux'
> 'CFLAGS=-O2 -march=i586 -mtune=i686' 'PKG_CONFIG_PATH=/usr/lib/pkgconfig
> 
> "
> 
> Note '--docdir=/usr/doc/postgresql-10.3' where everything else is 
> pointing at 10.2/

Hmm in the build script the difference is:

VERSION=${VERSION:-10.3}
PG_VERSION=${PG_VERSION:-10.3}

--docdir=/usr/doc/$PRGNAM-$VERSION \
--datadir=/usr/share/$PRGNAM-$PG_VERSION \

Wonder where the script is finding PG_VERSION?

Do you have env variable set for that?

> 
>>
>>> pg_config isn't used by the postgres binary to find paths, so 
>>> "fixing" it
>>> wouldn't help. The same paths that were compiled into pg_config are
>>> compiled into the postgres binary, and pg_config and postgres contain 
>>> the
>>> same relocation logic.
>>
>>    Okay.
>>
>>    I'll check with the SlackBuilds.org postgresql package maintainer and
>> confirm that re-building and re-installing 10.3 will not adversely affect
>> the data/ directory. Then I'll re-build and re-install it.
>>
>>    Why it worked flawlessly until today when I modified the 
>> postgresql.conf
>> file to add access by another host name and then broke is also quite odd.
>>
>> Thanks,
>>
>> Rich
>>
>>
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> Well there is something strange going. From a previous post:

Andrew,

   Yet it ran without a whimper from the upgrade last March 1st to this
morning when I modified postgresql.conf.

   It's the middle of the night in central Europe so I expect to hear from
the package maintainer tomorrow about a re-buid/re-installation not touching
/var/lib/pgsql/data/.

Thanks,

Rich


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 3:19 PM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> Well there is something strange going. From a previous post:
> 
> Andrew,
> 
>    Yet it ran without a whimper from the upgrade last March 1st to this
> morning when I modified postgresql.conf.
> 
>    It's the middle of the night in central Europe so I expect to hear from
> the package maintainer tomorrow about a re-buid/re-installation not 
> touching
> /var/lib/pgsql/data/.

Even with assurances I would back up that directory(assuming space 
available) before proceeding with a rebuild. Or do you have a recent 
dump of the cluster?

> 
> Thanks,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Adrian Klaver
Date:
On 10/31/18 3:19 PM, Rich Shepard wrote:
> On Wed, 31 Oct 2018, Adrian Klaver wrote:
> 
>> Well there is something strange going. From a previous post:
> 
> Andrew,
> 
>    Yet it ran without a whimper from the upgrade last March 1st to this
> morning when I modified postgresql.conf.
> 
>    It's the middle of the night in central Europe so I expect to hear from
> the package maintainer tomorrow about a re-buid/re-installation not 
> touching
> /var/lib/pgsql/data/.

The thing is from here:

http://slackbuilds.org/result/?search=postgres&sv=14.2

I only see Postgres 10.2.

> 
> Thanks,
> 
> Rich
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com


Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> Hmm in the build script the difference is:
>
> VERSION=${VERSION:-10.3}
> PG_VERSION=${PG_VERSION:-10.3}
>
> --docdir=/usr/doc/$PRGNAM-$VERSION \
> --datadir=/usr/share/$PRGNAM-$PG_VERSION \
>
> Wonder where the script is finding PG_VERSION?
> Do you have env variable set for that?

Adrian,

   The first two lines set the variables, along with the earlier PRGNAM
variable:

PRGNAM=postgresql
VERSION=${VERSION:-10.3}
BUILD=${BUILD:-1}
TAG=${TAG:-_SBo}

PG_VERSION=${PG_VERSION:-10.3}
PG_PORT=${PG_PORT:-5432}

   When I check the local repository of installed packages that's how it
displays:

$ ls /var/log/packages/ | grep postgresql
postgresql-10.3-i586-1_SBo

   So, /usr/doc/ should have postgresql-10.3, and it is:
$ ls /usr/doc/postgresql*
/usr/doc/postgresql-10.3

and

$ ls /usr/share/postgresql-10.3/

   So why 'pg_config --configure' has them at 10.2 doesn't follow from the
build configuration.

Regards,

Rich




Re: Broken postgres links need to find callers

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Adrian Klaver wrote:

> Even with assurances I would back up that directory(assuming space available) 
> before proceeding with a rebuild. Or do you have a recent dump of the 
> cluster?

Adrian,

   That's my thinking, too. No, an explicit pg_dumpall is too old to be
useful. I have daily uncremental backups made by dirvish but I'll copy the
451M data directory to a USB thumb drive for safe keeping.

Regards,

Rich


Re: Broken postgres links need to find callers [FIXED]

From
Rich Shepard
Date:
On Wed, 31 Oct 2018, Rich Shepard wrote:

> I managed to mess up postgresql-10.3 on this Slackware-14.2 desktop
> server/workstation. It worked OK until I tried adding access to an another
> application.

   The problems have been resolved by upgrading 10.3 to 10.5 using the
SlackBuild.org script. There was a minor error when that was changed from
version 9 to 10 and after correcting that the build went well. Long story
short: the server started and I can once again access databases using psql.
Now I go back to getting GRASS to connect to the database after modifying
postgresql.conf to accept connections from all hosts on my office network.

   My thanks to Adrian and Andrew for their patient help.

Best regards,

Rich