Thread: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Perumal Raj
Date:
Hi Team,

Please give me some pointers to resolve this issue.

-bash-4.2$ export OLDCLUSTER=/usr/pgsql-9.2
-bash-4.2$ export NEWCLUSTER=/usr/pgsql-11

-bash-4.2$ /usr/pgsql-11/bin/pg_upgrade --old-bindir=$OLDCLUSTER/bin --new-bindir=$NEWCLUSTER/bin --old-datadir=/data/db/data --new-datadir=/pgdata/11/data -p 5432 -P 5433 --check
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

*failure*
Consult the last few lines of "pg_upgrade_server.log" for
the probable cause of the failure.

connection to database failed: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/lib/pgsql/.s.PGSQL.5432"?

could not connect to source postmaster started with the command:
"/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start
Failure, exiting
-bash-4.2$ 


+++

-bash-4.2$ /usr/pgsql-9.2/bin/pg_ctl -D "/data/db/data" start -l logfile
server starting
-bash-4.2$

+++
-bash-4.2$ /usr/pgsql-11/bin/pg_upgrade --old-bindir=$OLDCLUSTER/bin --new-bindir=$NEWCLUSTER/bin --old-datadir=/data/db/data --new-datadir=/pgdata/11/data -p 5432 -P 5433  --check
Performing Consistency Checks on Old Live Server
------------------------------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for invalid "unknown" user columns                 ok
Checking for hash indexes                                   ok
Checking for roles starting with "pg_"                      ok
Checking for incompatible "line" data type                  ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok

*Clusters are compatible*
-bash-4.2$ 

+++

-bash-4.2$ /usr/pgsql-11/bin/pg_upgrade --old-bindir=$OLDCLUSTER/bin --new-bindir=$NEWCLUSTER/bin --old-datadir=/data/db/data --new-datadir=/pgdata/11/data -p 5432 -P 5433

There seems to be a postmaster servicing the old cluster.
Please shutdown that postmaster and try again.
Failure, exiting
-bash-4.2$ 

-bash-4.2$ /usr/pgsql-9.2/bin/pg_ctl -D "/data/db/data" stop -l logfilewaiting for server to shut down.... done
server stopped
-bash-4.2$ 


-bash-4.2$ /usr/pgsql-11/bin/pg_upgrade --old-bindir=$OLDCLUSTER/bin --new-bindir=$NEWCLUSTER/bin --old-datadir=/data/db/data --new-datadir=/pgdata/11/data -p 5432 -P 5433
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

*failure*
Consult the last few lines of "pg_upgrade_server.log" for
the probable cause of the failure.

connection to database failed: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/lib/pgsql/.s.PGSQL.5432"?

could not connect to source postmaster started with the command:
"/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start
Failure, exiting
-bash-4.2$ 


Thanks,

Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Luca Ferrari
Date:
On Tue, Jul 23, 2019 at 3:56 PM Perumal Raj <perucinci@gmail.com> wrote:
> could not connect to source postmaster started with the command:
> "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/data/db/data" -o "-p 5432 -b  -c listen_addresses=''
-cunix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start
 
> Failure, exiting

Is /var/lib/pgsql directory on the system?
Can you start the instance with the above command?


Luca



Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Perumal Raj
Date:
Hi Luca

-bash-4.2$ "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
-bash-4.2$

-bash-4.2$ pwd
/var/lib/pgsql
-bash-4.2$ 

upgrade log :


command: "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start >> "pg_upgrade_server.log" 2>&1
waiting for server to start....FATAL:  unrecognized configuration parameter "unix_socket_directory"
 stopped waiting
pg_ctl: could not start server
Examine the log output.


On Tue, Jul 23, 2019 at 7:15 AM Luca Ferrari <fluca1978@gmail.com> wrote:
On Tue, Jul 23, 2019 at 3:56 PM Perumal Raj <perucinci@gmail.com> wrote:
> could not connect to source postmaster started with the command:
> "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" start
> Failure, exiting

Is /var/lib/pgsql directory on the system?
Can you start the instance with the above command?


Luca

Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Adrian Klaver
Date:
On 7/23/19 7:17 AM, Perumal Raj wrote:
> Hi Luca
> 
> -bash-4.2$ "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D 
> "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c 
> unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" 
> start
> waiting for server to start.... stopped waiting
> pg_ctl: could not start server
> Examine the log output.
> -bash-4.2$
> 
> -bash-4.2$ pwd
> /var/lib/pgsql
> -bash-4.2$
> 
> upgrade log :
> 
> 
> command: "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D 
> "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c 
> unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" 
> start >> "pg_upgrade_server.log" 2>&1
> waiting for server to start....FATAL:  unrecognized configuration 
> parameter "unix_socket_directory"

Well unix_socket_directory changed to  unix_socket_directories in 9.3.

Are you sure that /usr/pgsql-9.2/bin/pg_ctl is really pointing to a 9.2 
binary?

>   stopped waiting
> pg_ctl: could not start server
> Examine the log output.
> 
> 
> On Tue, Jul 23, 2019 at 7:15 AM Luca Ferrari <fluca1978@gmail.com 
> <mailto:fluca1978@gmail.com>> wrote:
> 
>     On Tue, Jul 23, 2019 at 3:56 PM Perumal Raj <perucinci@gmail.com
>     <mailto:perucinci@gmail.com>> wrote:
>      > could not connect to source postmaster started with the command:
>      > "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D
>     "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c
>     unix_socket_permissions=0700 -c
>     unix_socket_directory='/var/lib/pgsql'" start
>      > Failure, exiting
> 
>     Is /var/lib/pgsql directory on the system?
>     Can you start the instance with the above command?
> 
> 
>     Luca
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Tom Lane
Date:
Adrian Klaver <adrian.klaver@aklaver.com> writes:
> On 7/23/19 7:17 AM, Perumal Raj wrote:
>> command: "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D 
>> "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c 
>> unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'" 
>> start >> "pg_upgrade_server.log" 2>&1
>> waiting for server to start....FATAL:  unrecognized configuration 
>> parameter "unix_socket_directory"

> Well unix_socket_directory changed to  unix_socket_directories in 9.3.

Yeah, this is clearly a version-skew problem.  pg_upgrade knows it
should say unix_socket_directory not unix_socket_directories to a pre-9.3
server, but that's going wrong somehow.

> Are you sure that /usr/pgsql-9.2/bin/pg_ctl is really pointing to a 9.2 
> binary?

For quite some time, Red Hat shipped versions of 9.2.x that were patched
to understand unix_socket_directories not unix_socket_directory.  I would
be suspicious that the source server was one of those, except that the
cited path doesn't match where the Red Hat RPMs put it.

            regards, tom lane



Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Perumal Raj
Date:
Hi All,

Yes, , bin file correct only as it came up with CentOS 7.6 .

Anyhow , thanks for the pointers which helped me to look at hacking solution :-) 

Currently , upgrade is running. will keep u posted with results.

Thanks,

On Tue, Jul 23, 2019 at 7:43 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Adrian Klaver <adrian.klaver@aklaver.com> writes:
> On 7/23/19 7:17 AM, Perumal Raj wrote:
>> command: "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D
>> "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c
>> unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'"
>> start >> "pg_upgrade_server.log" 2>&1
>> waiting for server to start....FATAL:  unrecognized configuration
>> parameter "unix_socket_directory"

> Well unix_socket_directory changed to  unix_socket_directories in 9.3.

Yeah, this is clearly a version-skew problem.  pg_upgrade knows it
should say unix_socket_directory not unix_socket_directories to a pre-9.3
server, but that's going wrong somehow.

> Are you sure that /usr/pgsql-9.2/bin/pg_ctl is really pointing to a 9.2
> binary?

For quite some time, Red Hat shipped versions of 9.2.x that were patched
to understand unix_socket_directories not unix_socket_directory.  I would
be suspicious that the source server was one of those, except that the
cited path doesn't match where the Red Hat RPMs put it.

                        regards, tom lane

Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Perumal Raj
Date:
Hi All,

Finally upgrade completed successfully after implementing the following Workaround.

mv /usr/bin/pg_ctl{,-orig}
echo '#!/bin/bash' > /usr/bin/pg_ctl
echo '"$0"-orig "${@/unix_socket_directory/unix_socket_directories}"' >>
     /usr/bin/pg_ctl
chmod +x /usr/bin/pg_ctl
Special thanks to ''Ziggy Crueltyfree Zeitgeister '



On Tue, Jul 23, 2019 at 7:51 AM Perumal Raj <perucinci@gmail.com> wrote:
Hi All,

Yes, , bin file correct only as it came up with CentOS 7.6 .

Anyhow , thanks for the pointers which helped me to look at hacking solution :-) 

Currently , upgrade is running. will keep u posted with results.

Thanks,

On Tue, Jul 23, 2019 at 7:43 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Adrian Klaver <adrian.klaver@aklaver.com> writes:
> On 7/23/19 7:17 AM, Perumal Raj wrote:
>> command: "/usr/pgsql-9.2/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D
>> "/data/db/data" -o "-p 5432 -b  -c listen_addresses='' -c
>> unix_socket_permissions=0700 -c unix_socket_directory='/var/lib/pgsql'"
>> start >> "pg_upgrade_server.log" 2>&1
>> waiting for server to start....FATAL:  unrecognized configuration
>> parameter "unix_socket_directory"

> Well unix_socket_directory changed to  unix_socket_directories in 9.3.

Yeah, this is clearly a version-skew problem.  pg_upgrade knows it
should say unix_socket_directory not unix_socket_directories to a pre-9.3
server, but that's going wrong somehow.

> Are you sure that /usr/pgsql-9.2/bin/pg_ctl is really pointing to a 9.2
> binary?

For quite some time, Red Hat shipped versions of 9.2.x that were patched
to understand unix_socket_directories not unix_socket_directory.  I would
be suspicious that the source server was one of those, except that the
cited path doesn't match where the Red Hat RPMs put it.

                        regards, tom lane

Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Tom Lane
Date:
Perumal Raj <perucinci@gmail.com> writes:
> Finally upgrade completed successfully after implementing the following
> Workaround.

You didn't say where you got these executables from, but if the 9.2 and 11
packages were from the same packager, you should complain to them about
it.  If they're patching 9.2 to have unix_socket_directories, they should
also patch pg_upgrade in later versions to understand that.

            regards, tom lane



Re: pg_upgrade : 9.X to 11.X issue CentoOS 7.6

From
Perumal Raj
Date:

On Tue, Jul 23, 2019 at 9:20 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Perumal Raj <perucinci@gmail.com> writes:
> Finally upgrade completed successfully after implementing the following
> Workaround.

You didn't say where you got these executables from, but if the 9.2 and 11
packages were from the same packager, you should complain to them about
it.  If they're patching 9.2 to have unix_socket_directories, they should
also patch pg_upgrade in later versions to understand that.

                        regards, tom lane