Thread: perl path issue

On 5/9/19 4:53 AM, Prakash Ramakrishnan wrote:
> 
> Hi Team,
> 
> We having perl issue for while during restore operation for using 
> pgbackrest backup tool can you please do the needful.

Looks to me like the Postgres client library libpq is not installed or 
cannot be found on this machine.

You will need to verify using your package manager.

> 
> 2019-05-08 18:42:09.415 P00   INFO: restore command begin 2.13: 
> --db-include=A4_Copy --log-level-console=info 
> --pg1-path=/Postgres/prakash_pgbackrest --process-max=2 
> --repo1-host=sydcosacpp001.enterprisenet.org 
> <http://sydcosacpp001.enterprisenet.org> 
> --repo1-host-config=/etc/pgbackrest.conf --repo1-host-user=postgres 
> --repo1-path=/pgBACKUP/A4_ sydcosacpp001  /backup/A4_ 
> sydcosacpp001   --stanza=A4_s sydcosacpp001  1
> 2019-05-08 18:42:15.082 P00  ERROR: [124]: remote process on ' 
> sydcosacpp001  .enterprisenet.org <http://enterprisenet.org>' terminated 
> unexpectedly [124]: ERROR: [124]: process terminated due to an unhandled 
> exception:
>                                                 Can't load 
> '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for module DBD::Pg: 
> libpq.so.5: cannot open shared object file: No such file or directory at 
> /usr/lib64/perl5/DynaLoader.pm line 190.
>                                                  at 
> /loader/0x226a528/pgBackRest/Db.pm line 10.
>                                                  at 
> /loader/0x226a528/pgBackRest/Main.pm line 12.
>                                                  
> pgBackRest::Main::__ANON__('Can\'t load 
> \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at 
> /usr/share/perl5/vendor_perl/Carp.pm line 100
> 
> 
> ==> cd /usr/lib64/perl5/
> postgres@sydcosausdd001.enterprisenet.org:/usr/lib64/perl5
> ==> ls
> arybase.pm <http://arybase.pm>     bits             CORE          
>   Fcntl.pm      _h2ph_pre.ph <http://h2ph_pre.ph> lib.pm 
> <http://lib.pm>   NDBM_File.pm  perllocal.pod stdarg.ph 
> <http://stdarg.ph> syslimits.ph <http://syslimits.ph>  Unicode
> asm            B.pm             Devel features.ph <http://features.ph>  
>   Hash          linux    ODBM_File.pm  POSIX.pm stdc-predef.ph 
> <http://stdc-predef.ph> syslog.ph <http://syslog.ph>     vendor_perl
> asm-generic    Config_git.pl    Digest         File          I18N        
>    machine  Opcode.pm     POSIX.pod stddef.ph <http://stddef.ph>      
>   Text wait.ph <http://wait.ph>
> attributes.pm <http://attributes.pm>  Config_heavy.pl  DynaLoader.pm  
> Filter        IO            Math     O.pm re.pm <http://re.pm>          
> sys             Tie xlocale.ph <http://xlocale.ph>
> auto           Config.pm endian.ph <http://endian.ph>      GDBM_File.pm  
> IO.pm         MIME ops.pm <http://ops.pm>        SDBM_File.pm   Sys      
>         Time
> B              Config.pod       Errno.pm       gnu           IPC mro.pm 
> <http://mro.pm>   PerlIO signal.ph <http://signal.ph> syscall.ph 
> <http://syscall.ph> time.ph <http://time.ph>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/9/19 9:39 AM, Adrian Klaver wrote:
> On 5/9/19 4:53 AM, Prakash Ramakrishnan wrote:
>>
>> We having perl issue for while during restore operation for using
>> pgbackrest backup tool can you please do the needful.
> 
> Looks to me like the Postgres client library libpq is not installed or
> cannot be found on this machine.

If pgBackRest was built from source here's the complete dependency list
for RHEL7:

yum install perl perl-Time-HiRes perl-Digest-SHA perl-DBD-Pg perl-JSON-PP

Regards,
-- 
-David
david@pgmasters.net

On 5/9/19 10:57 AM, Prakash Ramakrishnan wrote:
> 
> David am trying to restore the backup in dev server please find the
> below details ,

libpq should be installed with Postgres.  Have you installed Postgres yet?

-- 
-David
david@pgmasters.net

On 5/9/19 11:12 AM, Prakash Ramakrishnan wrote:
> Yes, I did. 

OK, then I would say something is wrong with your package configuration,
but I don't know what it might be.

Regards,
-- 
-David
david@pgmasters.net

On 5/9/19 8:12 AM, Prakash Ramakrishnan wrote:
> Yes, I did.

The server is there, but where the client libraries installed?

What does your package manager show for installed Postgres packages?

> 
> postgres 12546     1  0 Apr28 ?        00:03:07 
> /opt/app/PostgreSQL/10/bin/postgres
> postgres 12554 12546  0 Apr28 ?        00:04:12 postgres: logger process
> postgres 12557 12546  0 Apr28 ?        01:06:28 postgres: checkpointer 
> process
> postgres 12558 12546  0 Apr28 ?        00:01:57 postgres: writer process
> postgres 12559 12546  0 Apr28 ?        01:47:41 postgres: wal writer process
> postgres 12560 12546  0 Apr28 ?        00:02:05 postgres: autovacuum 
> launcher process
> postgres 12561 12546  0 Apr28 ?        00:26:30 postgres: archiver 
> process   failed on 000000010000142700000032
> postgres 12562 12546  0 Apr28 ?        00:13:20 postgres: stats 
> collector process
> postgres 12563 12546  0 Apr28 ?        00:00:02 postgres: bgworker: 
> logical replication launcher
> root     12858 28000  0 May09 ?        00:00:00 sshd: postgres [priv]
> postgres 12896 12858  0 May09 ?        00:00:00 sshd: postgres@pts/0
> 
> 
> On Thu, May 9, 2019 at 8:38 PM David Steele <david@pgmasters.net 
> <mailto:david@pgmasters.net>> wrote:
> 
>     On 5/9/19 10:57 AM, Prakash Ramakrishnan wrote:
>      >
>      > David am trying to restore the backup in dev server please find the
>      > below details ,
> 
>     libpq should be installed with Postgres.  Have you installed
>     Postgres yet?
> 
>     -- 
>     -David
>     david@pgmasters.net <mailto:david@pgmasters.net>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> I don't know about that client libraries can you please explain or share
> me the command I will show you the output and we need the solution for
> this pgbackrest issue .

yum install postgresql10

should do it, but I thought pqlib was installed with the server as well.
 Perhaps not.

-- 
-David
david@pgmasters.net

On 5/9/19 9:54 AM, David Steele wrote:
> On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
>> Hi Adrian,
>>
>> I don't know about that client libraries can you please explain or share
>> me the command I will show you the output and we need the solution for
>> this pgbackrest issue .
> 
> yum install postgresql10

Do you really want to do this as the server is already installed?

> 
> should do it, but I thought pqlib was installed with the server as well.
>   Perhaps not.
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/9/19 9:58 AM, Prakash Ramakrishnan wrote:
> Hi David,
> 
> I did the installation for using postgresql 10.4 version edb standard 
> edition.so again need to run the yum command or please advise me .

So you are using the EDB installer correct?

If so YUM is not going to help you.
> 
> 
> Regards,
> Prakash.R
> 
> On Thu, May 9, 2019, 22:24 David Steele <david@pgmasters.net 
> <mailto:david@pgmasters.net>> wrote:
> 
>     On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > I don't know about that client libraries can you please explain
>     or share
>      > me the command I will show you the output and we need the
>     solution for
>      > this pgbackrest issue .
> 
>     yum install postgresql10
> 
>     should do it, but I thought pqlib was installed with the server as well.
>       Perhaps not.
> 
>     -- 
>     -David
>     david@pgmasters.net <mailto:david@pgmasters.net>
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/9/19 12:58 PM, Adrian Klaver wrote:
> On 5/9/19 9:54 AM, David Steele wrote:
>> On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
>>> Hi Adrian,
>>>
>>> I don't know about that client libraries can you please explain or share
>>> me the command I will show you the output and we need the solution for
>>> this pgbackrest issue .
>>
>> yum install postgresql10
> 
> Do you really want to do this as the server is already installed?

Perhaps not, but I was hoping it might fix a broken package.  Since he's
running EDB it almost certainly won't help.

-- 
-David
david@pgmasters.net

On 5/9/19 12:58 PM, Prakash Ramakrishnan wrote:
> 
> I did the installation for using postgresql 10.4 version edb standard
> edition.so again need to run the yum command or please advise me .

It really depends on how you've installed EDB, packages or their
customer installer.

The custom installer installs stuff in non-standard locations so likely
won't work without a lot of tweaking.  I don't have any experience with
the EDB packages.

Devrim?

-- 
-David
david@pgmasters.net

On 5/9/19 9:58 AM, Prakash Ramakrishnan wrote:
> Hi David,
> 
> I did the installation for using postgresql 10.4 version edb standard 
> edition.so again need to run the yum command or please advise me .

Should have added to previous post. A heads up:

Postgres 10.x is the last Linux version you can use the EDB installer. 
Going forward you will need to use the native packages:

https://www.enterprisedb.com/downloads/postgres-postgresql-downloads

"Please Note: EDB no longer provides PostgreSQL installers for Linux, 
and users are encouraged to use the platform-native packages. For more 
information, please see this blog post on Platform Native EDB Packages 
for Linux Users."


https://postgresrocks.enterprisedb.com/t5/Postgres-Gems/Platform-Native-EDB-Packages-for-Linux-Users/ba-p/2362?_ga=2.116418510.996235935.1557421212-1173178781.1557421212

> 
> 
> Regards,
> Prakash.R
> 
> On Thu, May 9, 2019, 22:24 David Steele <david@pgmasters.net 
> <mailto:david@pgmasters.net>> wrote:
> 
>     On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > I don't know about that client libraries can you please explain
>     or share
>      > me the command I will show you the output and we need the
>     solution for
>      > this pgbackrest issue .
> 
>     yum install postgresql10
> 
>     should do it, but I thought pqlib was installed with the server as well.
>       Perhaps not.
> 
>     -- 
>     -David
"Please Note: EDB no longer provides PostgreSQL installers for Linux, 
and users are encouraged to use the platform-native packages. For more 
information, please see this blog post on Platform Native EDB Packages 
for Linux Users.
>     david@pgmasters.net <mailto:david@pgmasters.net>
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/9/19 9:58 AM, Prakash Ramakrishnan wrote:
> Hi David,
> 
> I did the installation for using postgresql 10.4 version edb standard 
> edition.so again need to run the yum command or please advise me .

I don't have time to go through the below, but some issues filed at 
pgbackrest about using EDB installed database:

https://github.com/pgbackrest/pgbackrest/issues?q=is%3Aissue+edb+is%3Aclosed

> 
> 
> Regards,
> Prakash.R
> 
> On Thu, May 9, 2019, 22:24 David Steele <david@pgmasters.net 
> <mailto:david@pgmasters.net>> wrote:
> 
>     On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > I don't know about that client libraries can you please explain
>     or share
>      > me the command I will show you the output and we need the
>     solution for
>      > this pgbackrest issue .
> 
>     yum install postgresql10
> 
>     should do it, but I thought pqlib was installed with the server as well.
>       Perhaps not.
> 
>     -- 
>     -David
>     david@pgmasters.net <mailto:david@pgmasters.net>
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/10/19 2:24 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> Some perl path is not working here please find the below details,

How did you install/build pgbackrest?

The basic issue as I see it is that pgBackRest and your EDB Postgres are 
living in separate worlds as far as each is concerned. To fix this you 
have two options as I see it:

1) To reconcile then you will need to either consult the pgBackRest or 
EDB folks for a solution, as I use neither and don't know how to make 
then play well together.

2) Convert your Postgres server from using the EDB installer to using 
the native packages. This is going to have to happen anyway going 
forward off version 10. I would probably stick to the community packages 
here:

https://www.postgresql.org/download/linux/redhat/


> 
> ==> ldd /opt/app/PostgreSQL/10/lib/postgresql/plperl.so
>          linux-vdso.so.1 =>  (0x00007ffccf4a3000)
> *        libperl.so => 
> /opt/edb/languagepack-10/Perl-5.24/lib/CORE/libperl.so (0x00007f9dac941000)*
>          libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9dac725000)
>          libc.so.6 => /lib64/libc.so.6 (0x00007f9dac358000)
>          libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f9dac13e000)
>          libdl.so.2 => /lib64/libdl.so.2 (0x00007f9dabf3a000)
>          libm.so.6 => /lib64/libm.so.6 (0x00007f9dabc38000)
>          libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f9daba01000)
>          libutil.so.1 => /lib64/libutil.so.1 (0x00007f9dab7fe000)
>          /lib64/ld-linux-x86-64.so.2 (0x00007f9dacf44000)
>          libfreebl3.so => /lib64/libfreebl3.so (0x00007f9dab5fb000)
> 
> but the pgbackrest need this perl path i thin -   libperl.so => 
> /usr/lib64/perl5/CORE/libperl.so (0x00007f955333b000)
> 
> 
> still we getting the error,
> 
> ==> pgbackrest --stanza=A4_sydcosafpp001 --log-level-console=info 
> --db-path=/Postgres/prakash_pgbackrest --delta  --process-max=2 
> --db-include=A4_Copy restore 
> --repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001
> 2019-05-10 19:23:39.525 P00   INFO: restore command begin 2.13: 
> --db-include=A4_Copy --delta --log-level-console=info 
> --pg1-path=/Postgres/prakash_pgbackrest --process-max=2 
> --repo1-host=sydcosafpp001.enterprisenet.org 
> <http://sydcosafpp001.enterprisenet.org> 
> --repo1-host-config=/etc/pgbackrest.conf --repo1-host-user=postgres 
> --repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001 
> --stanza=A4_sydcosafpp001
> Password:
> ERROR [124]: : remote process on 'sydcosafpp001.enterprisenet.org 
> <http://sydcosafpp001.enterprisenet.org>' terminated unexpectedly [124]: 
> ERROR: [124]: process terminated due to an unhandled exception:
> *Can't load '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for module 
> DBD::Pg: libpq.so.5: cannot open shared object file: No such file or 
> directory at /usr/lib64/perl5/DynaLoader.pm line 190.*
>          at /loader/0x103a528/pgBackRest/Db.pm line 10.
>          at /loader/0x103a528/pgBackRest/Main.pm line 12.
>          pgBackRest::Main::__ANON__('Can\'t load 
> \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at 
> /usr/share/perl5/vendor_perl/Carp.pm line 100
>          Carp::croak('Can\'t load 
> \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at 
> /usr/lib64/perl5/DynaLoader.pm line 98
>          DynaLoader::croak('Can\'t load 
> \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at 
> /usr/lib64/perl5/DynaLoader.pm line 190
>          DynaLoader::bootstrap('DBD::Pg', 'version=HASH(0x19b1a68)') 
> called at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 73
>          require DBD/Pg.pm called at /loader/0x103a528/pgBackRest/Db.pm 
> line 10
>          pgBackRest::Db::BEGIN() called at 
> /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          eval {...} called at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          require pgBackRest/Db.pm called at 
> /loader/0x103a528/pgBackRest/Archive/Common.pm line 16
>          pgBackRest::Archive::Common::BEGIN() called at 
> /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          eval {...} called at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          require pgBackRest/Archive/Common.pm called at 
> /loader/0x103a528/pgBackRest/Archive/Get/File.pm line 15
>          pgBackRest::Archive::Get::File::BEGIN() called at 
> /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          eval {...} called at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          require pgBackRest/Archive/Get/File.pm called at 
> /loader/0x103a528/pgBackRest/Protocol/Remote/Minion.pm line 17
>          pgBackRest::Protocol::Remote::Minion::BEGIN() called at 
> /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          eval {...} called at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
>          require pgBackRest/Protocol/Remote/Minion.pm called at 
> /loader/0x103a528/pgBackRest/Main.pm line 118
>          eval {...} called at /loader/0x103a528/pgBackRest/Main.pm line 286
>          pgBackRest::Main::main('remote') called at (eval 269) line 1
>          eval '($iResult, $bErrorC, $strMessage) = 
> pgBackRest::Main::main(\'remote\')
>         ;' called at /usr/lib64/perl5/vendor_perl/DBD/Pg.pm line 0
> 
> 
> can you please help this we are struggling to solve the issue,
> 
> Regards,
> Prakash.R
> 
> 
> 
> 
> 
> On Thu, May 9, 2019 at 10:58 PM Adrian Klaver <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 5/9/19 9:58 AM, Prakash Ramakrishnan wrote:
>      > Hi David,
>      >
>      > I did the installation for using postgresql 10.4 version edb
>     standard
>      > edition.so again need to run the yum command or please advise me .
> 
>     I don't have time to go through the below, but some issues filed at
>     pgbackrest about using EDB installed database:
> 
>     https://github.com/pgbackrest/pgbackrest/issues?q=is%3Aissue+edb+is%3Aclosed
> 
>      >
>      >
>      > Regards,
>      > Prakash.R
>      >
>      > On Thu, May 9, 2019, 22:24 David Steele <david@pgmasters.net
>     <mailto:david@pgmasters.net>
>      > <mailto:david@pgmasters.net <mailto:david@pgmasters.net>>> wrote:
>      >
>      >     On 5/9/19 12:51 PM, Prakash Ramakrishnan wrote:
>      >      > Hi Adrian,
>      >      >
>      >      > I don't know about that client libraries can you please
>     explain
>      >     or share
>      >      > me the command I will show you the output and we need the
>      >     solution for
>      >      > this pgbackrest issue .
>      >
>      >     yum install postgresql10
>      >
>      >     should do it, but I thought pqlib was installed with the
>     server as well.
>      >       Perhaps not.
>      >
>      >     --
>      >     -David
>      > david@pgmasters.net <mailto:david@pgmasters.net>
>     <mailto:david@pgmasters.net <mailto:david@pgmasters.net>>
>      >
> 
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/10/19 7:36 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> I did the installation for like ,
> 
> postgresql-10.4-1-linux-x64.run file.
> 
> pgbackrest - yum install 
> https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rp 
> <https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rpm>m
> 
> 
> yum install pgbackrest

As explained upstream that is not going to work. The package YUM is 
installing is assuming that it will be working with Postgres installed 
from packages in the same repo. That is not the case. You might be able 
to download the pgBackrest source and build it pointing at the Perl in 
the EDB install, but I have no idea how to make that happen.
> 
> 
> now what is the problem means we have to trying restore the backup prod 
> to dev server.
> 

So was the prod backup taken with pgBackRest?

-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/10/19 7:50 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> Production backup working fine now and previously dev server also 
> working fine.
> 
> Now the business team wants to restore the single database using 
> pgbackrest tool so we have setup the pgbackrest.conf file is like to 
> point prod .


The above is not much to go on. So a list of questions below. Please 
answer them all, thanks:

1) How was the production Postgres installed(EDB, native packages) and 
on what OS?

2) The dev Postgres is the one installed using EDB installer, correct?

3) Define working fine. In other words where you/are you using a backup 
method other then pgBackRest on the production and dev server?

4) When you say single database are you talking about a particular 
database in the Postgres cluster or the entire cluster?

5) Why is pgBackRest essential to this? In other words why not use the 
Postgres pg_dump/pg_restore tools?


As to below, this more then a Perl issue it is a package incompatibility 
issue.


> 
> and directly  restore the backup from prod so thats why not able to do 
> some perl issue .
> 
> database size is huge so they giving pressure and below is the config file,
> 
> ==> cat /etc/pgbackrest.conf
> #[global]
> #repo1-path=/Postgres/pgBACKUP/A4_sydcosausd001
> #retention-diff=4
> #retention-full=4
> #retention-archive=2
> #start-fast=y
> #process-max=4
> #archive-async=y
> 
> 
> #[global:archive-push]
> #process-max=4
> 
> #[A4_sydcosausd001]
> #db-path=/Postgres/pgDATA/data
> 
> *
> *
> *[global]*
> *repo1-host=sydcosafpp001.enterprisenet.org 
> <http://sydcosafpp001.enterprisenet.org>*
> *repo1-host-user=postgres*
> *repo1-host-config=/etc/pgbackrest.conf*
> *repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001*
> *
> *
> *[A4_sydcosafpp001]*
> *pg1-path=/Postgres/prakash_pgbackrest*
> postgres@sydcosausd001.enterprisenet.org:/home/postgres
> 
> 
> 
> On Fri, May 10, 2019 at 8:15 PM Adrian Klaver <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 5/10/19 7:36 AM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > I did the installation for like ,
>      >
>      > postgresql-10.4-1-linux-x64.run file.
>      >
>      > pgbackrest - yum install
>      >
>     https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rp
> 
>      >
>     <https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rpm>m
>      >
>      >
>      > yum install pgbackrest
> 
>     As explained upstream that is not going to work. The package YUM is
>     installing is assuming that it will be working with Postgres installed
>     from packages in the same repo. That is not the case. You might be able
>     to download the pgBackrest source and build it pointing at the Perl in
>     the EDB install, but I have no idea how to make that happen.
>      >
>      >
>      > now what is the problem means we have to trying restore the
>     backup prod
>      > to dev server.
>      >
> 
>     So was the prod backup taken with pgBackRest?
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/10/19 6:49 PM, Prakash Ramakrishnan wrote:
> Hi Adrian,

If I am following the below correctly you have EDB Postgres and 
pgBackRest running together on the prod server, correct?

If so what was done different/is different on the dev server?

> 
> 1) How was the production Postgres installed(EDB, native packages) and
> on what OS?
> 
> prod has been installed edb standard edition .run file and os CENTOS 7.5
> 
> 2) The dev Postgres is the one installed using EDB installer, correct?
> 
> yes
> 
> 3) Define working fine. In other words where you/are you using a backup
> method other then pgBackRest on the production and dev server?
> 
> Please find prod config file and backup details,
> 
> ==> cat /etc/pgbackrest.conf
> [global]
> repo1-path=/pgBACKUP/A4_sydcosafpp001
> retention-diff=3
> retention-full=1
> retention-archive=2
> start-fast=y
> process-max=12
> archive-async=y
> 
> 
> [global:archive-push]
> process-max=4
> 
> [A4_sydcosafpp001]
> pg1-path=/Postgres/pgDATA/data
> postgres@sydcosafpp001.enterprisenet.org:/home/postgres
> ==> cd /pgBACKUP/A4_sydcosafpp001/
> archive/ backup/
> postgres@sydcosafpp001.enterprisenet.org:/home/postgres
> ==> cd /pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001/
> postgres@sydcosafpp001.enterprisenet.org:/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001
> ==> ls -lrth
> total 114K
> drwxr-x---. 4 postgres postgres   44 Jan  7 01:08 backup.history
> drwxr-x---. 3 postgres postgres   96 May  4 21:30 20190504-190001F
> drwxr-x---. 3 postgres postgres   96 May  5 19:41 
> 20190504-190001F_20190505-190002I
> drwxr-x---. 3 postgres postgres   96 May  6 20:28 
> 20190504-190001F_20190506-190002I
> drwxr-x---. 3 postgres postgres   96 May  7 20:26 
> 20190504-190001F_20190507-190004I
> drwxr-x---. 3 postgres postgres   96 May  8 20:28 
> 20190504-190001F_20190508-190002I
> drwxr-x---. 3 postgres postgres   96 May  9 20:32 
> 20190504-190001F_20190509-190001I
> drwxr-x---. 3 postgres postgres   96 May 10 20:32 
> 20190504-190001F_20190510-190001I
> lrwxrwxrwx. 1 postgres postgres   33 May 10 20:32 latest -> 
> 20190504-190001F_20190510-190001I
> -rw-r-----. 1 postgres postgres 5.7K May 10 20:32 backup.info 
> <http://backup.info>
> -rw-r-----. 1 postgres postgres 5.7K May 10 20:32 backup.info.copy
> 
> 
> 
> 4) When you say single database are you talking about a particular
> database in the Postgres cluster or the entire cluster?
> 
> we need restore for single database for single cluster.
> 
> 5) Why is pgBackRest essential to this? In other words why not use the
> Postgres pg_dump/pg_restore tools?
> 
> pgbackrest tool need for backup and restore larger databases.
> if suppose am using pg_dump and restore they wont accept it and taking 
> too much time using parallel option also so we use for pgbackrest.
> 
> 
> On Fri, May 10, 2019 at 9:44 PM Adrian Klaver <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 5/10/19 7:50 AM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > Production backup working fine now and previously dev server also
>      > working fine.
>      >
>      > Now the business team wants to restore the single database using
>      > pgbackrest tool so we have setup the pgbackrest.conf file is like to
>      > point prod .
> 
> 
>     The above is not much to go on. So a list of questions below. Please
>     answer them all, thanks:
> 
>     1) How was the production Postgres installed(EDB, native packages) and
>     on what OS?
> 
>     2) The dev Postgres is the one installed using EDB installer, correct?
> 
>     3) Define working fine. In other words where you/are you using a backup
>     method other then pgBackRest on the production and dev server?
> 
>     4) When you say single database are you talking about a particular
>     database in the Postgres cluster or the entire cluster?
> 
>     5) Why is pgBackRest essential to this? In other words why not use the
>     Postgres pg_dump/pg_restore tools?
> 
> 
>     As to below, this more then a Perl issue it is a package
>     incompatibility
>     issue.
> 
> 
>      >
>      > and directly  restore the backup from prod so thats why not able
>     to do
>      > some perl issue .
>      >
>      > database size is huge so they giving pressure and below is the
>     config file,
>      >
>      > ==> cat /etc/pgbackrest.conf
>      > #[global]
>      > #repo1-path=/Postgres/pgBACKUP/A4_sydcosausd001
>      > #retention-diff=4
>      > #retention-full=4
>      > #retention-archive=2
>      > #start-fast=y
>      > #process-max=4
>      > #archive-async=y
>      >
>      >
>      > #[global:archive-push]
>      > #process-max=4
>      >
>      > #[A4_sydcosausd001]
>      > #db-path=/Postgres/pgDATA/data
>      >
>      > *
>      > *
>      > *[global]*
>      > *repo1-host=sydcosafpp001.enterprisenet.org
>     <http://sydcosafpp001.enterprisenet.org>
>      > <http://sydcosafpp001.enterprisenet.org>*
>      > *repo1-host-user=postgres*
>      > *repo1-host-config=/etc/pgbackrest.conf*
>      > *repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001*
>      > *
>      > *
>      > *[A4_sydcosafpp001]*
>      > *pg1-path=/Postgres/prakash_pgbackrest*
>      > postgres@sydcosausd001.enterprisenet.org:/home/postgres
>      >
>      >
>      >
>      > On Fri, May 10, 2019 at 8:15 PM Adrian Klaver
>     <adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
>      > <mailto:adrian.klaver@aklaver.com
>     <mailto:adrian.klaver@aklaver.com>>> wrote:
>      >
>      >     On 5/10/19 7:36 AM, Prakash Ramakrishnan wrote:
>      >      > Hi Adrian,
>      >      >
>      >      > I did the installation for like ,
>      >      >
>      >      > postgresql-10.4-1-linux-x64.run file.
>      >      >
>      >      > pgbackrest - yum install
>      >      >
>      >
>     https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rp
>      >
>      >      >
>      >   
>       <https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rpm>m
>      >      >
>      >      >
>      >      > yum install pgbackrest
>      >
>      >     As explained upstream that is not going to work. The package
>     YUM is
>      >     installing is assuming that it will be working with Postgres
>     installed
>      >     from packages in the same repo. That is not the case. You
>     might be able
>      >     to download the pgBackrest source and build it pointing at
>     the Perl in
>      >     the EDB install, but I have no idea how to make that happen.
>      >      >
>      >      >
>      >      > now what is the problem means we have to trying restore the
>      >     backup prod
>      >      > to dev server.
>      >      >
>      >
>      >     So was the prod backup taken with pgBackRest?
>      >
>      >     --
>      >     Adrian Klaver
>      > adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
>     <mailto:adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>>
>      >
>      >
>      >
>      > --
>      >
>      >
>      >
>      > Thanks,
>      > Prakash.R
>      > PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure
>     Team On
>      > call : +91-8939599426
> 
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

what does ls /usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so return?

If there is no such file, then it means that on the dev server perl DBD was installed differently
than prod. I am assuming LD_LIBRARY_PATH is also correctly set.

apologize if my advise is wrong. It has been eons since I worked with perl DBD.

Hi Adrian,

Yes am using edb standard edition and installed pgbackrest in prod and dev.

nothing different am trying to restore the backup prod to dev cluster .

       Can't load '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for module DBD::Pg: libpq.so.5: cannot open shared object file: No such file or directory at /usr/lib64/perl5/DynaLoader.pm line 190.

        at /loader/0x1ec1528/pgBackRest/Db.pm line 10.

        at /loader/0x1ec1528/pgBackRest/Main.pm line 12.

        pgBackRest::Main::__ANON__('Can\'t load \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at /usr/share/perl5/vendor_perl/Carp.pm line 100

May be above path is corrupt or something issue not able to load the path.

Regards,

Prakash.R

On Sat, May 11, 2019 at 7:18 PM Adrian Klaver <adrian.klaver@aklaver.com> wrote:

On 5/10/19 6:49 PM, Prakash Ramakrishnan wrote:
> Hi Adrian,

If I am following the below correctly you have EDB Postgres and
pgBackRest running together on the prod server, correct?

If so what was done different/is different on the dev server?

>
> 1) How was the production Postgres installed(EDB, native packages) and
> on what OS?
>
> prod has been installed edb standard edition .run file and os CENTOS 7.5
>
> 2) The dev Postgres is the one installed using EDB installer, correct?
>
> yes
>
> 3) Define working fine. In other words where you/are you using a backup
> method other then pgBackRest on the production and dev server?
>
> Please find prod config file and backup details,
>
> ==> cat /etc/pgbackrest.conf
> [global]
> repo1-path=/pgBACKUP/A4_sydcosafpp001
> retention-diff=3
> retention-full=1
> retention-archive=2
> start-fast=y
> process-max=12
> archive-async=y
>
>
> [global:archive-push]
> process-max=4
>
> [A4_sydcosafpp001]
> pg1-path=/Postgres/pgDATA/data
> postgres@sydcosafpp001.enterprisenet.org:/home/postgres
> ==> cd /pgBACKUP/A4_sydcosafpp001/
> archive/ backup/
> postgres@sydcosafpp001.enterprisenet.org:/home/postgres
> ==> cd /pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001/
> postgres@sydcosafpp001.enterprisenet.org:/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001
> ==> ls -lrth
> total 114K
> drwxr-x---. 4 postgres postgres   44 Jan  7 01:08 backup.history
> drwxr-x---. 3 postgres postgres   96 May  4 21:30 20190504-190001F
> drwxr-x---. 3 postgres postgres   96 May  5 19:41
> 20190504-190001F_20190505-190002I
> drwxr-x---. 3 postgres postgres   96 May  6 20:28
> 20190504-190001F_20190506-190002I
> drwxr-x---. 3 postgres postgres   96 May  7 20:26
> 20190504-190001F_20190507-190004I
> drwxr-x---. 3 postgres postgres   96 May  8 20:28
> 20190504-190001F_20190508-190002I
> drwxr-x---. 3 postgres postgres   96 May  9 20:32
> 20190504-190001F_20190509-190001I
> drwxr-x---. 3 postgres postgres   96 May 10 20:32
> 20190504-190001F_20190510-190001I
> lrwxrwxrwx. 1 postgres postgres   33 May 10 20:32 latest ->
> 20190504-190001F_20190510-190001I
> -rw-r-----. 1 postgres postgres 5.7K May 10 20:32 backup.info
> <http://backup.info>
> -rw-r-----. 1 postgres postgres 5.7K May 10 20:32 backup.info.copy
>
>
>
> 4) When you say single database are you talking about a particular
> database in the Postgres cluster or the entire cluster?
>
> we need restore for single database for single cluster.
>
> 5) Why is pgBackRest essential to this? In other words why not use the
> Postgres pg_dump/pg_restore tools?
>
> pgbackrest tool need for backup and restore larger databases.
> if suppose am using pg_dump and restore they wont accept it and taking
> too much time using parallel option also so we use for pgbackrest.
>
>
> On Fri, May 10, 2019 at 9:44 PM Adrian Klaver <adrian.klaver@aklaver.com
> <mailto:adrian.klaver@aklaver.com>> wrote:
>
>     On 5/10/19 7:50 AM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > Production backup working fine now and previously dev server also
>      > working fine.
>      >
>      > Now the business team wants to restore the single database using
>      > pgbackrest tool so we have setup the pgbackrest.conf file is like to
>      > point prod .
>
>
>     The above is not much to go on. So a list of questions below. Please
>     answer them all, thanks:
>
>     1) How was the production Postgres installed(EDB, native packages) and
>     on what OS?
>
>     2) The dev Postgres is the one installed using EDB installer, correct?
>
>     3) Define working fine. In other words where you/are you using a backup
>     method other then pgBackRest on the production and dev server?
>
>     4) When you say single database are you talking about a particular
>     database in the Postgres cluster or the entire cluster?
>
>     5) Why is pgBackRest essential to this? In other words why not use the
>     Postgres pg_dump/pg_restore tools?
>
>
>     As to below, this more then a Perl issue it is a package
>     incompatibility
>     issue.
>
>
>      >
>      > and directly  restore the backup from prod so thats why not able
>     to do
>      > some perl issue .
>      >
>      > database size is huge so they giving pressure and below is the
>     config file,
>      >
>      > ==> cat /etc/pgbackrest.conf
>      > #[global]
>      > #repo1-path=/Postgres/pgBACKUP/A4_sydcosausd001
>      > #retention-diff=4
>      > #retention-full=4
>      > #retention-archive=2
>      > #start-fast=y
>      > #process-max=4
>      > #archive-async=y
>      >
>      >
>      > #[global:archive-push]
>      > #process-max=4
>      >
>      > #[A4_sydcosausd001]
>      > #db-path=/Postgres/pgDATA/data
>      >
>      > *
>      > *
>      > *[global]*
>      > *repo1-host=sydcosafpp001.enterprisenet.org
>     <http://sydcosafpp001.enterprisenet.org>
>      > <http://sydcosafpp001.enterprisenet.org>*
>      > *repo1-host-user=postgres*
>      > *repo1-host-config=/etc/pgbackrest.conf*
>      > *repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001*
>      > *
>      > *
>      > *[A4_sydcosafpp001]*
>      > *pg1-path=/Postgres/prakash_pgbackrest*
>      > postgres@sydcosausd001.enterprisenet.org:/home/postgres
>      >
>      >
>      >
>      > On Fri, May 10, 2019 at 8:15 PM Adrian Klaver
>     <adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
>      > <mailto:adrian.klaver@aklaver.com
>     <mailto:adrian.klaver@aklaver.com>>> wrote:
>      >
>      >     On 5/10/19 7:36 AM, Prakash Ramakrishnan wrote:
>      >      > Hi Adrian,
>      >      >
>      >      > I did the installation for like ,
>      >      >
>      >      > postgresql-10.4-1-linux-x64.run file.
>      >      >
>      >      > pgbackrest - yum install
>      >      >
>      >
>     https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rp
>      >
>      >      >
>      >   
>       <https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rpm>m
>      >      >
>      >      >
>      >      > yum install pgbackrest
>      >
>      >     As explained upstream that is not going to work. The package
>     YUM is
>      >     installing is assuming that it will be working with Postgres
>     installed
>      >     from packages in the same repo. That is not the case. You
>     might be able
>      >     to download the pgBackrest source and build it pointing at
>     the Perl in
>      >     the EDB install, but I have no idea how to make that happen.
>      >      >
>      >      >
>      >      > now what is the problem means we have to trying restore the
>      >     backup prod
>      >      > to dev server.
>      >      >
>      >
>      >     So was the prod backup taken with pgBackRest?
>      >
>      >     --
>      >     Adrian Klaver
>      > adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
>     <mailto:adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>>
>      >
>      >
>      >
>      > --
>      >
>      >
>      >
>      > Thanks,
>      > Prakash.R
>      > PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure
>     Team On
>      > call : +91-8939599426
>
>
>     --
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
>
>
>
> --
>
>
>
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On
> call : +91-8939599426


--
Adrian Klaver
adrian.klaver@aklaver.com

--

Thanks,

Prakash.R

PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On call : +91-8939599426

-- 
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt

> and perl DBD also installed but it cant load that is the issue please advise me.

what is the output of LD_LIBRARY_PATH on both dev and prod

Hi Ravi,

Please find the below details,

Dev:

==> echo $LD_LIBRARY_PATH

/usr/lib/perl5/5.10.0/i386-linux-thread-multi/CORE:/opt/rh/rh-perl526/root/usr/lib64:/usr/lib64/perl5/CORE:/opt/edb/languagepack-10/Perl-5.24/lib/CORE

postgres@sydcosausd001.enterprisenet.org:/home/postgres

Prod:

==========

==> echo $LD_LIBRARY_PATH

/usr/lib64/perl5/CORE:/opt/app/PostgreSQL/10/lib:/opt/edb/languagepack-10/Perl-5.24/lib/CORE:/opt/app/PostgreSQL/10/lib

postgres@sydcosafpp001.enterprisenet.org:/home/postgres

On Mon, May 13, 2019 at 3:20 PM Ravi Krishna <ravi_krishna@aol.com> wrote:

> and perl DBD also installed but it cant load that is the issue please advise me.

what is the output of LD_LIBRARY_PATH on both dev and prod

--

Thanks,

Prakash.R

PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On call : +91-8939599426

-- 
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt

On 5/13/19 2:20 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> Yes am using edb standard edition and installed pgbackrest in prod and dev.

To be clear your have on the prod server:

1) Postgres installed via EDB installer.

2) You have pgBackRest.

3) You can take a backup using pgBackRest from the Postgres server.


If that is the case how did you install pgBackRest on the prod server?

In other words what is different between the prod and dev setups of 
Postgres and pgBackRest?

> 
> nothing different am trying to restore the backup prod to dev cluster .
> 
> * Can't load '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for module 
> DBD::Pg: libpq.so.5: cannot open shared object file: No such file or 
> directory at /usr/lib64/perl5/DynaLoader.pm line 190.*
>          at /loader/0x1ec1528/pgBackRest/Db.pm line 10.
>          at /loader/0x1ec1528/pgBackRest/Main.pm line 12.
>          pgBackRest::Main::__ANON__('Can\'t load 
> \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at 
> /usr/share/perl5/vendor_perl/Carp.pm line 100
> 
> May be above path is corrupt or something issue not able to load the path.
> 
> Regards,
> Prakash.R
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/13/19 7:03 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> I did in pgbackrest installation below commands in prod server.
> 
> yum install 
> https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rp 
> <https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-redhat96-9.6-3.noarch.rpm>m
> 
> 
> yum install pgbackrest
> 
> 
> and prod server backup working fine .
> 
> prod and dev file system mount points is different naming conversation.
> 
> so we have taken the restore directly using to prod .

Hmm. I would follow Achilleas's suggestion and do:

ldd /usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so

-- 
Adrian Klaver
adrian.klaver@aklaver.com

>
> ==> ldd /usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so
>         linux-vdso.so.1 =>  (0x00007fffddd8f000)
>         libpq.so.5 => /usr/lib64/perl5/CORE/libpq.so.5 (0x00007f5ecdbd6000)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I may be wrong, but is the above path in the LD_LIBRARY_PATH. Why do you expect it to be picked auto magically.

On 5/13/19 8:14 AM, Prakash Ramakrishnan wrote:
> Hi Ravi,
> 
> I tried manually export the lib path multiple times not working.
> 

> ==> pgbackrest --stanza=A4_sydcosafpp001 --log-level-console=info 
> --db-path=/Postgres/prakash_pgbackrest --delta  --process-max=2 
> --db-include=A4_Copy restore 
> --repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001         
>                                                                          
>             2019-05-13 19:55:05.954 P00   INFO: restore command begin 
> 2.13: --db-include=A4_Copy --delta --log-level-console=info 
> --pg1-path=/Postgres/prakash_pgbackrest --process-max=2 
> --repo1-host=sydcosafpp001.enterprisenet.org 
> <http://sydcosafpp001.enterprisenet.org> 
> --repo1-host-config=/etc/pgbackrest.conf --repo1-host-user=postgres 
> --repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001 
> --stanza=A4_sydcosafpp001
> Password:
> ERROR [124]: : remote process on 'sydcosafpp001.enterprisenet.org 
> <http://sydcosafpp001.enterprisenet.org>' terminated unexpectedly [124]: 

Trying to figure out above:

sydcosausd001.enterprisenet.org is your dev

sydcosafpp001.enterprisenet.org  is your prod

Correct?

You are trying to run the pgBackRest on the dev machine, so why is there 
a remote process failure on the prod?


> ERROR: [124]: process terminated due to an unhandled exception:
> *  Can't load '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for 
> module DBD::Pg: libpq.so.5: cannot open shared object file: No such file 
> or directory at /usr/lib64/perl5/DynaLoader.pm line 190.*
>         at /loader/0xc54528/pgBackRest/Db.pm line 10.
>         at /loader/0xc54528/pgBackRest/Main.pm line 12.
>         pgBackRest::Main::__ANON__('Can\'t load 
> \'/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so\...') called at 
> /usr/share/perl5/vendor_perl/Carp.pm line 100
> 
> 
> Regards,
> Prakash.R
> 
> On Mon, May 13, 2019 at 8:39 PM Ravi Krishna <ravi_krishna@aol.com 
> <mailto:ravi_krishna@aol.com>> wrote:
> 
>      >
>      > ==> ldd /usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so
>      >         linux-vdso.so.1 =>  (0x00007fffddd8f000)
>      >         libpq.so.5 => /usr/lib64/perl5/CORE/libpq.so.5
>     (0x00007f5ecdbd6000)
>                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
>     I may be wrong, but is the above path in the LD_LIBRARY_PATH. Why do
>     you expect it to be picked auto magically.
> 
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

>
> Note - if am taking same prod single database backup and restore in new cluster no use for us and it will take more
time.
> so business and team they need every 3 weeks for restore in dev server one single database and cant we do it in
pg_dumpand restore . 
> They want using pgbackrest tool the db size is huge so tats why am trying remote restore option.

I am baffled.  Are you telling that you are restoring it back on prod (using remote restore option),
which effectively means overwriting prod db.  Also you never gave this information until now. You
should have shared full details.

I have not used pgbackest, but I have read the FAQ. If I am not mistaken it has single db restore option too. So you
canrestore 
the db in dev.

>
>
> Not , am saying we have the daily backup and full backup in prod server only and there is one database like a4 the db
sizeis 1.5TB. 
> so am not restore again in prod .
> Am taking directly single backup restore in dev its means in dev server only restore the database in new cluster.
> existing cluster we cant restore backups.
> so thats why am struggling the issue getting some perl issue and remote option terminated otherwise wont disturb
anyoneand need to solve this issue. 
> this activity has been planned every 3 weeks.
>

Sorry I am not able to follow the above. Did anyone else ?

Which part confused you Ravi?
Here’s my take
> On May 14, 2019, at 4:06 AM, Ravi Krishna <ravi_krishna@aol.com> wrote:
>
>>
>>
>> Not , am saying we have the daily backup and full backup in prod server only and there is one database like a4 the
dbsize is 1.5TB. 
>> so am not restore again in prod .
Production server is fine and generating a new back-up daily.  (Presumably with pgBackrest?)

>> Am taking directly single backup restore in dev its means in dev server only restore the database in new cluster.
>> existing cluster we cant restore backups.
This is the part that I might not understand: a new postgres instance on the same dev server? both old and new failing
torestore at this time?  
Does this imply that dev is the source of the dump being restored (as opposed to using the daily from prod)?

>> so thats why am struggling the issue getting some perl issue and remote option terminated otherwise wont disturb
anyoneand need to solve this issue. 
>> this activity has been planned every 3 weeks.
>>
The hope is to reload dev every three weeks.
>
> Sorry I am not able to follow the above. Did anyone else ?
>
>
>

I wonder if rebuilding dev server (or a different server) from the ground up is an option with all PG related parts
installas per prod? 

-----------------------------------  Original Message  ------------------------------

> On May 14, 2019, at 9:06 AM, Rob Sargent <robjsargent@gmail.com> wrote:
>
>
> Which part confused you Ravi?

Same as you, this one
"Am taking directly single backup restore in dev its means in dev server only restore the database in new cluster
existingcluster we cant restore backups." 

I don't want to second guess and I would prefer all information supplied in one post, rather than in installments.

On 5/14/19 2:51 AM, Prakash Ramakrishnan wrote:
> Hi Ravi,
> 
> Not , am saying we have the daily backup and full backup in prod server 
> only and there is one database like a4 the db size is 1.5TB.
> so am not restore again in prod .
> Am taking directly single backup restore in dev its means in dev server 
> only restore the database in new cluster.
> existing cluster we cant restore backups.
> so thats why am struggling the issue getting some perl issue and remote 
> option terminated otherwise wont disturb anyone and need to solve this 
> issue.
> this activity has been planned every 3 weeks.

I think to make this clearer for everyone a graphical layout might help. 
Something like:

prod Pg db --pgBackRest--> some_host/some_dir/some_file  @daily

dev Pg db <--pgBackRest -- some_host/some_dir/some_file  @3 weeks

Of course the above is just made up.


What I think we know so far, please check and correct as necessary:

1) Prod server CentOS 7, Postgres 10 from EDB installer, pgBackRest from 
PGDG repo

2) Dev server CentOS 7, Postgres 10 from EDB installer, pgBackRest from 
PGDG repo

3) sydcosausd001.enterprisenet.org is your dev

4) sydcosafpp001.enterprisenet.org  is your prod


5) pgBackRest works on prod server.

6) pgBackRest fails on dev server with:

Can't load '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for module 
DBD::Pg: libpq.so.5: cannot open shared object file: No such file or 
directory at /usr/lib64/perl5/DynaLoader.pm line 190.

7) ldd of Pg.so. From dev correct?:

  ldd /usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so
         linux-vdso.so.1 =>  (0x00007fffddd8f000)
         libpq.so.5 => /usr/lib64/perl5/CORE/libpq.so.5 (0x00007f5ecdbd6000)
         libm.so.6 => /lib64/libm.so.6 (0x00007f5ecd8d4000)
         libc.so.6 => /lib64/libc.so.6 (0x00007f5ecd507000)
         libssl.so.1.0.0 => /usr/lib64/perl5/CORE/libssl.so.1.0.0 
(0x00007f5ecd297000)
         libcrypto.so.1.0.0 => /usr/lib64/perl5/CORE/libcrypto.so.1.0.0 
(0x00007f5ecce5d000)
         libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 
(0x00007f5eccc10000)
         libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2 
(0x00007f5ecc9b1000)
         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5ecc795000)
         /lib64/ld-linux-x86-64.so.2 (0x00007f5ece056000)
         libdl.so.2 => /lib64/libdl.so.2 (0x00007f5ecc591000)
         libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f5ecc2a8000)
         libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f5ecc075000)
         libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f5ecbe71000)
         libkrb5support.so.0 => /lib64/libkrb5support.so.0 
(0x00007f5ecbc61000)
         libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f5ecba5d000)
         libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5ecb844000)
         liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f5ecb635000)
         libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f5ecb418000)
         libssl.so.10 => /lib64/libssl.so.10 (0x00007f5ecb1a6000)
         libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f5ecad45000)
         libssl3.so => /lib64/libssl3.so (0x00007f5ecaaf3000)
         libsmime3.so => /lib64/libsmime3.so (0x00007f5eca8cc000)
         libnss3.so => /lib64/libnss3.so (0x00007f5eca59f000)
         libnssutil3.so => /lib64/libnssutil3.so (0x00007f5eca370000)
         libplds4.so => /lib64/libplds4.so (0x00007f5eca16c000)
         libplc4.so => /lib64/libplc4.so (0x00007f5ec9f67000)
         libnspr4.so => /lib64/libnspr4.so (0x00007f5ec9d29000)
         libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f5ec9b02000)
         libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f5ec98cb000)
         libz.so.1 => /lib64/libz.so.1 (0x00007f5ec96b5000)
         librt.so.1 => /lib64/librt.so.1 (0x00007f5ec94ad000)
         libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f5ec924b000)
         libfreebl3.so => /lib64/libfreebl3.so (0x00007f5ec9048000)
postgres@sydcosausd001.enterprisenet.org:/home/postgres
> 
> Regards,
> Prakash.R
> 
> On Tue, May 14, 2019 at 3:10 PM Ravi Krishna <ravi_krishna@aol.com 
> <mailto:ravi_krishna@aol.com>> wrote:
> 
>      >
>      > Note - if am taking same prod single database backup and restore
>     in new cluster no use for us and it will take more time.
>      > so business and team they need every 3 weeks for restore in dev
>     server one single database and cant we do it in pg_dump and restore .
>      > They want using pgbackrest tool the db size is huge so tats why
>     am trying remote restore option.
> 
>     I am baffled.  Are you telling that you are restoring it back on
>     prod (using remote restore option),
>     which effectively means overwriting prod db.  Also you never gave
>     this information until now. You
>     should have shared full details.
> 
>     I have not used pgbackest, but I have read the FAQ. If I am not
>     mistaken it has single db restore option too. So you can restore
>     the db in dev.
> 
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/14/19 7:28 AM, Prakash Ramakrishnan wrote:
> Yes, that is correct.

How about graphical layout of your backup/restore procedure?

> 
> On Tue, May 14, 2019 at 7:54 PM Adrian Klaver <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 5/14/19 2:51 AM, Prakash Ramakrishnan wrote:
>      > Hi Ravi,
>      >
>      > Not , am saying we have the daily backup and full backup in prod
>     server
>      > only and there is one database like a4 the db size is 1.5TB.
>      > so am not restore again in prod .
>      > Am taking directly single backup restore in dev its means in dev
>     server
>      > only restore the database in new cluster.
>      > existing cluster we cant restore backups.
>      > so thats why am struggling the issue getting some perl issue and
>     remote
>      > option terminated otherwise wont disturb anyone and need to solve
>     this
>      > issue.
>      > this activity has been planned every 3 weeks.
> 
>     I think to make this clearer for everyone a graphical layout might
>     help.
>     Something like:
> 
>     prod Pg db --pgBackRest--> some_host/some_dir/some_file  @daily
> 
>     dev Pg db <--pgBackRest -- some_host/some_dir/some_file  @3 weeks
> 
>     Of course the above is just made up.
> 
> 
>     What I think we know so far, please check and correct as necessary:
> 
>     1) Prod server CentOS 7, Postgres 10 from EDB installer, pgBackRest
>     from
>     PGDG repo
> 
>     2) Dev server CentOS 7, Postgres 10 from EDB installer, pgBackRest from
>     PGDG repo
> 
>     3) sydcosausd001.enterprisenet.org
>     <http://sydcosausd001.enterprisenet.org> is your dev
> 
>     4) sydcosafpp001.enterprisenet.org
>     <http://sydcosafpp001.enterprisenet.org>  is your prod
> 
> 
>     5) pgBackRest works on prod server.
> 
>     6) pgBackRest fails on dev server with:
> 
>     Can't load '/usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so' for module
>     DBD::Pg: libpq.so.5: cannot open shared object file: No such file or
>     directory at /usr/lib64/perl5/DynaLoader.pm line 190.
> 
>     7) ldd of Pg.so. From dev correct?:
> 
>        ldd /usr/lib64/perl5/vendor_perl/auto/DBD/Pg/Pg.so
>               linux-vdso.so.1 =>  (0x00007fffddd8f000)
>               libpq.so.5 => /usr/lib64/perl5/CORE/libpq.so.5
>     (0x00007f5ecdbd6000)
>               libm.so.6 => /lib64/libm.so.6 (0x00007f5ecd8d4000)
>               libc.so.6 => /lib64/libc.so.6 (0x00007f5ecd507000)
>               libssl.so.1.0.0 => /usr/lib64/perl5/CORE/libssl.so.1.0.0
>     (0x00007f5ecd297000)
>               libcrypto.so.1.0.0 =>
>     /usr/lib64/perl5/CORE/libcrypto.so.1.0.0
>     (0x00007f5ecce5d000)
>               libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2
>     (0x00007f5eccc10000)
>               libldap_r-2.4.so.2 => /lib64/libldap_r-2.4.so.2
>     (0x00007f5ecc9b1000)
>               libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5ecc795000)
>               /lib64/ld-linux-x86-64.so.2 (0x00007f5ece056000)
>               libdl.so.2 => /lib64/libdl.so.2 (0x00007f5ecc591000)
>               libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f5ecc2a8000)
>               libk5crypto.so.3 => /lib64/libk5crypto.so.3
>     (0x00007f5ecc075000)
>               libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f5ecbe71000)
>               libkrb5support.so.0 => /lib64/libkrb5support.so.0
>     (0x00007f5ecbc61000)
>               libkeyutils.so.1 => /lib64/libkeyutils.so.1
>     (0x00007f5ecba5d000)
>               libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5ecb844000)
>               liblber-2.4.so.2 => /lib64/liblber-2.4.so.2
>     (0x00007f5ecb635000)
>               libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f5ecb418000)
>               libssl.so.10 => /lib64/libssl.so.10 (0x00007f5ecb1a6000)
>               libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f5ecad45000)
>               libssl3.so => /lib64/libssl3.so (0x00007f5ecaaf3000)
>               libsmime3.so => /lib64/libsmime3.so (0x00007f5eca8cc000)
>               libnss3.so => /lib64/libnss3.so (0x00007f5eca59f000)
>               libnssutil3.so => /lib64/libnssutil3.so (0x00007f5eca370000)
>               libplds4.so => /lib64/libplds4.so (0x00007f5eca16c000)
>               libplc4.so => /lib64/libplc4.so (0x00007f5ec9f67000)
>               libnspr4.so => /lib64/libnspr4.so (0x00007f5ec9d29000)
>               libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f5ec9b02000)
>               libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f5ec98cb000)
>               libz.so.1 => /lib64/libz.so.1 (0x00007f5ec96b5000)
>               librt.so.1 => /lib64/librt.so.1 (0x00007f5ec94ad000)
>               libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f5ec924b000)
>               libfreebl3.so => /lib64/libfreebl3.so (0x00007f5ec9048000)
>     postgres@sydcosausd001.enterprisenet.org:/home/postgres
>      >
>      > Regards,
>      > Prakash.R
>      >
>      > On Tue, May 14, 2019 at 3:10 PM Ravi Krishna
>     <ravi_krishna@aol.com <mailto:ravi_krishna@aol.com>
>      > <mailto:ravi_krishna@aol.com <mailto:ravi_krishna@aol.com>>> wrote:
>      >
>      >      >
>      >      > Note - if am taking same prod single database backup and
>     restore
>      >     in new cluster no use for us and it will take more time.
>      >      > so business and team they need every 3 weeks for restore
>     in dev
>      >     server one single database and cant we do it in pg_dump and
>     restore .
>      >      > They want using pgbackrest tool the db size is huge so
>     tats why
>      >     am trying remote restore option.
>      >
>      >     I am baffled.  Are you telling that you are restoring it back on
>      >     prod (using remote restore option),
>      >     which effectively means overwriting prod db.  Also you never gave
>      >     this information until now. You
>      >     should have shared full details.
>      >
>      >     I have not used pgbackest, but I have read the FAQ. If I am not
>      >     mistaken it has single db restore option too. So you can restore
>      >     the db in dev.
>      >
>      >
>      >
>      >
>      > --
>      >
>      >
>      >
>      > Thanks,
>      > Prakash.R
>      > PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure
>     Team On
>      > call : +91-8939599426
> 
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/14/19 7:40 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> Prod server retention periods ,
> 
> [global]
> repo1-path=/pgBACKUP/A4_sydcosafpp001
> *retention-diff=3*
> *retention-full=1*
> *retention-archive=2*
> start-fast=y
> process-max=12
> archive-async=y
> 
> Dev server currently we dont have only restore need to do prod to dev.

The issue is we are having trouble figuring out how the backups are 
getting from one location to another and in what order. So could you 
layout that out like I showed earlier:

prod Pg db --pgBackRest--> some_host/some_dir/some_file  @daily

dev Pg db <--pgBackRest -- some_host/some_dir/some_file  @3 weeks

> 
> Regards,
> Prakash.R
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com

On 5/15/19 2:37 AM, Prakash Ramakrishnan wrote:
> Hi Adrian,
> 
> I don't much about pgbackrest i think it will take backup from archives 
> Now only one the issue means perl cant load.

I don't use pgBackRest so I am speculating now, but I believe your issue 
is due to mixing local and remote operations. From a previous post:

[global]
repo1-host=sydcosafpp001.enterprisenet.org
repo1-host-user=postgres
repo1-host-config=/etc/pgbackrest.conf
repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001

[A4_sydcosafpp001]
pg1-path=/Postgres/prakash_pgbackrest

postgres(at)sydcosausd001(dot)enterprisenet(dot)org:/home/postgres

restore command - pgbackrest --stanza=A4_sydcosafpp001
--log-level-console=info --db-path=/Postgres/prakash_pgbackrest --delta
--process-max=2 --db-include=A4_Copy restore
--repo1-path=/pgBACKUP/A4_sydcosafpp001/backup/A4_sydcosafpp001

Looks to me are trying to use both a local(repo1-path) and 
remote(repo1-host*) repo at the same time. Given that the pgBackRest/EDB 
Postgres combination works on the prod server I would try doing a local 
only backup on the dev machine of the dev Postgres instance using the 
dev pgBackRest. This will help confirm whether there is a Perl issue on 
the dev machine or not.

> 
> Regards,
> Prakash.R
> 
> On Tue, May 14, 2019 at 8:15 PM Adrian Klaver <adrian.klaver@aklaver.com 
> <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 5/14/19 7:40 AM, Prakash Ramakrishnan wrote:
>      > Hi Adrian,
>      >
>      > Prod server retention periods ,
>      >
>      > [global]
>      > repo1-path=/pgBACKUP/A4_sydcosafpp001
>      > *retention-diff=3*
>      > *retention-full=1*
>      > *retention-archive=2*
>      > start-fast=y
>      > process-max=12
>      > archive-async=y
>      >
>      > Dev server currently we dont have only restore need to do prod to
>     dev.
> 
>     The issue is we are having trouble figuring out how the backups are
>     getting from one location to another and in what order. So could you
>     layout that out like I showed earlier:
> 
>     prod Pg db --pgBackRest--> some_host/some_dir/some_file  @daily
> 
>     dev Pg db <--pgBackRest -- some_host/some_dir/some_file  @3 weeks
> 
>      >
>      > Regards,
>      > Prakash.R
>      >
> 
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 
> 
> 
> -- 
> 
> 
> 
> Thanks,
> Prakash.R
> PostgreSQL - Offshore DBA support TCS / Nielsen Infrastructure Team On 
> call : +91-8939599426


-- 
Adrian Klaver
adrian.klaver@aklaver.com