Re: parallel dump fails to dump large tables - Mailing list pgsql-general

From Shanker Singh
Subject Re: parallel dump fails to dump large tables
Date
Msg-id 961471F4049EF94EAD4D0165318BD881625906EF@Corp-MBXE3.iii.com
Whole thread Raw
In response to Re: parallel dump fails to dump large tables  (Sterfield <sterfield@gmail.com>)
List pgsql-general

Hi,

There is no 2nd server involved. The pg_dump runs on the server where postgres server is running. The regular dump runs the same way and it completes

Successfully within 3 hours. It’s only parallel dump fails on the table with size about 48GB. We run the pg_dump through an script on the postgresql server with

Command similar to “nohup ./postages_dumpdb &”;

 

Thanks

shanker

 

From: Sterfield [mailto:sterfield@gmail.com]
Sent: Sunday, February 22, 2015 8:50 AM
To: Shanker Singh
Cc: Tom Lane; rod@iol.ie; pgsql-general@postgresql.org
Subject: Re: [GENERAL] parallel dump fails to dump large tables

 

 

 

2015-02-20 14:26 GMT-08:00 Shanker Singh <ssingh@iii.com>:

I tried turning off ssl renegotiation by setting "ssl_renegotiation_limit = 0" in postgresql.conf but it had no effect. The parallel dump still fails on large tables consistently.

Thanks
Shanker

 

HI,

Maybe you could try to setup an SSH connection between the two servers, with a keepalive option, and left it open for a long time (at least the duration of your backup), just to test if your ssh connection is still being cut after some time.

That way, you will be sure if the problem is related to SSH or related to Postgresql.

Thanks,

Guillaume

 

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Saturday, February 14, 2015 9:00 AM
To: rod@iol.ie
Cc: Shanker Singh; pgsql-general@postgresql.org
Subject: Re: [GENERAL] parallel dump fails to dump large tables

"Raymond O'Donnell" <rod@iol.ie> writes:
> On 14/02/2015 15:42, Shanker Singh wrote:
>> Hi,
>> I am having problem using parallel pg_dump feature in postgres
>> release 9.4. The size of the table is large(54GB). The dump fails
>> with the
>> error: "pg_dump: [parallel archiver] a worker process died
>> unexpectedly". After this error the pg_dump aborts. The error log
>> file gets the following message:
>>
>> 2015-02-09 15:22:04 PST [8636]: [2-1]
>> user=pdroot,db=iii,appname=pg_dump
>> STATEMENT:  COPY iiirecord.varfield (id, field_type_tag, marc_tag,
>> marc_ind1, marc_ind2, field_content, field_group_id, occ_num,
>> record_id) TO stdout;
>> 2015-02-09 15:22:04 PST [8636]: [3-1]
>> user=pdroot,db=iii,appname=pg_dump
>> FATAL:  connection to client lost

> There's your problem - something went wrong with the network.

I'm wondering about SSL renegotiation failures as a possible cause of the disconnect --- that would explain why it only happens on large tables.

                        regards, tom lane


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

 

pgsql-general by date:

Previous
From: Sterfield
Date:
Subject: Re: parallel dump fails to dump large tables
Next
From: Eric Hanson
Date:
Subject: express composite type literal as text