Re: pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists - Mailing list pgsql-general

From Tom Lane
Subject Re: pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists
Date
Msg-id 24468.1436278339@sss.pgh.pa.us
Whole thread Raw
In response to Re: pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists  (John McKown <john.archie.mckown@gmail.com>)
Responses Re: pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists
List pgsql-general
John McKown <john.archie.mckown@gmail.com> writes:
> On Tue, Jul 7, 2015 at 4:26 AM, pinker <pinker@onet.eu> wrote:
>> I made mistake in a filename in pg_dump command, i.e. have used path from
>> another server, which not exists on this one. pg_dump instead of checking
>> permissions / existence of output file first dumped the whole database and
>> at the end (after some time ... ) threw an error:
>>
>> (...)
>> pg_dump: saving database definition
>> pg_dump: [archiver] could not open output file
>> "/home/.../dum-...._20150707_1059.sql": No such file or directory
>>
>> Is it correct behavior? Why wasting so much time and resources leaving
>> checking the output file at the last moment?

> ​What version of PostgreSQL? What OS? What was the command line? On Linux
> x86_64, Fedora 22​, PostgreSQL version 9.4.4, I did:
> pg_dump -f /junk/x tsh009
> and, almost immediately, got back:
> pg_dump: [archiver] could not open output file "/junk/x": No such file or
> directory

There is a noticeable delay if you dump a database with a significant
number of objects in it, eg with the regression test database I get:

$ time pg_dump -f /z/q regression
pg_dump: [archiver] could not open output file "/z/q": No such file or directory

real    0m1.164s
user    0m0.062s
sys     0m0.034s

However, I don't see that with any of the non-plain-text output formats:

$ time pg_dump -f /z/q -Fc regression
pg_dump: [custom archiver] could not open output file "/z/q": No such file or directory

real    0m0.005s
user    0m0.004s
sys     0m0.001s

The reason the non-plain-text output formats fail promptly is that they
try to open the output file in InitArchiveFmt_Custom and siblings.
But plain-text format does not, leaving it until RestoreArchive; which is
code that is shared with pg_restore.  I'm not sure how painful that would
be to fix without breaking pg_restore, but almost certainly it would
involve duplicating the file open/close logic into two different places.
Dunno that this problem is significant enough to justify such effort.
You would need an awful lot of objects (not data, but DDL objects) for
the delay to be major.

            regards, tom lane


pgsql-general by date:

Previous
From: pinker
Date:
Subject: Re: pg_dump (PostgreSQL) 9.4.1 - delay in checking if file exists
Next
From: Filipe Pina
Date:
Subject: Re: database-level lockdown