Thread: BUG #7794: pg_dump: errors when using pipes/streams

BUG #7794: pg_dump: errors when using pipes/streams

From
s.reiser@tu-bs.de
Date:
The following bug has been logged on the website:

Bug reference:      7794
Logged by:          Stefan Reiser
Email address:      s.reiser@tu-bs.de
PostgreSQL version: 9.1.7
Operating system:   Windows 7 Pro 64-Bit
Description:        =


I intend to stream the output of pg_dump into a zip file (along with some
other files and comments).

pg_dump shows one or more warnings when used with non-seekable streams:
"pg_dump: [custom archiver] WARNING: ftell mismatch with expected position
-- ftell used"

Examples (on Windows 7 Pro 64 Bit, PG 9.1.7 and 9.1.6, 64 Bit)

1) pg_dump --format=3Dcustom --schema-only mydatabase | zip -0 TEST.ZIP -
2) pg_dump --format=3Dcustom --schema-only mydatabase > NUL
3) pg_dump --format=3Dcustom --schema-only --file TEST3 mydatabase
4) pg_dump --format=3Dcustom --schema-only mydatabase > TEST4

1 and 2 print warnings, while pg_dump's exit code is 0 (as can be verified
with example 2),
3 and 4 are ok.

I've used "--schema-only" to emphasize that it's not a question of file size
(without "--schema-only" the number of warnings increases, supposedly one
per table).
The "zip" in example 1 is Info-ZIP 3.0 - but you can as well use anything
that consumes STDIN, like gzip, ...)

Files TEST3 and TEST4 are of the same size, the file packaged in TEST.ZIP is
slightly bigger.
With all three generated files pg_restore produces exactly the same script,
so the backup seems ok - but why the warnings then? (And what do they mean,
anyway?) Are those backups reliable or could they as well become corrupted?
(after all, it seems pg_dump wants to write something somewhere for some
reason, but can't...)

A similar problem has been reported as "BUG #6726"
(http://archives.postgresql.org/pgsql-bugs/2012-07/msg00074.php) but I
couldn't find any replies ... so I filed this new report.
(Formats tar and plain don't seem to produce these warnings, so I'll work
around the problem by using the tar format.)

Re: BUG #7794: pg_dump: errors when using pipes/streams

From
Tom Lane
Date:
s.reiser@tu-bs.de writes:
> pg_dump shows one or more warnings when used with non-seekable streams:
> "pg_dump: [custom archiver] WARNING: ftell mismatch with expected position
> -- ftell used"

> Examples (on Windows 7 Pro 64 Bit, PG 9.1.7 and 9.1.6, 64 Bit)

> 1) pg_dump --format=custom --schema-only mydatabase | zip -0 TEST.ZIP -

Hm.  Apparently, pg_dump's checkSeek() function is succeeding even
though later attempts to seek the file don't work.  We had a previous
go-round with issues of this sort,
http://archives.postgresql.org/pgsql-hackers/2010-06/msg01355.php
but apparently Windows is "helpful" enough to let a no-op SEEK_SET
call succeed too.  (Gee thanks, Microsoft.)

Anyone know how to test for seekability of a file in a way that
works reliably on Windows?

> With all three generated files pg_restore produces exactly the same script,
> so the backup seems ok - but why the warnings then?

My guess is that it's failing to update the table of contents on
completion, which will cause parallel pg_restore to fail, but it's
harmless for ordinary pg_restore.  If we fix checkSeek the warnings
would go away, but you'd still have an archive that's not usable
for parallel restores.

            regards, tom lane