Thread: BUG #8097: pg_dump incomplete data output

BUG #8097: pg_dump incomplete data output

From
wouter.boasson@rivm.nl
Date:
The following bug has been logged on the website:

Bug reference:      8097
Logged by:          Wouter Boasson
Email address:      wouter.boasson@rivm.nl
PostgreSQL version: 9.1.9
Operating system:   Scientific Linux 6.1 32bit
Description:        =


pg_dump produces incomplete data dumps for large tables in directory mode
(-Fd).
The maximum output file size seems to be 2147483647 bytes (2GB), when
setting --compress=3D0 the output seems to be complete (uncompressed size 5=
GB
and 20GB).
When making the dump, no error is displayed, which makes this a dangerous
bug in disaster recovery scenario's, only when restoring error's are
displayed (incomplete/no data).

Re: BUG #8097: pg_dump incomplete data output

From
Tom Lane
Date:
wouter.boasson@rivm.nl writes:
> pg_dump produces incomplete data dumps for large tables in directory mode
> (-Fd).
> The maximum output file size seems to be 2147483647 bytes (2GB), when
> setting --compress=0 the output seems to be complete (uncompressed size 5GB
> and 20GB).

I can't replicate that.  I surmise that you're running on a platform
where largefile support isn't default (and configure failed to figure
out how to turn it on).  Or possibly you're looking at a configuration
bug in zlib.  I don't see anything in the pg_dump sources that would
particularly constrain the size of an output file.

> When making the dump, no error is displayed, which makes this a dangerous
> bug in disaster recovery scenario's, only when restoring error's are
> displayed (incomplete/no data).

It does look like pg_dump might not notice errors reported while
writing.  But that might or might not have helped you here --- it's
not clear exactly which component is failing.

            regards, tom lane