Re: pg_dump large-file support > 16GB - Mailing list pgsql-general

From Rafael Martinez Guerrero
Subject Re: pg_dump large-file support > 16GB
Date
Msg-id 1111070767.6434.123.camel@bbking.uio.no
Whole thread Raw
In response to Re: pg_dump large-file support > 16GB  (Lonni J Friedman <netllama@gmail.com>)
Responses Re: pg_dump large-file support > 16GB  (Marco Colombo <pgsql@esiway.net>)
List pgsql-general
On Thu, 2005-03-17 at 15:09, Lonni J Friedman wrote:
> On Thu, 17 Mar 2005 14:05:35 +0100, Rafael Martinez Guerrero
> <r.m.guerrero@usit.uio.no> wrote:
> > Hello
> >
> > We are having problems with pg_dump.
> >
> > We are trying to dump a 30GB+ database using pg_dump with the --file
> > option. In the beginning everything works fine, pg_dump runs and we get
> > a dumpfile. But when this file becomes 16GB it disappears from the
> > filesystem, pg_dump continues working without giving an error until it
> > finnish (even when the file does not exist)(The filesystem has free
> > space).
> >
> > I can generate without problems files bigger than 16GB with other
> > programs.
> >
> > Some information:
> > ---------------------------
> > OS: Red Hat Enterprise Linux WS release 3 (Taroon Update 4)
> > Kernel: 2.4.21-27.0.2.ELsmp #1 SMP i686
> > PG: 7.4.7
> >
> > LVM version 1.0.8-2(26/05/2004)
> > EXT3 FS 2.4-0.9.19, 19 August 2002 on lvm(58,6), internal journal
> > EXT3-fs: mounted filesystem with ordered data mode.
> > ----------------------------
> >
> > Any ideas? It looks like pg_dump has a limit of 16GB? How can we solve
> > this?
> >
>
> Have you tried piping the dump through split so that each file's size
> is limited?

It is a possibility. But when you have several hundred databases, the
amount of backup files you have to take care of, grow in a exponencial
form ;).

We already have several well tested backup/maintenace/restore scripts
that we would like to continue using.

My question is why is this limit (16GB) there, when my OS does not have
that limit? Is it possible to take it away in a easy way? It looks like
pg_dump is compiled with large-file support because it can work with
files bigger than 4GB.

More ideas? :)

Regards.
--
Rafael Martinez, <r.m.guerrero@usit.uio.no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/

Attachment

pgsql-general by date:

Previous
From: Rafael Martinez Guerrero
Date:
Subject: Re: pg_dump large-file support > 16GB
Next
From: Scott Marlowe
Date:
Subject: Re: rserv question