Re: pg_dump with 1100 schemas being a bit slow - Mailing list pgsql-general

From Loic d'Anterroches
Subject Re: pg_dump with 1100 schemas being a bit slow
Date
Msg-id 8e2f2cb20910070920l1b63cfdek7391a9480dda73e3@mail.gmail.com
Whole thread Raw
In response to Re: pg_dump with 1100 schemas being a bit slow  (Bill Moran <wmoran@potentialtech.com>)
List pgsql-general
On Wed, Oct 7, 2009 at 5:54 PM, Bill Moran <wmoran@potentialtech.com> wrote:
> In response to "Loic d'Anterroches" <diaeresis@gmail.com>:
>
>> On Wed, Oct 7, 2009 at 4:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> > "Loic d'Anterroches" <diaeresis@gmail.com> writes:
>> >> Each night I am running:
>> >> pg_dump --blobs --schema=%s --no-acl -U postgres indefero | gzip >
>> >> /path/to/backups/%s/%s-%s.sql.gz
>> >> this for each installation, so 1100 times. Substitution strings are to
>> >> timestamp and get the right schema.
>
> Have you tested the speed without the gzip?

This is the first thing I did but in that case I was not able to get
any significant improvement. The data to gzip is very small "per
schema" so this is not the bottleneck.

> We found that compressing the dump takes considerably longer than pg_dump
> does, but pg_dump can't release its locks until gzip has completely
> processed all of the data, because of the pipe.

Good tip, I keep that in mind for the future!

Thanks,
loïc

--
Loïc d'Anterroches - Céondo Ltd - http://www.ceondo.com

pgsql-general by date:

Previous
From: "Massa, Harald Armin"
Date:
Subject: Re: pg_dump with 1100 schemas being a bit slow
Next
From: "Loic d'Anterroches"
Date:
Subject: Re: pg_dump with 1100 schemas being a bit slow