Thread: pg_dump error - out of memory, Failed on request of size 536870912

Hello,

We have a bit of a problem with the daily db backup. At the moment the
backup db size is half of the main db, and we have the following errors
in the log file:

ERROR: out of memory
DETAIL: Failed on request of size 536870912

(There's a bunch of those.)

There were some 'permission denied for language c' errors earlier in the
log, but then the backup continued for a while.

This is the command we're running:

pg_dump -c -h $DB_HOST -U $DB_USER $DB_NAME | PGOPTIONS='-c
maintenance_work_mem=1500MB -c sort_mem=64MB'
psql --quiet -U $DB_DUMP_USER -h $DB_DUMP_HOST $DB_DUMP_NAME;

Both databases (main and backup) are Postgres 8.3.

I'd appreciate any suggestions on how to fix this problem. Also, if you
have a better idea on running a daily backup please let me know.
Thanks!

Marcin

Hello,

I forgot to add - all those 'out of memory' errors happen when backup db
is trying to create index. Every 'CREATE INDEX' operation is followed by
'out of memory' error.
Thanks!

Marcin


Marcin Citowicki wrote:
> Hello,
>
> We have a bit of a problem with the daily db backup. At the moment the
> backup db size is half of the main db, and we have the following
> errors in the log file:
>
> ERROR: out of memory
> DETAIL: Failed on request of size 536870912
>
> (There's a bunch of those.)
>
> There were some 'permission denied for language c' errors earlier in
> the log, but then the backup continued for a while.
>
> This is the command we're running:
>
> pg_dump -c -h $DB_HOST -U $DB_USER $DB_NAME | PGOPTIONS='-c
> maintenance_work_mem=1500MB -c sort_mem=64MB'
> psql --quiet -U $DB_DUMP_USER -h $DB_DUMP_HOST $DB_DUMP_NAME;
>
> Both databases (main and backup) are Postgres 8.3.
>
> I'd appreciate any suggestions on how to fix this problem. Also, if
> you have a better idea on running a daily backup please let me know.
> Thanks!
>
> Marcin

Hello

you have some data files broken.

try to use http://svana.org/kleptog/pgsql/pgfsck.html - but DO
DATABASE CLUSTER FILES BACKUP BEFORE. pgfsck doesn't support 8.3, but
it can help you to search currupted rows. Then you have remove these
rows, thats all. Later check your hardware - probably your server has
problems with memory or controller.

regards
Pavel Stehule


2008/8/6 Marcin Citowicki <marcin.citowicki@m4n.nl>:
> Hello,
>
> I forgot to add - all those 'out of memory' errors happen when backup db is
> trying to create index. Every 'CREATE INDEX' operation is followed by 'out
> of memory' error.
> Thanks!
>
> Marcin
>
>
> Marcin Citowicki wrote:
>>
>> Hello,
>>
>> We have a bit of a problem with the daily db backup. At the moment the
>> backup db size is half of the main db, and we have the following errors in
>> the log file:
>>
>> ERROR: out of memory
>> DETAIL: Failed on request of size 536870912
>>
>> (There's a bunch of those.)
>>
>> There were some 'permission denied for language c' errors earlier in the
>> log, but then the backup continued for a while.
>>
>> This is the command we're running:
>>
>> pg_dump -c -h $DB_HOST -U $DB_USER $DB_NAME | PGOPTIONS='-c
>> maintenance_work_mem=1500MB -c sort_mem=64MB'
>> psql --quiet -U $DB_DUMP_USER -h $DB_DUMP_HOST $DB_DUMP_NAME;
>>
>> Both databases (main and backup) are Postgres 8.3.
>>
>> I'd appreciate any suggestions on how to fix this problem. Also, if you
>> have a better idea on running a daily backup please let me know.
>> Thanks!
>>
>> Marcin
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
>

Marcin Citowicki wrote:
> Hello,
>
> I forgot to add - all those 'out of memory' errors happen when backup db
> is trying to create index. Every 'CREATE INDEX' operation is followed by
> 'out of memory' error.

are you sure that your OS (or ulimit) is able to support a
maintenance_work_setting that large ? - try reducing to a say 128MB for
a start and try again.


Stefan