Home > mailing lists
BUG #2744: pg_dump, ERROR: out of memory, Failed on request of size 270344621 - Mailing list pgsql-bugs

From	aTz
Subject	BUG #2744: pg_dump, ERROR: out of memory, Failed on request of size 270344621
Date	November 10, 2006 05:13:02
Msg-id	200611081815.kA8IF7Oh006822@wwwmaster.postgresql.org Whole thread Raw
List	pgsql-bugs
Tree view
The following bug has been logged online:

Bug reference:      2744
Logged by:          aTz
Email address:      atz@mail.ru
PostgreSQL version: 8.1.5 (Release)
Operating system:   FreeBSD 6.1-RELEASE
Description:        pg_dump, ERROR: out of memory, Failed on request of size
270344621
Details:

Hello!
Sorry for english.

I have a database with table "documents", that have column type "bytea".
Database size is ~1.5 GB (99% of objects is images, documents, pdf's in
bytea fields, middle size of each object ~ 20Mb).

Database initialized: initdb --encoding=UTF8 --locale=ru_RU.UTF-8

SysV kernel:
kern.ipc.msgmax: 16384
kern.ipc.msgmni: 40
kern.ipc.msgmnb: 2048
kern.ipc.msgtql: 40
kern.ipc.msgssz: 8
kern.ipc.msgseg: 2048
kern.ipc.semmap: 128
kern.ipc.semmni: 64
kern.ipc.semmns: 1024
kern.ipc.semmnu: 30
kern.ipc.semmsl: 128
kern.ipc.semopm: 100
kern.ipc.semume: 10
kern.ipc.semusz: 92
kern.ipc.semvmx: 32767
kern.ipc.semaem: 16384
kern.ipc.shmmax: 268435456
kern.ipc.shmmin: 1
kern.ipc.shmmni: 128
kern.ipc.shmseg: 64
kern.ipc.shmall: 65536

When i try to dump database:
pg_dump --encoding=WIN1251 -U postgres docs > docs.sql

It failed with message:
=======================
pg_dump: ERROR:  out of memory
DETAIL:  Failed on request of size 270344621.
pg_dump: SQL-ÐºÐ¾Ð¼Ð°Ð½Ð´Ð° Ð¿Ð¾ Ð²ÑÐ³ÑÑÐ·ÐºÐµ ÑÐ¾Ð´ÐµÑÐ¶Ð¸Ð¼Ð¾Ð³Ð¾
ÑÐ°Ð±Ð»Ð¸ÑÑ "documents" Ð½Ðµ ÑÐ´Ð°Ð»Ð°ÑÑ: PQendcopy()
Ð²ÑÐ¿Ð¾Ð»Ð½Ð¸Ð»Ð°ÑÑ Ð½ÐµÐ½Ð¾ÑÐ¼Ð°Ð»ÑÐ½Ð¾.
pg_dump: ÐÐ¾Ð¼Ð°Ð½Ð´Ð° Ð±ÑÐ»Ð°: COPY public.documents (id, project_id,
doc_group_id, is_archive, title, status, version, modified, description,
filedata, filename, filet
ype, filesize) TO stdout;

In log file:
============
TopMemoryContext: 65176 total in 6 blocks; 4544 free (15 chunks); 60632
used
Record information cache: 8192 total in 1 blocks; 1864 free (0 chunks); 6328
used
Type information cache: 8192 total in 1 blocks; 1864 free (0 chunks); 6328
used
Operator class cache: 8192 total in 1 blocks; 4936 free (0 chunks); 3256
used
TopTransactionContext: 8192 total in 1 blocks; 7496 free (3 chunks); 696
used
MessageContext: 8192 total in 1 blocks; 5480 free (1 chunks); 2712 used
CFuncHash: 8192 total in 1 blocks; 4936 free (0 chunks); 3256 used
smgr relation table: 8192 total in 1 blocks; 1840 free (0 chunks); 6352
used
Portal hash: 8192 total in 1 blocks; 3912 free (0 chunks); 4280 used
PortalMemory: 8192 total in 1 blocks; 8040 free (0 chunks); 152 used
PortalHeapMemory: 67206196 total in 7 blocks; 29936 free (22 chunks);
67176260 used
COPY TO: 90785952 total in 3 blocks; 7240 free (6 chunks); 90778712 used
Relcache by OID: 8192 total in 1 blocks; 3376 free (0 chunks); 4816 used
CacheMemoryContext: 1040384 total in 7 blocks; 294304 free (1 chunks);
746080 used
pg_toast_16443_index: 1024 total in 1 blocks; 328 free (0 chunks); 696 used
pg_description_o_c_o_index: 1024 total in 1 blocks; 256 free (0 chunks); 768
used
pg_database_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_largeobject_loid_pn_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_trigger_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_trigger_tgconstrrelid_index: 1024 total in 1 blocks; 392 free (0 chunks);
632 used
pg_trigger_tgconstrname_index: 1024 total in 1 blocks; 392 free (0 chunks);
632 used
pg_attrdef_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_cast_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_rewrite_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_attrdef_adrelid_adnum_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_depend_reference_index: 1024 total in 1 blocks; 256 free (0 chunks); 768
used
pg_depend_depender_index: 1024 total in 1 blocks; 256 free (0 chunks); 768
used
pg_tablespace_spcname_index: 1024 total in 1 blocks; 392 free (0 chunks);
632 used
pg_tablespace_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_constraint_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_constraint_contypid_index: 1024 total in 1 blocks; 392 free (0 chunks);
632 used
pg_constraint_conrelid_index: 1024 total in 1 blocks; 392 free (0 chunks);
632 used
pg_constraint_conname_nsp_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_index_indrelid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_roles: 7168 total in 3 blocks; 2824 free (0 chunks); 4344 used
pg_type_typname_nsp_index: 1024 total in 1 blocks; 328 free (0 chunks); 696
used
pg_type_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_trigger_tgrelid_tgname_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_statistic_relid_att_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_auth_members_member_role_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_auth_members_role_member_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_rewrite_rel_rulename_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_proc_proname_args_nsp_index: 1024 total in 1 blocks; 256 free (0 chunks);
768 used
pg_proc_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_operator_oprname_l_r_n_index: 1024 total in 1 blocks; 192 free (0
chunks); 832 used
pg_operator_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_opclass_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_opclass_am_name_nsp_index: 1024 total in 1 blocks; 256 free (0 chunks);
768 used
pg_namespace_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_namespace_nspname_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_language_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_language_name_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_inherits_relid_seqno_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_index_indexrelid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_authid_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_authid_rolname_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_database_datname_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_conversion_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
pg_conversion_name_nsp_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_conversion_default_index: 1024 total in 1 blocks; 192 free (0 chunks);
832 used
pg_class_relname_nsp_index: 1024 total in 1 blocks; 328 free (0 chunks); 696
used
pg_class_oid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632 used
pg_cast_source_target_index: 1024 total in 1 blocks; 328 free (0 chunks);
696 used
pg_attribute_relid_attnum_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_attribute_relid_attnam_index: 1024 total in 1 blocks; 328 free (0
chunks); 696 used
pg_amproc_opc_proc_index: 1024 total in 1 blocks; 256 free (0 chunks); 768
used
pg_amop_opr_opc_index: 1024 total in 1 blocks; 328 free (0 chunks); 696
used
pg_amop_opc_strat_index: 1024 total in 1 blocks; 256 free (0 chunks); 768
used
pg_aggregate_fnoid_index: 1024 total in 1 blocks; 392 free (0 chunks); 632
used
MdSmgr: 8192 total in 1 blocks; 6136 free (0 chunks); 2056 used
LockTable (locallock hash): 8192 total in 1 blocks; 1856 free (0 chunks);
6336 used
Timezones: 47592 total in 2 blocks; 5968 free (0 chunks); 41624 used
ErrorContext: 8192 total in 1 blocks; 8176 free (0 chunks); 16 used
[1-1] postgres on docs: ERROR:  out of memory
[1-2] postgres on docs: DETAIL:  Failed on request of size 270344621.
[1-3] postgres on docs: STATEMENT:  COPY public.documents (id, project_id,
doc_group_id, is_archive, title, status, version, modified,description,
filedata, filename, filetype, filesize) TO stdout;

But if try
pg_dump --encoding=UTF8 -U postgres docs > docs.sql
ALL OK!

BUT when i try to restore it to database with encoding 'WIN1251' it failed
with the same error.

part of my postgresql.conf:
===========================
shared_buffers = 10000
work_mem = 7158
maintenance_work_mem = 16384

output of "pg_config" command:
==============================
BINDIR = /usr/local/postgres/bin
DOCDIR =
INCLUDEDIR = /usr/local/postgres/include
PKGINCLUDEDIR = /usr/local/postgres/include
INCLUDEDIR-SERVER = /usr/local/postgres/include/server
LIBDIR = /usr/local/postgres/lib
PKGLIBDIR = /usr/local/postgres/lib
LOCALEDIR = /usr/local/postgres/share/locale
MANDIR = /usr/local/postgres/man
SHAREDIR = /usr/local/postgres/share
SYSCONFDIR = /usr/local/postgres/etc
PGXS = /usr/local/postgres/lib/pgxs/src/makefiles/pgxs.mk
CONFIGURE = '--prefix=/usr/local/postgres' '--without-docdir' '--enable-nls'
'--cache-file=/dev/null' 'CPPFLAGS=-I/usr/local/include'
'LDFLAGS=-L/usr/local/lib'
CC = gcc
CPPFLAGS = -I/usr/local/include
CFLAGS = -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline
-Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing
CFLAGS_SL = -fPIC -DPIC
LDFLAGS = -L/usr/local/lib -Wl,-R/usr/local/postgres/lib
LDFLAGS_SL =
LIBS = -lpgport -lintl -lz -lreadline -lcrypt -lcompat -lm -lutil
VERSION = PostgreSQL 8.1.5

Same errors in PostgreSQL 8.2 beta 2 too!

Is this a bug? Please, help! :)
pgsql-bugs by date:
From: "Stephen haberman"
Date: 10 November 2006, 05:13:01
Subject: BUG #2750: information_schema broken with primary and foreign key on the same column
From: ""
Date: 10 November 2006, 05:13:19
Subject: BUG #2745: upgrade.bat refers to msi file name for version 8.1 ms
BUG #2744: pg_dump, ERROR: out of memory, Failed on request of size 270344621 - Mailing list pgsql-bugs

Previous

Next
