Thread: Core was generated by `postgres: kazeon KazDB 192.168.50.131(37625) SELECT '.

Hi,

I am using Cent OS 6 () and Postgresql 9.0.4. I was running Enterprise setup
of 4 node cluster. I got following error in postgres

 30619 2012-08-24 15:11:13.430 PDT LOG:  server process (PID 17036) was
terminated by signal 11: Segmentation fault
 30619 2012-08-24 15:11:13.454 PDT LOG:  terminating any other active server
processes
192.168.50.131 17035 2012-08-24 15:11:13.454 PDT WARNING:  terminating
connection because of crash of another server process
192.168.50.131 17035 2012-08-24 15:11:13.454 PDT DETAIL:  The postmaster has
commanded this server process to roll back the current transaction and exit,
because another server process exited abnormally and possibly corrupted
shared memory.
192.168.50.131 17035 2012-08-24 15:11:13.454 PDT HINT:  In a moment you
should be able to reconnect to the database and repeat your command.
 30623 2012-08-24 15:11:13.454 PDT WARNING:  terminating connection because
of crash of another server process
 30623 2012-08-24 15:11:13.454 PDT DETAIL:  The postmaster has commanded
this server process to roll back the current transaction and exit, because
another server process exited abnormally and possibly corrupted shared
memory.
 30623 2012-08-24 15:11:13.454 PDT HINT:  In a moment you should be able to
reconnect to the database and repeat your command.
192.168.50.131 17041 2012-08-24 15:11:13.455 PDT WARNING:  terminating
connection because of crash of another server process
192.168.50.131 17041 2012-08-24 15:11:13.455 PDT DETAIL:  The postmaster has
commanded this server process to roll back the current transaction and exit,
because another server process exited abnormally and possibly corrupted
shared memory.
192.168.50.131 17041 2012-08-24 15:11:13.455 PDT HINT:  In a moment you
should be able to reconnect to the database and repeat your command.
 30619 2012-08-24 15:11:13.570 PDT LOG:  all server processes terminated;
reinitializing
 17205 2012-08-24 15:11:13.688 PDT LOG:  database system was interrupted;
last known up at 2012-08-24 14:50:00 PDT
 17205 2012-08-24 15:11:13.740 PDT LOG:  database system was not properly
shut down; automatic recovery in progress
 17205 2012-08-24 15:11:13.743 PDT LOG:  consistent recovery state reached
at 2/3F2E3788
 17205 2012-08-24 15:11:13.744 PDT LOG:  redo starts at 2/3F2E3788
 17205 2012-08-24 15:11:13.817 PDT LOG:  record with zero length at
2/3F331F60
 17205 2012-08-24 15:11:13.818 PDT LOG:  redo done at 2/3F331F20
 17205 2012-08-24 15:11:13.818 PDT LOG:  last completed transaction was at
log time 2012-08-24 15:08:51.681231-07
 30619 2012-08-24 15:11:13.880 PDT LOG:  database system is ready to accept
connections
 17210 2012-08-24 15:11:13.880 PDT LOG:  autovacuum launcher started


Core details:

Reading symbols from /opt/openkaz/lib/kazsql.so...done.
Loaded symbols for /opt/openkaz/lib/kazsql.so
Core was generated by `postgres: kazeon KazDB 192.168.50.131(37625) SELECT
'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007f63bc79e0b8 in memcpy () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
kazsqldb-1.0.2-9.0.4.x86_64
(gdb) bt
#0 0x00007f63bc79e0b8 in memcpy () from /lib64/libc.so.6
#1 0x000000000044d989 in heap_fill_tuple ()
#2 0x000000000044dd91 in heap_form_minimal_tuple ()
#3 0x00000000006ce356 in copytup_heap ()
#4 0x00000000006ccfd1 in tuplesort_puttupleslot ()
#5 0x000000000055e591 in ExecSort ()
#6 0x00000000005480b8 in ExecProcNode ()
#7 0x0000000000554e62 in ExecAgg ()
#8 0x0000000000548098 in ExecProcNode ()
#9 0x0000000000546f32 in standard_ExecutorRun ()
#10 0x00000000005fc517 in PortalRunSelect ()
#11 0x00000000005fd788 in PortalRun ()
#12 0x00000000005fb223 in PostgresMain ()
#13 0x00000000005c89e9 in ServerLoop ()
#14 0x00000000005cafbc in PostmasterMain ()
#15 0x0000000000572d98 in main ()
(gdb) quit

Is somebody else is getting the same error?
Can you let know the reason for segfault in postmastermain and
suggestion/resolution for the same.

I would appreciate your help.

Thanks,
Anjali



--
View this message in context:
http://postgresql.1045698.n5.nabble.com/Core-was-generated-by-postgres-kazeon-KazDB-192-168-50-131-37625-SELECT-tp5722451.html
Sent from the PostgreSQL - bugs mailing list archive at Nabble.com.

anjali_524 <anjali_524@yahoo.co.in> writes:
> I am using Cent OS 6 () and Postgresql 9.0.4. I was running Enterprise setup
> of 4 node cluster. I got following error in postgres

> Reading symbols from /opt/openkaz/lib/kazsql.so...done.
> Loaded symbols for /opt/openkaz/lib/kazsql.so
> Core was generated by `postgres: kazeon KazDB 192.168.50.131(37625) SELECT
> '.
> Program terminated with signal 11, Segmentation fault.
> #0 0x00007f63bc79e0b8 in memcpy () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install
> kazsqldb-1.0.2-9.0.4.x86_64
> (gdb) bt
> #0 0x00007f63bc79e0b8 in memcpy () from /lib64/libc.so.6
> #1 0x000000000044d989 in heap_fill_tuple ()
> #2 0x000000000044dd91 in heap_form_minimal_tuple ()
> #3 0x00000000006ce356 in copytup_heap ()
> #4 0x00000000006ccfd1 in tuplesort_puttupleslot ()
> #5 0x000000000055e591 in ExecSort ()
> #6 0x00000000005480b8 in ExecProcNode ()
> #7 0x0000000000554e62 in ExecAgg ()
> #8 0x0000000000548098 in ExecProcNode ()
> #9 0x0000000000546f32 in standard_ExecutorRun ()
> #10 0x00000000005fc517 in PortalRunSelect ()
> #11 0x00000000005fd788 in PortalRun ()
> #12 0x00000000005fb223 in PostgresMain ()
> #13 0x00000000005c89e9 in ServerLoop ()
> #14 0x00000000005cafbc in PostmasterMain ()
> #15 0x0000000000572d98 in main ()

Apparently the Sort step is trying to copy an input tuple that is
corrupted somehow.  That could be because of corrupted data on-disk,
or there could be a software bug that mangled the tuple during an
earlier processing step within this query.

Can you pg_dump the table(s) this query was reading without error?
If not, then it's on-disk data corruption.  If so, then it's a
processing bug, but we'd need sufficient information to reproduce
the problem before we can fix it.

BTW, 9.0.4 is over a year old --- you might consider updating to
the current 9.0.x release, which is 9.0.9.  Also, it sounds like
you might be using EDB's commercial version of Postgres, in which
case you really ought to go to them for support.

            regards, tom lane