Thread: server closed the connection unexpectedly
Hi all,
Errors running a psql batch that has worked for weeks without a
problem:
$ uname -a
Linux s1 2.6.6-1.435.2.3 #1 Thu Jul 1 08:25:29 EDT 2004 i686 athlon
i386 GNU/Linux
$ rpm -q postgresql
postgresql-7.4.2-1
$ psql KakaoStats
Bem-vindo ao psql 7.4.2, o terminal iterativo do PostgreSQL.
select insert_usuarios_producao();
psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: server closed the
connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: connection to server
was lost
select deleta_antigos()
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
vacuum KakaoStats
psql: FATAL: the database system is starting up
These commands are some of the last of a script which runs normaly
until then.
Regards
Clodoaldo Pinto
_______________________________________________________
Yahoo! Mail agora com 100MB, anti-spam e antivírus grátis!
http://br.info.mail.yahoo.com/
It seems that postgresql crashed recently, tried to restart automatically, and has not succesfully restarted. You will need to stop the postmaster daemon manually, probably delete the postmaster.pid file, and restart the postmaster daemon. Chances are that anyone trying to query the database is not able to.... HTH On Fri, 2004-07-16 at 21:05, Clodoaldo Pinto Neto wrote: > Hi all, > > Errors running a psql batch that has worked for weeks without a > problem: > $ uname -a > Linux s1 2.6.6-1.435.2.3 #1 Thu Jul 1 08:25:29 EDT 2004 i686 athlon > i386 GNU/Linux > $ rpm -q postgresql > postgresql-7.4.2-1 > $ psql KakaoStats > Bem-vindo ao psql 7.4.2, o terminal iterativo do PostgreSQL. > > > select insert_usuarios_producao(); > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: server closed the > connection unexpectedly > This probably means the server terminated abnormally > before or while processing the request. > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: connection to server > was lost > > select deleta_antigos() > server closed the connection unexpectedly > This probably means the server terminated abnormally > before or while processing the request. > > vacuum KakaoStats > psql: FATAL: the database system is starting up > > These commands are some of the last of a script which runs normaly > until then. > > Regards > Clodoaldo Pinto > > > > > > > > _______________________________________________________ > Yahoo! Mail agora com 100MB, anti-spam e antivírus grátis! > http://br.info.mail.yahoo.com/ > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org
There was no crash that I know of. # su postgres $ cd /var/lib/pgsql/data $ cat postmaster.pid 14877 /var/lib/pgsql/data 5432001 393216 $ pg_ctl stop -D ./ waiting for postmaster to shut down......done $ rm postmaster.pid rm: imposível remover `postmaster.pid': Arquivo ou diretório não encontrado #(means file not found) $ postmaster -D /var/lib/pgsql/data > logfile 2>&1 & [1] 19256 Now as user cpn: $ pg_dump -Fc KakaoStats > KakaoStats.dump pg_dump: conector (socket) não está aberto #(means not open) pg_dump: Comando SQL para copiar o conteúdo da tabela "usuarios" falhou: PQendcopy() falhou. #(means failed) pg_dump: Mensagem de erro do servidor: conector (socket) não está aberto #(not open) pg_dump: O comando foi: COPY public.usuarios (usuario, data, n_time, wus, pontos) TO stdout; After reboot this was the log file: (forgot to get it before) # cat logfile LOG: database system was shut down at 2004-07-17 05:41:04 BRT LOG: checkpoint record is at 7/C63CFA44 LOG: redo record is at 7/C63CFA44; undo record is at 0/0; shutdown TRUE LOG: next transaction ID: 7704; next OID: 426301 LOG: database system is ready LOG: server process (PID 19261) was terminated by signal 11 LOG: terminating any other active server processes LOG: all server processes terminated; reinitializing LOG: database system was interrupted at 2004-07-17 05:45:22 BRT LOG: checkpoint record is at 7/C63CFA44 LOG: redo record is at 7/C63CFA44; undo record is at 0/0; shutdown TRUE LOG: next transaction ID: 7704; next OID: 426301 LOG: database system was not properly shut down; automatic recovery in progress LOG: record with zero length at 7/C63CFA84 LOG: redo is not required LOG: database system is ready LOG: received fast shutdown request LOG: shutting down LOG: database system is shut down After reboot it worked OK. CPN --- mike g <mike@thegodshalls.com> escreveu: > It seems that postgresql crashed recently, tried to restart > automatically, and has not succesfully restarted. > > You will need to stop the postmaster daemon manually, probably delete > the postmaster.pid file, and restart the postmaster daemon. > > Chances are that anyone trying to query the database is not able > to.... > > HTH > > > On Fri, 2004-07-16 at 21:05, Clodoaldo Pinto Neto wrote: > > Hi all, > > > > Errors running a psql batch that has worked for weeks without a > > problem: > > $ uname -a > > Linux s1 2.6.6-1.435.2.3 #1 Thu Jul 1 08:25:29 EDT 2004 i686 athlon > > i386 GNU/Linux > > $ rpm -q postgresql > > postgresql-7.4.2-1 > > $ psql KakaoStats > > Bem-vindo ao psql 7.4.2, o terminal iterativo do PostgreSQL. > > > > > > select insert_usuarios_producao(); > > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: server closed the > > connection unexpectedly > > This probably means the server terminated abnormally > > before or while processing the request. > > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: connection to > server > > was lost > > > > select deleta_antigos() > > server closed the connection unexpectedly > > This probably means the server terminated abnormally > > before or while processing the request. > > > > vacuum KakaoStats > > psql: FATAL: the database system is starting up > > > > These commands are some of the last of a script which runs normaly > > until then. > > > > Regards > > Clodoaldo Pinto > > > > > > > > > > > > > > > > _______________________________________________________ > > Yahoo! Mail agora com 100MB, anti-spam e antivÃrus grátis! > > http://br.info.mail.yahoo.com/ > > > > ---------------------------(end of > broadcast)--------------------------- > > TIP 6: Have you searched our list archives? > > > > http://archives.postgresql.org > _______________________________________________________ Yahoo! Mail agora com 100MB, anti-spam e antivírus grátis! http://br.info.mail.yahoo.com/
Glad to hear it worked out. This section indicates it had crashed at some point: LOG: database system was not properly shut down; automatic recovery in > progress > Mike On Sat, 2004-07-17 at 04:30, Clodoaldo Pinto Neto wrote: > There was no crash that I know of. > > # su postgres > $ cd /var/lib/pgsql/data > $ cat postmaster.pid > 14877 > /var/lib/pgsql/data > 5432001 393216 > $ pg_ctl stop -D ./ > waiting for postmaster to shut down......done > $ rm postmaster.pid > rm: imposível remover `postmaster.pid': Arquivo ou diretório não > encontrado #(means file not found) > $ postmaster -D /var/lib/pgsql/data > logfile 2>&1 & > [1] 19256 > > Now as user cpn: > $ pg_dump -Fc KakaoStats > KakaoStats.dump > pg_dump: conector (socket) não está aberto #(means not open) > pg_dump: Comando SQL para copiar o conteúdo da tabela "usuarios" > falhou: PQendcopy() falhou. #(means failed) > pg_dump: Mensagem de erro do servidor: conector (socket) não está > aberto #(not open) > pg_dump: O comando foi: COPY public.usuarios (usuario, data, n_time, > wus, pontos) TO stdout; > > After reboot this was the log file: (forgot to get it before) > > # cat logfile > LOG: database system was shut down at 2004-07-17 05:41:04 BRT > LOG: checkpoint record is at 7/C63CFA44 > LOG: redo record is at 7/C63CFA44; undo record is at 0/0; shutdown > TRUE > LOG: next transaction ID: 7704; next OID: 426301 > LOG: database system is ready > LOG: server process (PID 19261) was terminated by signal 11 > LOG: terminating any other active server processes > LOG: all server processes terminated; reinitializing > LOG: database system was interrupted at 2004-07-17 05:45:22 BRT > LOG: checkpoint record is at 7/C63CFA44 > LOG: redo record is at 7/C63CFA44; undo record is at 0/0; shutdown > TRUE > LOG: next transaction ID: 7704; next OID: 426301 > LOG: database system was not properly shut down; automatic recovery in > progress > LOG: record with zero length at 7/C63CFA84 > LOG: redo is not required > LOG: database system is ready > LOG: received fast shutdown request > LOG: shutting down > LOG: database system is shut down > > After reboot it worked OK. > CPN > > --- mike g <mike@thegodshalls.com> escreveu: > It seems that postgresql > crashed recently, tried to restart > > automatically, and has not succesfully restarted. > > > > You will need to stop the postmaster daemon manually, probably delete > > the postmaster.pid file, and restart the postmaster daemon. > > > > Chances are that anyone trying to query the database is not able > > to.... > > > > HTH > > > > > > On Fri, 2004-07-16 at 21:05, Clodoaldo Pinto Neto wrote: > > > Hi all, > > > > > > Errors running a psql batch that has worked for weeks without a > > > problem: > > > $ uname -a > > > Linux s1 2.6.6-1.435.2.3 #1 Thu Jul 1 08:25:29 EDT 2004 i686 athlon > > > i386 GNU/Linux > > > $ rpm -q postgresql > > > postgresql-7.4.2-1 > > > $ psql KakaoStats > > > Bem-vindo ao psql 7.4.2, o terminal iterativo do PostgreSQL. > > > > > > > > > select insert_usuarios_producao(); > > > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: server closed the > > > connection unexpectedly > > > This probably means the server terminated abnormally > > > before or while processing the request. > > > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: connection to > > server > > > was lost > > > > > > select deleta_antigos() > > > server closed the connection unexpectedly > > > This probably means the server terminated abnormally > > > before or while processing the request. > > > > > > vacuum KakaoStats > > > psql: FATAL: the database system is starting up > > > > > > These commands are some of the last of a script which runs normaly > > > until then. > > > > > > Regards > > > Clodoaldo Pinto > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________________ > > > Yahoo! Mail agora com 100MB, anti-spam e antivÃrus grátis! > > > http://br.info.mail.yahoo.com/ > > > > > > ---------------------------(end of > > broadcast)--------------------------- > > > TIP 6: Have you searched our list archives? > > > > > > http://archives.postgresql.org > > > > > > > > > _______________________________________________________ > Yahoo! Mail agora com 100MB, anti-spam e antivírus grátis! > http://br.info.mail.yahoo.com/ > > ---------------------------(end of broadcast)--------------------------- > TIP 7: don't forget to increase your free space map settings
Mike, Thank you for you reply. There is no sign in /var/log/messages of system crash or reboot when or before the problem happened. Also I always stop postmaster with pg_ctl stop and there was sometime that I last stoped it. In case it is worth something this is the /var/log/messages from when the problem was happening: Jul 17 05:34:07 s1 kernel: <3>Debug: sleeping function called from invalid context at include/linux/rwsem.h:43 Jul 17 05:34:07 s1 kernel: in_atomic():0, irqs_disabled():1 Jul 17 05:34:07 s1 kernel: [<02116834>] __might_sleep+0x80/0x8a Jul 17 05:34:07 s1 kernel: [<0213fce7>] rw_vm+0x9b/0x218 Jul 17 05:34:07 s1 kernel: [<0212bf93>] find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: [<0212bf93>] find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: [<02140096>] get_user_size+0x2e/0x55 Jul 17 05:34:07 s1 kernel: [<0212bf93>] find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: [<0211430e>] __is_prefetch+0x1a7/0x295 Jul 17 05:34:07 s1 kernel: [<021156ed>] recalc_task_prio+0x128/0x133 Jul 17 05:34:07 s1 kernel: [<0212bf93>] find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: [<0212bf93>] find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: [<0211466a>] do_page_fault+0x26e/0x446 Jul 17 05:34:07 s1 kernel: [<02115e8c>] __wake_up_common+0x32/0x54 Jul 17 05:34:07 s1 kernel: [<02115ebf>] __wake_up+0x11/0x1a Jul 17 05:34:07 s1 kernel: [<0213639b>] follow_page_pfn+0xec/0xfd Jul 17 05:34:07 s1 kernel: [<0213fe44>] rw_vm+0x1f8/0x218 Jul 17 05:34:07 s1 kernel: [<021143fc>] do_page_fault+0x0/0x446 Jul 17 05:34:07 s1 kernel: [<0212bf93>] find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: [<0212c24c>] do_generic_mapping_read+0xb0/0x2a9 Jul 17 05:34:07 s1 kernel: [<0212c69d>] __generic_file_aio_read+0x157/0x171 Jul 17 05:34:07 s1 kernel: [<0212c445>] file_read_actor+0x0/0x101 Jul 17 05:34:07 s1 kernel: [<0212c6f7>] generic_file_aio_read+0x40/0x47 Jul 17 05:34:07 s1 kernel: [<02141f2e>] do_sync_read+0x68/0x9d Jul 17 05:34:07 s1 kernel: [<0213639b>] follow_page_pfn+0xec/0xfd Jul 17 05:34:07 s1 kernel: [<021156ed>] recalc_task_prio+0x128/0x133 Jul 17 05:34:07 s1 kernel: [<0214201b>] vfs_read+0xb8/0xe4 Jul 17 05:34:07 s1 kernel: [<021421f4>] sys_read+0x2c/0x42 Jul 17 05:34:07 s1 kernel: Jul 17 05:34:07 s1 kernel: Unable to handle kernel paging request at virtual address 00040000 Jul 17 05:34:07 s1 kernel: printing eip: Jul 17 05:34:07 s1 kernel: 0212bf93 Jul 17 05:34:07 s1 kernel: *pde = 00000000 Jul 17 05:34:07 s1 kernel: Oops: 0000 [#12] Jul 17 05:34:07 s1 kernel: Modules linked in: nfsd exportfs lockd ipv6 parport_pc lp parport autofs4 sunrpc via_rhine mii sg scsi_mod dm_mod uhci_hcd ehci_hcd button battery asus_acpi ac ext3 jbd Jul 17 05:34:07 s1 kernel: CPU: 0 Jul 17 05:34:07 s1 kernel: EIP: 0060:[<0212bf93>] Not tainted Jul 17 05:34:07 s1 kernel: EFLAGS: 00010006 (2.6.6-1.435.2.3) Jul 17 05:34:07 s1 kernel: EIP is at find_get_page+0x11/0x24 Jul 17 05:34:07 s1 kernel: eax: 00040000 ebx: 00000000 ecx: 00040000 edx: 00040000 Jul 17 05:34:07 s1 kernel: esi: 00001000 edi: 00001000 ebp: 00000000 esp: 1c206e18 Jul 17 05:34:07 s1 kernel: ds: 007b es: 007b ss: 0068 Jul 17 05:34:07 s1 kernel: Process postmaster (pid: 19187, threadinfo=1c206000 task=1bcf18b0) Jul 17 05:34:07 s1 kernel: Stack: 0212c24c 00000000 000094af 247d0e8c 14c8e9c0 14c8ea08 247d0f24 000094a8 Jul 17 05:34:07 s1 kernel: 00000020 00000020 000094af 000094c8 00000020 00000041 00000010 00000020 Jul 17 05:34:07 s1 kernel: 00000000 00000000 1c206eb8 1c206ec0 00000000 00000000 00002000 0212c69d Jul 17 05:34:07 s1 kernel: Call Trace: Jul 17 05:34:07 s1 kernel: [<0212c24c>] do_generic_mapping_read+0xb0/0x2a9 Jul 17 05:34:07 s1 kernel: [<0212c69d>] __generic_file_aio_read+0x157/0x171 Jul 17 05:34:07 s1 kernel: [<0212c445>] file_read_actor+0x0/0x101 Jul 17 05:34:07 s1 kernel: [<0212c6f7>] generic_file_aio_read+0x40/0x47 Jul 17 05:34:07 s1 kernel: [<02141f2e>] do_sync_read+0x68/0x9d Jul 17 05:34:07 s1 kernel: [<0213639b>] follow_page_pfn+0xec/0xfd Jul 17 05:34:07 s1 kernel: [<021156ed>] recalc_task_prio+0x128/0x133 Jul 17 05:34:07 s1 kernel: [<0214201b>] vfs_read+0xb8/0xe4 Jul 17 05:34:07 s1 kernel: [<021421f4>] sys_read+0x2c/0x42 Jul 17 05:34:07 s1 kernel: Jul 17 05:34:07 s1 kernel: Code: 8b 00 a9 00 00 08 00 74 03 8b 51 0c ff 42 04 fb 89 c8 c3 fa Regards, CPN --- mike g <mike@thegodshalls.com> escreveu: > Glad to hear it worked out. > > This section indicates it had crashed at some point: > LOG: database system was not properly shut down; automatic recovery > in > > progress > > > > Mike > > On Sat, 2004-07-17 at 04:30, Clodoaldo Pinto Neto wrote: > > There was no crash that I know of. > > > > # su postgres > > $ cd /var/lib/pgsql/data > > $ cat postmaster.pid > > 14877 > > /var/lib/pgsql/data > > 5432001 393216 > > $ pg_ctl stop -D ./ > > waiting for postmaster to shut down......done > > $ rm postmaster.pid > > rm: imposÃvel remover `postmaster.pid': Arquivo ou diretório não > > encontrado #(means file not found) > > $ postmaster -D /var/lib/pgsql/data > logfile 2>&1 & > > [1] 19256 > > > > Now as user cpn: > > $ pg_dump -Fc KakaoStats > KakaoStats.dump > > pg_dump: conector (socket) não está aberto #(means not open) > > pg_dump: Comando SQL para copiar o conteúdo da tabela "usuarios" > > falhou: PQendcopy() falhou. #(means failed) > > pg_dump: Mensagem de erro do servidor: conector (socket) não está > > aberto #(not open) > > pg_dump: O comando foi: COPY public.usuarios (usuario, data, > n_time, > > wus, pontos) TO stdout; > > > > After reboot this was the log file: (forgot to get it before) > > > > # cat logfile > > LOG: database system was shut down at 2004-07-17 05:41:04 BRT > > LOG: checkpoint record is at 7/C63CFA44 > > LOG: redo record is at 7/C63CFA44; undo record is at 0/0; shutdown > > TRUE > > LOG: next transaction ID: 7704; next OID: 426301 > > LOG: database system is ready > > LOG: server process (PID 19261) was terminated by signal 11 > > LOG: terminating any other active server processes > > LOG: all server processes terminated; reinitializing > > LOG: database system was interrupted at 2004-07-17 05:45:22 BRT > > LOG: checkpoint record is at 7/C63CFA44 > > LOG: redo record is at 7/C63CFA44; undo record is at 0/0; shutdown > > TRUE > > LOG: next transaction ID: 7704; next OID: 426301 > > LOG: database system was not properly shut down; automatic > recovery in > > progress > > LOG: record with zero length at 7/C63CFA84 > > LOG: redo is not required > > LOG: database system is ready > > LOG: received fast shutdown request > > LOG: shutting down > > LOG: database system is shut down > > > > After reboot it worked OK. > > CPN > > > > --- mike g <mike@thegodshalls.com> escreveu: > It seems that > postgresql > > crashed recently, tried to restart > > > automatically, and has not succesfully restarted. > > > > > > You will need to stop the postmaster daemon manually, probably > delete > > > the postmaster.pid file, and restart the postmaster daemon. > > > > > > Chances are that anyone trying to query the database is not able > > > to.... > > > > > > HTH > > > > > > > > > On Fri, 2004-07-16 at 21:05, Clodoaldo Pinto Neto wrote: > > > > Hi all, > > > > > > > > Errors running a psql batch that has worked for weeks without a > > > > problem: > > > > $ uname -a > > > > Linux s1 2.6.6-1.435.2.3 #1 Thu Jul 1 08:25:29 EDT 2004 i686 > athlon > > > > i386 GNU/Linux > > > > $ rpm -q postgresql > > > > postgresql-7.4.2-1 > > > > $ psql KakaoStats > > > > Bem-vindo ao psql 7.4.2, o terminal iterativo do PostgreSQL. > > > > > > > > > > > > select insert_usuarios_producao(); > > > > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: server closed > the > > > > connection unexpectedly > > > > This probably means the server terminated abnormally > > > > before or while processing the request. > > > > psql:/KakaoStats/scripts/sql/kakaoStats.sql:92: connection to > > > server > > > > was lost > > > > > > > > select deleta_antigos() > > > > server closed the connection unexpectedly > > > > This probably means the server terminated abnormally > > > > before or while processing the request. > > > > > > > > vacuum KakaoStats > > > > psql: FATAL: the database system is starting up > > > > > > > > These commands are some of the last of a script which runs > normaly > > > > until then. > > > > > > > > Regards > > > > Clodoaldo Pinto _______________________________________________________ Yahoo! Mail agora com 100MB, anti-spam e antivírus grátis! http://br.info.mail.yahoo.com/
On Sun, Jul 18, 2004 at 08:18:07AM -0300, Clodoaldo Pinto Neto wrote: > Thank you for you reply. There is no sign in /var/log/messages of > system crash or reboot when or before the problem happened. Also I > always stop postmaster with pg_ctl stop and there was sometime that I > last stoped it. This is clearly a kernel problem. Postgres does not have enough privileges to trigger this kind of mess. Or maybe you have bad hardware; RAM, most likely. Have you run hardware testing tools, like Memtest86+ ? > Jul 17 05:34:07 s1 kernel: <3>Debug: sleeping function called from > invalid context at include/linux/rwsem.h:43 > Jul 17 05:34:07 s1 kernel: > Jul 17 05:34:07 s1 kernel: Unable to handle kernel paging request at > virtual address 00040000 -- Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) "I dream about dreams about dreams", sang the nightingale under the pale moon (Sandman)
Thank you Alvaro, Just filled a RMA for the memory that is 20 days old. It only works with the safest memory settings. Should have done it before. Have never seem a (very slow) memory test catch bad memory, so won't waste my time. Regards, Clodoaldo Pinto --- Alvaro Herrera <alvherre@dcc.uchile.cl> escreveu: > On Sun, Jul 18, 2004 at 08:18:07AM -0300, Clodoaldo Pinto Neto wrote: > > > Thank you for you reply. There is no sign in /var/log/messages of > > system crash or reboot when or before the problem happened. Also I > > always stop postmaster with pg_ctl stop and there was sometime that > I > > last stoped it. > > This is clearly a kernel problem. Postgres does not have enough > privileges to trigger this kind of mess. > > Or maybe you have bad hardware; RAM, most likely. Have you run > hardware > testing tools, like Memtest86+ ? > > > Jul 17 05:34:07 s1 kernel: <3>Debug: sleeping function called from > > invalid context at include/linux/rwsem.h:43 > > > Jul 17 05:34:07 s1 kernel: > > Jul 17 05:34:07 s1 kernel: Unable to handle kernel paging request > at > > virtual address 00040000 > > -- > Alvaro Herrera (<alvherre[a]dcc.uchile.cl>) > "I dream about dreams about dreams", sang the nightingale > under the pale moon (Sandman) > > > ---------------------------(end of > broadcast)--------------------------- > TIP 7: don't forget to increase your free space map settings > > _______________________________________________________ Yahoo! Mail agora com 100MB, anti-spam e antivírus grátis! http://br.info.mail.yahoo.com/