Thread: postmaster dies (6.5.3)
Hi, I got seriuos problem this night with postgres which is running as db backend to apache. I have cron job which vacuuming database every hour and it worked for weeks without problem (well, there is problem with concurrent processes under high load, but this night was very quiet) Here is the processes currently seen: 167 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle 168 ? S 0:00 /usr/local/pgsql/bin/postgreslocalhost httpd nature idle 169 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpdnature idle 170 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle 171 ? S 0:00 /usr/local/pgsql/bin/postgreslocalhost httpd nature idle 26578 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle 29372 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd discovery idle from apache's error log: [Wed Dec 15 02:10:08 1999] [error] DBI->connect failed: connectDB() -- couldn'tsend startup packet: errno=32 Broken pipeat /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 [Wed Dec 15 02:10:08 1999] [error] DBI->connect failed: pqReadData() -- backend closed the channel unexpectedly. This probably means the backend terminated abnormally before or while processingthe request.at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 [Wed Dec 15 02:10:43 1999] [error] DBI->connect failed: connectDB() -- connect()failed: No such file or directory Is the postmaster running at 'localhost' and accepting connections on Unix socke t '5432'?at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 fortunately postmaster was started with debug option: StartTransactionCommand query: SET client_encoding = 'KOI8' ProcessUtility: SET client_encoding = 'KOI8' CommitTransactionCommand postmaster: StreamConnection: accept: Invalid argument /usr/local/pgsql/bin/postmaster: ServerLoop: handling reading 6 /usr/local/pgsql/bin/postmaster: ServerLoop: handling reading 6 FATAL 1: ReleaseLruFile: No opened files - no one can be closed proc_exit(0) [#0] shmem_exit(0) [#0] exit(0) Aha, >From system log files I found probable explanation - file table overflow ! Could this be a reason of postmaster dead and how to avoid this ? Dec 15 01:47:27 zeus kernel: Unable to load interpreter Dec 15 02:09:28 zeus xntpd[103]: kernel pll status change 89 Dec 15 02:10:08 zeus squid[133]: file_open: error opening file /d4/squid/cache/0 2/69/00001692: (23) File table overflow Dec 15 02:10:08 zeus squid[133]: storeSwapInStart: Failed for 'http://xyz.tvcom. ru/99/12/100_2.jpg' Dec 15 02:10:12 zeus kernel: Unable to load interpreter Dec 15 03:26:16 zeus xntpd[103]: kernel pll status change 89 Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Thanks for the patch. I think I'm going to upgrade to FreeBSD-3.3 and PG-6.5.3 tonight. Will I still need the patch with 6.5.3? I'm also going to do a connection test on another offline server to see if it is indeed a load problem. I'll post the results if anyone is interested. Thank you for the help, Matthew At 11:15 AM 12/15/99 +0300, Oleg Bartunov wrote: >Hi, > >I got seriuos problem this night with postgres which is running as >db backend to apache. I have cron job which vacuuming database >every hour and it worked for weeks without problem >(well, there is problem with concurrent processes under high load, >but this night was very quiet) > >Here is the processes currently seen: > > 167 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle > 168 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle > 169 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle > 170 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle > 171 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle >26578 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature idle >29372 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd discovery >idle > > >from apache's error log: >[Wed Dec 15 02:10:08 1999] [error] DBI->connect failed: connectDB() -- >couldn't > send startup packet: errno=32 >Broken pipe > at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 > >[Wed Dec 15 02:10:08 1999] [error] DBI->connect failed: pqReadData() -- >backend >closed the channel unexpectedly. > This probably means the backend terminated abnormally > before or while processing the request. > at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 > >[Wed Dec 15 02:10:43 1999] [error] DBI->connect failed: connectDB() -- >connect() > failed: No such file or directory >Is the postmaster running at 'localhost' and accepting connections on Unix >socke >t '5432'? > at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 > > >fortunately postmaster was started with debug option: > >StartTransactionCommand >query: SET client_encoding = 'KOI8' >ProcessUtility: SET client_encoding = 'KOI8' >CommitTransactionCommand >postmaster: StreamConnection: accept: Invalid argument >/usr/local/pgsql/bin/postmaster: ServerLoop: handling reading 6 >/usr/local/pgsql/bin/postmaster: ServerLoop: handling reading 6 >FATAL 1: ReleaseLruFile: No opened files - no one can be closed >proc_exit(0) [#0] >shmem_exit(0) [#0] >exit(0) > >Aha, > >>From system log files I found probable explanation - file table overflow ! >Could this be a reason of postmaster dead and how to avoid this ? > >Dec 15 01:47:27 zeus kernel: Unable to load interpreter >Dec 15 02:09:28 zeus xntpd[103]: kernel pll status change 89 >Dec 15 02:10:08 zeus squid[133]: file_open: error opening file >/d4/squid/cache/0 >2/69/00001692: (23) File table overflow >Dec 15 02:10:08 zeus squid[133]: storeSwapInStart: Failed for >'http://xyz.tvcom. >ru/99/12/100_2.jpg' >Dec 15 02:10:12 zeus kernel: Unable to load interpreter >Dec 15 03:26:16 zeus xntpd[103]: kernel pll status change 89 > > > Regards, > > Oleg >_____________________________________________________________ >Oleg Bartunov, sci.researcher, hostmaster of AstroNet, >Sternberg Astronomical Institute, Moscow University (Russia) >Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ >phone: +007(095)939-16-83, +007(095)939-23-83 > > >************
Umm, sorry, I hit reply to the wrong message... Matthew At 02:40 PM 12/15/99 -0500, Matthew Hagerty wrote: >Thanks for the patch. I think I'm going to upgrade to FreeBSD-3.3 and >PG-6.5.3 tonight. Will I still need the patch with 6.5.3? I'm also going >to do a connection test on another offline server to see if it is indeed a >load problem. I'll post the results if anyone is interested. > >Thank you for the help, >Matthew > >At 11:15 AM 12/15/99 +0300, Oleg Bartunov wrote: >>Hi, >> >>I got seriuos problem this night with postgres which is running as >>db backend to apache. I have cron job which vacuuming database >>every hour and it worked for weeks without problem >>(well, there is problem with concurrent processes under high load, >>but this night was very quiet) >> >>Here is the processes currently seen: >> >> 167 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature >idle >> 168 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature >idle >> 169 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature >idle >> 170 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature >idle >> 171 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature >idle >>26578 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd nature >idle >>29372 ? S 0:00 /usr/local/pgsql/bin/postgres localhost httpd discovery >>idle >> >> >>from apache's error log: >>[Wed Dec 15 02:10:08 1999] [error] DBI->connect failed: connectDB() -- >>couldn't >> send startup packet: errno=32 >>Broken pipe >> at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 >> >>[Wed Dec 15 02:10:08 1999] [error] DBI->connect failed: pqReadData() -- >>backend >>closed the channel unexpectedly. >> This probably means the backend terminated abnormally >> before or while processing the request. >> at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 >> >>[Wed Dec 15 02:10:43 1999] [error] DBI->connect failed: connectDB() -- >>connect() >> failed: No such file or directory >>Is the postmaster running at 'localhost' and accepting connections on Unix >>socke >>t '5432'? >> at /opt/perl5/lib/site_perl/5.005/Apache/DBI.pm line 138 >> >> >>fortunately postmaster was started with debug option: >> >>StartTransactionCommand >>query: SET client_encoding = 'KOI8' >>ProcessUtility: SET client_encoding = 'KOI8' >>CommitTransactionCommand >>postmaster: StreamConnection: accept: Invalid argument >>/usr/local/pgsql/bin/postmaster: ServerLoop: handling reading 6 >>/usr/local/pgsql/bin/postmaster: ServerLoop: handling reading 6 >>FATAL 1: ReleaseLruFile: No opened files - no one can be closed >>proc_exit(0) [#0] >>shmem_exit(0) [#0] >>exit(0) >> >>Aha, >> >>>From system log files I found probable explanation - file table overflow ! >>Could this be a reason of postmaster dead and how to avoid this ? >> >>Dec 15 01:47:27 zeus kernel: Unable to load interpreter >>Dec 15 02:09:28 zeus xntpd[103]: kernel pll status change 89 >>Dec 15 02:10:08 zeus squid[133]: file_open: error opening file >>/d4/squid/cache/0 >>2/69/00001692: (23) File table overflow >>Dec 15 02:10:08 zeus squid[133]: storeSwapInStart: Failed for >>'http://xyz.tvcom. >>ru/99/12/100_2.jpg' >>Dec 15 02:10:12 zeus kernel: Unable to load interpreter >>Dec 15 03:26:16 zeus xntpd[103]: kernel pll status change 89 >> >> >> Regards, >> >> Oleg >>_____________________________________________________________ >>Oleg Bartunov, sci.researcher, hostmaster of AstroNet, >>Sternberg Astronomical Institute, Moscow University (Russia) >>Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ >>phone: +007(095)939-16-83, +007(095)939-23-83 >> >> >>************ > > >************