Thread: Re: Getting FATAL: terminating connection due to administrator command
--------Tom Lane <tgl@sss.pgh.pa.us> wrote-------- Subject: Re: [GENERAL] Getting FATAL: terminating connection due to administrator command Date: 15.09.2010 16:07 >Peter Hopfgartner <peter.hopfgartner@r3-gis.com> writes: >> Since some days we are getting the above message. >> Also in the PostgreSQL logs we get: >> FATAL: terminating connection due to administrator command > >This is a result of something sending SIGTERM to the backend process. > >I have heard reports of "load management" software that SIGTERM's >processes more or less at random whenever it decides the system is >overloaded. If you have any such junkware installed on your server, >try disabling it. The server is a rather bare bone server for web mapping, so basically PostgreSQL/PostGIS, Apache, PHP, Tomcat and littleother stuff. The Dell software was the only which did not come from CentOS/EPEL/argeo/in-house RPM packages. I've removedthe Dell stuff completely, but the problem is still there. > >> The server is from Dell, Dell's hardware monitoring, OpenManage, says >that the hardware, in particular memory and disk, are ok. > >Never dealt with OpenManage before, but you should cast a wary eye >upon any Dell-specific software on the machine. This behavior is >definitely not normal for Unix systems, so you need to look for >nonstandard software (and what's more, nonstandard software running with >root privileges, else it couldn't SIGTERM postgres processes). > Other informations: disks are costly SAS drives in a RAID 1 array, memory is with ECC. Security level is disabled SELinux is Permissive. The server acts as a XEN host Is it reasonable to restrict the problem to kernel/hardware and/or PostgreSQL/PostGIS itself? Can I trace where the SIGTERM comes from? > regards, tom lane > Regards, Peter
Peter Hopfgartner <peter.hopfgartner@r3-gis.com> writes: > --------Tom Lane <tgl@sss.pgh.pa.us> wrote-------- >> This is a result of something sending SIGTERM to the backend process. > Can I trace where the SIGTERM comes from? If this is a recent Red-Hat-based release, I think that systemtap could probably be used to determine that. There's a script here that solves a related problem: http://sourceware.org/systemtap/examples/process/sigmon.stp regards, tom lane
Re: Getting FATAL: terminating connection due to administrator command
From
"Joshua J. Kugler"
Date:
On Wednesday 15 September 2010, Peter Hopfgartner elucidated thus: > The server is a rather bare bone server for web mapping, so basically > PostgreSQL/PostGIS, Apache, PHP, Tomcat and little other stuff. The > Dell software was the only which did not come from > CentOS/EPEL/argeo/in-house RPM packages. I've removed the Dell stuff > completely, but the problem is still there. Are you running out of memory and getting killed by the OOM killer? j -- Joshua Kugler Part-Time System Admin/Programmer http://www.eeinternet.com - Fairbanks, AK PGP Key: http://pgp.mit.edu/ ID 0x73B13B6A
"Joshua J. Kugler" <joshua@eeinternet.com> writes:
> On Wednesday 15 September 2010, Peter Hopfgartner elucidated thus:
>> The server is a rather bare bone server for web mapping, so basically
>> PostgreSQL/PostGIS, Apache, PHP, Tomcat and little other stuff. The
>> Dell software was the only which did not come from
>> CentOS/EPEL/argeo/in-house RPM packages. I've removed the Dell stuff
>> completely, but the problem is still there.
> Are you running out of memory and getting killed by the OOM killer?
The OOM killer hits its victims with SIGKILL (kill -9), so we can rule
that out as not matching Peter's symptoms. This is definitely an
unwanted SIGTERM not SIGKILL.
regards, tom lane