Thread: postmaster going down own its on

postmaster going down own its on

From
"surabhi.ahuja"
Date:
hi, is it possible for postmaster to go doen on its own?
 
all what the logs say is FATAL: terminating connection dur to administrator's command.
 
thanks,
regards
Surabhi

Re: postmaster going down own its on

From
James Cradock
Date:
It's not normal. 

What's the installation? OS, applications connecting to the server, etc. 

On Apr 7, 2006, at 8:20 AM, surabhi.ahuja wrote:

hi, is it possible for postmaster to go doen on its own?
 
all what the logs say is FATAL: terminating connection dur to administrator's command.
 
thanks,
regards
Surabhi

Re: postmaster going down own its on

From
Richard Huxton
Date:
surabhi.ahuja wrote:
> hi, is it possible for postmaster to go doen on its own?
>
> all what the logs say is FATAL: terminating connection dur to
> administrator's command.

Someone or something is issuing a kill command. It couldn't be the
infamous Linux out-of-memory handler, could it? Check your system logs
for details.

--
   Richard Huxton
   Archonet Ltd

Re: postmaster going down own its on

From
Douglas McNaught
Date:
Richard Huxton <dev@archonet.com> writes:

> surabhi.ahuja wrote:
>> hi, is it possible for postmaster to go doen on its own?
>> all what the logs say is FATAL: terminating connection dur to
>> administrator's command.
>
> Someone or something is issuing a kill command. It couldn't be the
> infamous Linux out-of-memory handler, could it? Check your system logs
> for details.

I'm not sure it's that--the OOM killer uses SIGKILL which would take
down the server before it could write that log entry.

-Doug

Re: postmaster going down own its on

From
Richard Huxton
Date:
Douglas McNaught wrote:
> Richard Huxton <dev@archonet.com> writes:
>
>> surabhi.ahuja wrote:
>>> hi, is it possible for postmaster to go doen on its own?
>>> all what the logs say is FATAL: terminating connection dur to
>>> administrator's command.
>> Someone or something is issuing a kill command. It couldn't be the
>> infamous Linux out-of-memory handler, could it? Check your system logs
>> for details.
>
> I'm not sure it's that--the OOM killer uses SIGKILL which would take
> down the server before it could write that log entry.

Hmm... (tests it) you're right. What would be sending SIGTERM to a backend?

--
   Richard Huxton
   Archonet Ltd

Re: postmaster going down own its on

From
Martijn van Oosterhout
Date:
On Fri, Apr 07, 2006 at 03:03:09PM +0100, Richard Huxton wrote:
> >I'm not sure it's that--the OOM killer uses SIGKILL which would take
> >down the server before it could write that log entry.
>
> Hmm... (tests it) you're right. What would be sending SIGTERM to a backend?

The only other thing I've ever heard of is some systems do a sigterm
when you pass a quota limit?

Not sure though,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

Attachment

Re: postmaster going down own its on

From
Tom Lane
Date:
Martijn van Oosterhout <kleptog@svana.org> writes:
> On Fri, Apr 07, 2006 at 03:03:09PM +0100, Richard Huxton wrote:
>> What would be sending SIGTERM to a backend?

> The only other thing I've ever heard of is some systems do a sigterm
> when you pass a quota limit?

Could be.  The actual standard use of SIGTERM is to kill processes
belonging to your terminal process group when you log out.  So, for
example, manually starting the postmaster from a terminal window and
then closing that window could cause this to happen.

I suppose the OP is bright enough to have noticed the connection between
closing the window and losing the postmaster, so it's probably not quite
that simple, but anyway something like this is what to look for.

            regards, tom lane

Re: postmaster going down own its on

From
Douglas McNaught
Date:
Tom Lane <tgl@sss.pgh.pa.us> writes:

> Martijn van Oosterhout <kleptog@svana.org> writes:
>> On Fri, Apr 07, 2006 at 03:03:09PM +0100, Richard Huxton wrote:
>>> What would be sending SIGTERM to a backend?
>
>> The only other thing I've ever heard of is some systems do a sigterm
>> when you pass a quota limit?
>
> Could be.  The actual standard use of SIGTERM is to kill processes
> belonging to your terminal process group when you log out.  So, for
> example, manually starting the postmaster from a terminal window and
> then closing that window could cause this to happen.

I thought that was SIGHUP?

-Doug

Re: postmaster going down own its on

From
Tom Lane
Date:
Douglas McNaught <doug@mcnaught.org> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> Could be.  The actual standard use of SIGTERM is to kill processes
>> belonging to your terminal process group when you log out.

> I thought that was SIGHUP?

Doh.  Not enough caffeine absorbed yet.

As penance, here's a comment that I think is actually correct: sending
SIGINT to the postmaster will make it turn around and send SIGTERM to
all the backends.  So there are two different explanations for the
backends giving the "administrator command" error: either some outside
force sent them SIGTERM directly, or some outside force sent the
postmaster SIGINT.  The SIGINT-the-postmaster theory is the more likely,
I suspect, and that again could be associated with having carelessly
left the postmaster attached to one's terminal.  In any case, the first
thing to do is look in the postmaster log and see if you see a message
about "received fast shutdown request", which would be proof one way or
the other.

            regards, tom lane

Re: postmaster going down own its on

From
"surabhi.ahuja"
Date:

the scenario in which the above took place was somewhat like this
we have a script to stop some of the processes running in the background.

this script was run, and all the processes got stopped

then another script will start these processes one after the other in background
that script was run, and postmaster also got started.

and then all of a sudden, it went down.

thanks for the suggestions, i ll also look into the log essages and look for
"received fast shutdown" message.

but in case the postmaster received a fast shutdown signal, when the above mentioned script (to stop all processes in background).

but again the script to start the processes including postmaster was run

and at that time, postmaster did start, then how come it suddently went down.

will provide u with more info on the same,

thanks,
regards
surabhi

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Fri 4/7/2006 8:51 PM
To: Douglas McNaught
Cc: Martijn van Oosterhout; Richard Huxton; surabhi.ahuja; pgsql-general@postgresql.org
Subject: Re: [GENERAL] postmaster going down own its on

***********************
Your mail has been scanned by InterScan VirusWall.
***********-***********


Douglas McNaught <doug@mcnaught.org> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> Could be.  The actual standard use of SIGTERM is to kill processes
>> belonging to your terminal process group when you log out.

> I thought that was SIGHUP?

Doh.  Not enough caffeine absorbed yet.

As penance, here's a comment that I think is actually correct: sending
SIGINT to the postmaster will make it turn around and send SIGTERM to
all the backends.  So there are two different explanations for the
backends giving the "administrator command" error: either some outside
force sent them SIGTERM directly, or some outside force sent the
postmaster SIGINT.  The SIGINT-the-postmaster theory is the more likely,
I suspect, and that again could be associated with having carelessly
left the postmaster attached to one's terminal.  In any case, the first
thing to do is look in the postmaster log and see if you see a message
about "received fast shutdown request", which would be proof one way or
the other.

                        regards, tom lane



Re: postmaster going down own its on

From
Martijn van Oosterhout
Date:
On Sat, Apr 08, 2006 at 08:02:04PM +0530, surabhi.ahuja wrote:
>
> the scenario in which the above took place was somewhat like this
> we have a script to stop some of the processes running in the background.
>
> this script was run, and all the processes got stopped
>
> then another script will start these processes one after the other in background
> that script was run, and postmaster also got started.
>
> and then all of a sudden, it went down.

So you stopped it and it stopped. You started it up again, and you say
it stopped again.

I'm sorry, you're going to have to post your logs before we can help
you further. When you stop the postmaster you are sending a shutdown
request, so it's not totally surprising that that message appears.

Have a nice day,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

Attachment

Re: postmaster going down own its on

From
"surabhi.ahuja"
Date:
 
hi,
i noticed the script, and at places it says
 
 
received fast shutdown request
<2006-04-10 10:25:05 IST%>LOG:  aborting any active transactions
<2006-04-10 10:25:05 IST%idle>FATAL:  terminating connection due to administrator command
<2006-04-10 10:25:06 IST%idle>FATAL:  terminating connection due to administrator command
 
when does this happen?
 
onr more thing in have noticed is that if i do a normal shutdown of postmaster by doing
pg_ctl stop -D /export/home1/sdc_image_pool/dbx/ -s -m fast
 
 
nothing gets logged.
why isnt the shutting down of postmaster in the above way getting logged?
 
thanks,
regards
Surabhi

From: pgsql-general-owner@postgresql.org on behalf of surabhi.ahuja
Sent: Sat 4/8/2006 8:02 PM
To: Tom Lane; Douglas McNaught
Cc: Martijn van Oosterhout; Richard Huxton; pgsql-general@postgresql.org
Subject: Re: [GENERAL] postmaster going down own its on

***********************
Your mail has been scanned by InterScan VirusWall.
***********-***********


the scenario in which the above took place was somewhat like this
we have a script to stop some of the processes running in the background.

this script was run, and all the processes got stopped

then another script will start these processes one after the other in background
that script was run, and postmaster also got started.

and then all of a sudden, it went down.

thanks for the suggestions, i ll also look into the log essages and look for
"received fast shutdown" message.

but in case the postmaster received a fast shutdown signal, when the above mentioned script (to stop all processes in background).

but again the script to start the processes including postmaster was run

and at that time, postmaster did start, then how come it suddently went down.

will provide u with more info on the same,

thanks,
regards
surabhi

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Fri 4/7/2006 8:51 PM
To: Douglas McNaught
Cc: Martijn van Oosterhout; Richard Huxton; surabhi.ahuja; pgsql-general@postgresql.org
Subject: Re: [GENERAL] postmaster going down own its on

***********************
Your mail has been scanned by InterScan VirusWall.
***********-***********


Douglas McNaught <doug@mcnaught.org> writes:
> Tom Lane <tgl@sss.pgh.pa.us> writes:
>> Could be.  The actual standard use of SIGTERM is to kill processes
>> belonging to your terminal process group when you log out.

> I thought that was SIGHUP?

Doh.  Not enough caffeine absorbed yet.

As penance, here's a comment that I think is actually correct: sending
SIGINT to the postmaster will make it turn around and send SIGTERM to
all the backends.  So there are two different explanations for the
backends giving the "administrator command" error: either some outside
force sent them SIGTERM directly, or some outside force sent the
postmaster SIGINT.  The SIGINT-the-postmaster theory is the more likely,
I suspect, and that again could be associated with having carelessly
left the postmaster attached to one's terminal.  In any case, the first
thing to do is look in the postmaster log and see if you see a message
about "received fast shutdown request", which would be proof one way or
the other.

                        regards, tom lane