Re: [Pgbuildfarm-members] Version 4.10 of buildfarm client released. - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: [Pgbuildfarm-members] Version 4.10 of buildfarm client released.
Date
Msg-id 50F0707C.2030701@dunslane.net
Whole thread Raw
In response to Re: [Pgbuildfarm-members] Version 4.10 of buildfarm client released.  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On 01/11/2013 01:39 PM, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> There was a stray postgres instance running on the box, which I killed:
> FWIW, we've seen an awful lot of persistent buildfarm failures that
> seemed to be due to port conflicts with leftover postmasters.  I think
> the buildfarm script needs to try harder to ensure that it's killed
> everything after a run.  No good ideas how to go about that exactly.
> You could look through "ps" output for postmasters, but what if there's
> a regular Postgres installation on the same box?  Can we just document
> that the buildfarm had better not be run as "postgres"?  (If so, its
> attempt to kill an unowned postmaster would fail anyway; else we need
> a reliable way to tell which ones to kill.)
>
>             


The buildfarm never builds with the standard port unless someone is 
quite perverse indeed. The logic that governs it is:
   $buildport = $PGBuild::conf{base_port};   if ($branch =~ /REL(\d+)_(\d+)/)   {        $buildport += (10 * ($1 - 7))
+$2;   }
 

Certainly the script should not be run as the standard postgres user.

Part of the trouble with detecting rogue postmasters it might have left 
lying around is that various things like to decide what port to run on, 
so it's not always easy for the buildfarm to know what it should be 
looking for.

For branches >= 9.2 this is somewhat ameliorated by the existence of 
EXTRA_REGRESS_OPTS, although we might need a slight adjustment to 
pg_upgrade's test.sh to stop it from trampling on that willy-nilly.

I'm certainly reluctant to be trying to kill anything we aren't dead 
certain is ours. We could possibly detect very early that there is a 
suspected rogue postmaster.

One major source of these rogue processes has almost certainly been this 
piece of logic in pg_ctl:
   * The postmaster should create postmaster.pid very soon after being   * started.  If it's not there after we've
waited5 or more seconds,   * assume startup failed and give up waiting.
 

WHen that happens, pg_ctl fails, and thus so does the buildfarmj client, 
but if it has in fact started a postmaster that was just very slow in 
writing its pid file it has left a postmastr lying around.

ISTR we discussed this phenomenon relatively recently, but I can't find 
a reference to it readily. In any case, nothing has changed on that front.

cheers

andrew





pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)
Next
From: Tom Lane
Date:
Subject: Re: Re: [PATCH] unified frontend support for pg_malloc et al and palloc/pfree mulation (was xlogreader-v4)