Thread: dropdb/contrib-regression

[original seems lost - sorry if it's a dup]

We still seem to have occasional problems with dropdb while running 
contrib installcheck. The symptoms look like this:

============== dropping database "regression"         ==============
dropdb: database removal failed: ERROR:  database "regression" is being 
accessed by other users
============== creating database "regression"         ==============
createdb: database creation failed: ERROR:  database "regression" 
already exists
pg_regress: createdb failed



Example: (FBSD again) 
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=cockatoo&dt=2004-12-13%2008:10:01 


I guess we could put a small sleep before dropdb in pg_regress.sh to 
make sure a backend from a previous client had enough time to clean up 
after itself, but I am wondering if this is a symptom of a larger problem?

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:
> We still seem to have occasional problems with dropdb while running 
> contrib installcheck. The symptoms look like this:

I see this regularly on some platforms, and seldom/never on others.
I've also seen it happen when scripting a tight loop around the
core regression tests.

I think it must be related to details of the kernel scheduler's
behavior.  Some schedulers will let the exiting backend have cycles
right about then, and some won't.  (IIRC, I've confirmed by strace'ing
that when this happens, the exiting backend has in fact been granted
*no* cycles between when the old psql quits and when the new backend
gets to the point of failing the dropdb.  I suppose there's some sort
of foreground/background priority heuristic involved.)

> I guess we could put a small sleep before dropdb in pg_regress.sh to 

I'd prefer to put it in contrib/Makefile's installcheck rule, so that
we don't pay the price in contexts where it's not needed.

A more radical solution would be to tweak libpq's PQfinish() to be a
synchronous close, ie, wait for the backend to exit before returning.
I think we've speculated about doing that in the past, but never been
fully convinced that it's worth the downside of usually-unnecessary
client delay.
        regards, tom lane

Tom Lane wrote:

>Andrew Dunstan <andrew@dunslane.net> writes:
>
>
>>I guess we could put a small sleep before dropdb in pg_regress.sh to
>>
>>
>
>I'd prefer to put it in contrib/Makefile's installcheck rule, so that
>we don't pay the price in contexts where it's not needed.
>
>

tiny patch attached. For reasons Tom cited related to scheduler it isn't
guaranteed to fix the problem, only to lower the likelihood of occurrence.

>A more radical solution would be to tweak libpq's PQfinish() to be a
>synchronous close, ie, wait for the backend to exit before returning.
>I think we've speculated about doing that in the past, but never been
>fully convinced that it's worth the downside of usually-unnecessary
>client delay.
>
>

Sufficient unto the day .... seems to me making it synchronous could be
a cure as bad as the disease.

cheers

andrew



Index: contrib/Makefile
===================================================================
RCS file: /home/cvsmirror/pgsql/contrib/Makefile,v
retrieving revision 1.52
diff -c -r1.52 Makefile
*** contrib/Makefile    4 Nov 2004 06:09:17 -0000    1.52
--- contrib/Makefile    14 Dec 2004 01:38:00 -0000
***************
*** 60,63 ****
--- 60,64 ----
  all install installdirs uninstall clean distclean maintainer-clean check installcheck:
      @for dir in $(WANTED_DIRS); do \
          $(MAKE) -C $$dir $@ || exit; \
+         sleep 2 ; \
      done

On Mon, 2004-12-13 at 19:29 -0500, Tom Lane wrote:
> A more radical solution would be to tweak libpq's PQfinish() to be a
> synchronous close, ie, wait for the backend to exit before returning.
> I think we've speculated about doing that in the past, but never been
> fully convinced that it's worth the downside of usually-unnecessary
> client delay.

I can see this being useful in some situations, but I think the current
behavior is preferable for most clients. Perhaps we could make this a
per-connection option, or supply a variant of PQfinish() ?

-Neil

Neil Conway <neilc@samurai.com> writes:
> On Mon, 2004-12-13 at 19:29 -0500, Tom Lane wrote:
>> A more radical solution would be to tweak libpq's PQfinish() to be a
>> synchronous close,

> I can see this being useful in some situations, but I think the current
> behavior is preferable for most clients. Perhaps we could make this a
> per-connection option, or supply a variant of PQfinish() ?

If it's optional, then to make it useful for the problem at hand, we'd
have to expose the option clear up to the psql command line, which seems
like a lotta work.

At the moment I think a "sleep" in the contrib makefile is sufficient
(though note I intended to apply it only to installcheck not the other
actions ;-))
        regards, tom lane

Tom Lane said:
.
>
> At the moment I think a "sleep" in the contrib makefile is sufficient
> (though note I intended to apply it only to installcheck not the other
> actions ;-))

If you're going to make a separate rule for installcheck (which I agree is a
good idea), please also consider making it fault tolerant as I suggested
here:http://archives.postgresql.org/pgsql-patches/2004-09/msg00337.php


cheers

andrew

"Andrew Dunstan" <andrew@dunslane.net> writes:
> Tom Lane said:
>> At the moment I think a "sleep" in the contrib makefile is sufficient
>> (though note I intended to apply it only to installcheck not the other
>> actions ;-))

> If you're going to make a separate rule for installcheck (which I agree is a
> good idea), please also consider making it fault tolerant as I suggested
> here:http://archives.postgresql.org/pgsql-patches/2004-09/msg00337.php

Done.
        regards, tom lane