Thread: Going for "all green" buildfarm results
I've been making another pass over getting rid of buildfarm failures.
The remaining ones I see at the moment are:
firefly HEAD: intermittent failures in the stats test. We seem to have
fixed every other platform back in January, but not this one.
kudu HEAD: one-time failure 6/1/06 in statement_timeout test, never seen
before. Is it possible system was under enough load that the 1-second
timeout fired before control reached the exception block?
tapir HEAD: pilot error, insufficient SysV shmem settings
carp various: carp seems to have *serious* hardware problems, as it
has been failing randomly in all branches for a long time. I suggest
putting that poor machine out to pasture.
penguin 8.0: fails in tsearch2. Previous investigation says that the
failure is unfixable without initdb, which we are not going to force
for 8.0 branch. I suggest retiring penguin from checking 8.0, as
there's not much point in continuing to see a failure there. Or is
it worth improving buildfarm to be able to skip specific tests?
penguin 7.4: fails in initdb, with what seems to be a variant of the
alignment issue that kills tsearch2 in 8.0. We won't fix this either,
so again might as well stop tracking this branch on this machine.
cobra, stoat, sponge 7.4: pilot error. Either install Tk or configure
--without-tk.
firefly 7.4: dblink test fails, with what looks like an rpath problem.
Another one that we fixed awhile ago, and the fix worked on every
platform but this one.
firefly 7.3: trivial regression diffs; we could install variant
comparison files if anyone cared.
cobra, stoat, caribou 7.3: same Tk configuration error as in 7.4 branch
Firefly is obviously the outlier here. I dunno if anyone cares enough
about SCO to spend time investigating it (I don't). Most of the others
just need a little bit of attention from the machine owner.
regards, tom lane
Tom Lane wrote: > I've been making another pass over getting rid of buildfarm failures. > The remaining ones I see at the moment are: > > firefly HEAD: intermittent failures in the stats test. We seem to have > fixed every other platform back in January, but not this one. > > kudu HEAD: one-time failure 6/1/06 in statement_timeout test, never seen > before. Is it possible system was under enough load that the 1-second > timeout fired before control reached the exception block? [...] FWIW: lionfish had a weird make check error 3 weeks ago which I (unsuccessfully) tried to reproduce multiple times after that: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-05-12%2005:30:14 [...] > cobra, stoat, sponge 7.4: pilot error. Either install Tk or configure > --without-tk. sorry for that but the issue with sponge on 7.4 was fixed nearly a week ago though there have been no changes until today to trigger a new build ;-) Stefan
Tom Lane wrote: > I've been making another pass over getting rid of buildfarm failures. > The remaining ones I see at the moment are: > > firefly HEAD: intermittent failures in the stats test. We seem to > have fixed every other platform back in January, but not this one. > > > firefly 7.4: dblink test fails, with what looks like an rpath problem. > Another one that we fixed awhile ago, and the fix worked on every > platform but this one. > > firefly 7.3: trivial regression diffs; we could install variant > comparison files if anyone cared. > > > Firefly is obviously the outlier here. I dunno if anyone cares > enough about SCO to spend time investigating it (I don't). Most of > the others just need a little bit of attention from the machine > owner. If I generate fixes for firefly (I'm the owner), would they have a prayer Of being applied? LER > > regards, tom lane > > ---------------------------(end of > broadcast)--------------------------- > TIP 2: Don't 'kill -9' the postmaster -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3683 US
Larry Rosenman said: > Tom Lane wrote: >> I've been making another pass over getting rid of buildfarm failures. >> The remaining ones I see at the moment are: >> >> firefly HEAD: intermittent failures in the stats test. We seem to >> have fixed every other platform back in January, but not this one. >> >> >> firefly 7.4: dblink test fails, with what looks like an rpath problem. >> Another one that we fixed awhile ago, and the fix worked on every >> platform but this one. >> >> firefly 7.3: trivial regression diffs; we could install variant >> comparison files if anyone cared. >> >> >> Firefly is obviously the outlier here. I dunno if anyone cares >> enough about SCO to spend time investigating it (I don't). Most of >> the others just need a little bit of attention from the machine >> owner. > > If I generate fixes for firefly (I'm the owner), would they have a > prayer Of being applied? > Sure, although I wouldn't bother with 7.3 - just take 7.3 out of firefly's build schedule. That's not carte blanche on fixes, of course - we'd have to see them. cheers andrew
Tom Lane wrote: > Or is > it worth improving buildfarm to be able to skip specific tests? > > > There is a session on buildfarm improvements scheduled for the Toronto conference. This is certainly one possibility. cheers andrew
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: > FWIW: lionfish had a weird make check error 3 weeks ago which I > (unsuccessfully) tried to reproduce multiple times after that: > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-05-12%2005:30:14 Weird. SELECT ''::text AS eleven, unique1, unique2, stringu1 FROM onek WHERE unique1 < 50 ORDER BYunique1 DESC LIMIT 20 OFFSET 39; ! ERROR: could not open relation with OID 27035 AFAICS, the only way to get that error in HEAD is if ScanPgRelation can't find a pg_class row with the mentioned OID. Presumably 27035 belongs to "onek" or one of its indexes. The very next command also refers to "onek", and doesn't fail, so what we seem to have here is a transient lookup failure. We've found a btree bug like that once before ... wonder if there's still one left? regards, tom lane
"Andrew Dunstan" <andrew@dunslane.net> writes:
> Larry Rosenman said:
>> If I generate fixes for firefly (I'm the owner), would they have a
>> prayer Of being applied?
> Sure, although I wouldn't bother with 7.3 - just take 7.3 out of firefly's
> build schedule. That's not carte blanche on fixes, of course - we'd have to
> see them.
What he said ... it'd depend entirely on how ugly the fixes are ;-)
regards, tom lane
Tom Lane wrote: > "Andrew Dunstan" <andrew@dunslane.net> writes: >> Larry Rosenman said: >>> If I generate fixes for firefly (I'm the owner), would they have a >>> prayer Of being applied? > >> Sure, although I wouldn't bother with 7.3 - just take 7.3 out of >> firefly's build schedule. That's not carte blanche on fixes, of >> course - we'd have to see them. > > What he said ... it'd depend entirely on how ugly the fixes are ;-) > Ok, 7.3 is out of firefly's crontab. I'll look into 7.4. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
Tom Lane wrote: > Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: > >>FWIW: lionfish had a weird make check error 3 weeks ago which I >>(unsuccessfully) tried to reproduce multiple times after that: > > >>http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-05-12%2005:30:14 > > > Weird. > > SELECT ''::text AS eleven, unique1, unique2, stringu1 > FROM onek WHERE unique1 < 50 > ORDER BY unique1 DESC LIMIT 20 OFFSET 39; > ! ERROR: could not open relation with OID 27035 > > AFAICS, the only way to get that error in HEAD is if ScanPgRelation > can't find a pg_class row with the mentioned OID. Presumably 27035 > belongs to "onek" or one of its indexes. The very next command also > refers to "onek", and doesn't fail, so what we seem to have here is > a transient lookup failure. We've found a btree bug like that once > before ... wonder if there's still one left? If there is still one left it must be quite hard to trigger (using the regression tests). Like i said before - I tried quite hard to reproduce the issue back then - without any success. Stefan
Larry Rosenman wrote: > Tom Lane wrote: >> "Andrew Dunstan" <andrew@dunslane.net> writes: >>> Larry Rosenman said: >>>> If I generate fixes for firefly (I'm the owner), would they have a >>>> prayer Of being applied? >> >>> Sure, although I wouldn't bother with 7.3 - just take 7.3 out of >>> firefly's build schedule. That's not carte blanche on fixes, of >>> course - we'd have to see them. >> >> What he said ... it'd depend entirely on how ugly the fixes are ;-) >> > Ok, 7.3 is out of firefly's crontab. > > I'll look into 7.4. > > LER I've taken the cheaters way out for 7.4, and turned off the perl stuff for now. as to HEAD, I've played with the system send/recv space parms, and let's see if that helps the stats stuff. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
Larry Rosenman wrote: > Larry Rosenman wrote: >> Tom Lane wrote: >>> "Andrew Dunstan" <andrew@dunslane.net> writes: >>>> Larry Rosenman said: >>>>> If I generate fixes for firefly (I'm the owner), would they have a >>>>> prayer Of being applied? >>> >>>> Sure, although I wouldn't bother with 7.3 - just take 7.3 out of >>>> firefly's build schedule. That's not carte blanche on fixes, of >>>> course - we'd have to see them. >>> >>> What he said ... it'd depend entirely on how ugly the fixes are ;-) >>> >> Ok, 7.3 is out of firefly's crontab. >> >> I'll look into 7.4. >> >> LER > > I've taken the cheaters way out for 7.4, and turned off the perl > stuff for now. > > as to HEAD, I've played with the system send/recv space parms, and > let's see if > that helps the stats stuff. > > LER well, the changes didn't help. I've pulled ALL the cronjobs from firefly. consider it dead. Since it is an outlier, it's not useful. LER -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 512-248-2683 E-Mail: ler@lerctr.org US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
> -------- Original Message -------- > From: Tom Lane <tgl@sss.pgh.pa.us> > > kudu HEAD: one-time failure 6/1/06 in statement_timeout test, never seen > before. Is it possible system was under enough load that the 1-second > timeout fired before control reached the exception block? > The load here was no different than any other day. As to whether it's a real issue or not I have no idea. It is a virtual machine that is subject to the load on other VMs, but none of them were scheduled to do anything at the time. Kris Jurka
Larry Rosenman wrote: > well, the changes didn't help. > > I've pulled ALL the cronjobs from firefly. > > consider it dead. > > Since it is an outlier, it's not useful. > > > OK, I am marking firefly as retired. That means we have no coverage for Unixware. cheers andrew
I can take other if that helps. Larry, could you help me in the setup? Regards, On Thu, 8 Jun 2006, Andrew Dunstan wrote: > Date: Thu, 08 Jun 2006 10:54:09 -0400 > From: Andrew Dunstan <andrew@dunslane.net> > Newsgroups: pgsql.hackers > Subject: Re: Going for 'all green' buildfarm results > > Larry Rosenman wrote: > > well, the changes didn't help. > > > > I've pulled ALL the cronjobs from firefly. > > > > consider it dead. > > > > Since it is an outlier, it's not useful. > > > > > > > > > > OK, I am marking firefly as retired. That means we have no coverage for > Unixware. > > cheers > > andrew > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > > -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 15, Chemin des Monges +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
On Fri, 9 Jun 2006 ohp@pyrenet.fr wrote: > Date: Fri, 9 Jun 2006 11:12:07 +0200 > From: ohp@pyrenet.fr > To: Andrew Dunstan <andrew@dunslane.net>, Larry Rosenman <ler@lerctr.org> > Newsgroups: pgsql.hackers > Subject: Re: Going for 'all green' buildfarm results > > I can take other if that helps. Ooops... takeover :) > > Larry, could you help me in the setup? > > Regards, > On Thu, 8 Jun 2006, Andrew Dunstan wrote: > > > Date: Thu, 08 Jun 2006 10:54:09 -0400 > > From: Andrew Dunstan <andrew@dunslane.net> > > Newsgroups: pgsql.hackers > > Subject: Re: Going for 'all green' buildfarm results > > > > Larry Rosenman wrote: > > > well, the changes didn't help. > > > > > > I've pulled ALL the cronjobs from firefly. > > > > > > consider it dead. > > > > > > Since it is an outlier, it's not useful. > > > > > > > > > > > > > > > > > OK, I am marking firefly as retired. That means we have no coverage for > > Unixware. > > > > cheers > > > > andrew > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: if posting/reading through Usenet, please send an appropriate > > subscribe-nomail command to majordomo@postgresql.org so that your > > message can get through to the mailing list cleanly > > > > > > -- Olivier PRENANT Tel: +33-5-61-50-97-00 (Work) 15, Chemin des Monges +33-5-61-50-97-01 (Fax) 31190 AUTERIVE +33-6-07-63-80-64 (GSM) FRANCE Email: ohp@pyrenet.fr ------------------------------------------------------------------------------ Make your life a dream, make your dream a reality. (St Exupery)
Mark Wong wrote: >> >> Now why are we failing on 7.3? What version of flex do you have? If >> it's too modern we'll just need to take 7.3 out of the cobra and >> stoat rotations - we'd really only make supercritical fixes on that >> branch these days. > > > Flex is 2.5.33 on both systems. I'm assuming that's too modern so > I'll go ahead and stop building 7.3 for those systems. You could be lucky the others build. I believe our supported version is still 2.5.4, which is what all my linux systems have. cheers andrew
On Thu, 2006-06-22 at 12:52 -0400, Andrew Dunstan wrote: > I believe our supported version is still 2.5.4, which is > what all my linux systems have. Its not clear to me why some people have such antipathy toward recent flex releases, but if our only supported flex version is 2.5.4, I think this should be documented: the current installation instructions only say that "Flex 2.5.4 or later" should be used for CVS builds. -Neil
Neil Conway <neilc@samurai.com> writes:
> On Thu, 2006-06-22 at 12:52 -0400, Andrew Dunstan wrote:
>> I believe our supported version is still 2.5.4, which is
>> what all my linux systems have.
> Its not clear to me why some people have such antipathy toward recent
> flex releases, but if our only supported flex version is 2.5.4, I think
> this should be documented: the current installation instructions only
> say that "Flex 2.5.4 or later" should be used for CVS builds.
Some of them appear to be actively broken :-(. If we knew exactly which
ones they were, we'd document that, but I don't think we have a handle
on why some 2.5.x flexes barf and others don't.
regards, tom lane
Andrew Dunstan <andrew@dunslane.net> writes:
> Mark Wong wrote:
>> Flex is 2.5.33 on both systems. I'm assuming that's too modern so
>> I'll go ahead and stop building 7.3 for those systems.
> You could be lucky the others build. I believe our supported version is
> still 2.5.4, which is what all my linux systems have.
I checked into this and it seems that what we'd need to do is backport
some subset of these 7.4 changes:
2003-09-13 22:18 tgl
* contrib/seg/: Makefile, README.seg, seg.c, seg.sql.in,segparse.y, segscan.l, expected/seg.out: Make contrib/seg work
withflex2.5.31. Fix it up to have a real btree operator class, too,since in PG 7.4 you can't GROUP without one.
2003-09-13 21:52 tgl
* contrib/cube/: Makefile, README.cube, cube.c, cube.sql.in,cubeparse.y, cubescan.l, expected/cube.out: Make
contrib/cubeworkwith flex 2.5.31. Fix it up to have a real btree operator class,too, since in PG 7.4 you can't GROUP
withoutone.
2003-09-14 14:44 tgl
* contrib/: tsearch/parser.l, tsearch2/wordparser/parser.l:Persuade tsearch/tsearch2 to work (or at least pass
theirregressiontests) when using flex 2.5.31. The fix is to *not* tryto use palloc and pfree for allocations within
thelexer; when youdo that, the yy_buffer_stack gets freed at inopportune times. Thecode is already set up to do manual
deallocation,so I see noparticular advantage to using palloc anyway.
This is probably not worth doing. We're really only maintaining 7.3 for
legacy platforms (the only one I care about is RHEL3 ;-)) and a legacy
platform is likely to have an old flex. It's a tad annoying to lose
buildfarm coverage on it though ...
regards, tom lane
Tom Lane wrote: > Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: >> FWIW: lionfish had a weird make check error 3 weeks ago which I >> (unsuccessfully) tried to reproduce multiple times after that: > >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-05-12%2005:30:14 > > Weird. > > SELECT ''::text AS eleven, unique1, unique2, stringu1 > FROM onek WHERE unique1 < 50 > ORDER BY unique1 DESC LIMIT 20 OFFSET 39; > ! ERROR: could not open relation with OID 27035 > > AFAICS, the only way to get that error in HEAD is if ScanPgRelation > can't find a pg_class row with the mentioned OID. Presumably 27035 > belongs to "onek" or one of its indexes. The very next command also > refers to "onek", and doesn't fail, so what we seem to have here is > a transient lookup failure. We've found a btree bug like that once > before ... wonder if there's still one left? FYI: lionfish just managed to hit that problem again: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-07-29%2023:30:06 Stefan
Stefan Kaltenbrunner wrote: > Tom Lane wrote: > > Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: > >> FWIW: lionfish had a weird make check error 3 weeks ago which I > >> (unsuccessfully) tried to reproduce multiple times after that: > > > >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-05-12%2005:30:14 > > > > Weird. > > > > SELECT ''::text AS eleven, unique1, unique2, stringu1 > > FROM onek WHERE unique1 < 50 > > ORDER BY unique1 DESC LIMIT 20 OFFSET 39; > > ! ERROR: could not open relation with OID 27035 > > > > AFAICS, the only way to get that error in HEAD is if ScanPgRelation > > can't find a pg_class row with the mentioned OID. Presumably 27035 > > belongs to "onek" or one of its indexes. The very next command also > > refers to "onek", and doesn't fail, so what we seem to have here is > > a transient lookup failure. We've found a btree bug like that once > > before ... wonder if there's still one left? > > FYI: lionfish just managed to hit that problem again: > > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-07-29%2023:30:06 The error message this time is ! ERROR: could not open relation with OID 27006 It's worth mentioning that the portals_p2 test, which happens in the parallel group previous to where this test is run, also accesses the onek table successfully. It may be interesting to see exactly what relation is 27006. The test alter_table, which is on the same parallel group as limit (the failing test), contains these lines: ALTER INDEX onek_unique1 RENAME TO tmp_onek_unique1; ALTER INDEX tmp_onek_unique1 RENAME TO onek_unique1; Maybe this is related. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera wrote: > Stefan Kaltenbrunner wrote: >> Tom Lane wrote: >>> Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: >>>> FWIW: lionfish had a weird make check error 3 weeks ago which I >>>> (unsuccessfully) tried to reproduce multiple times after that: >>>> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-05-12%2005:30:14 >>> Weird. >>> >>> SELECT ''::text AS eleven, unique1, unique2, stringu1 >>> FROM onek WHERE unique1 < 50 >>> ORDER BY unique1 DESC LIMIT 20 OFFSET 39; >>> ! ERROR: could not open relation with OID 27035 >>> >>> AFAICS, the only way to get that error in HEAD is if ScanPgRelation >>> can't find a pg_class row with the mentioned OID. Presumably 27035 >>> belongs to "onek" or one of its indexes. The very next command also >>> refers to "onek", and doesn't fail, so what we seem to have here is >>> a transient lookup failure. We've found a btree bug like that once >>> before ... wonder if there's still one left? >> FYI: lionfish just managed to hit that problem again: >> >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-07-29%2023:30:06 > > The error message this time is > > ! ERROR: could not open relation with OID 27006 yeah and before it was: ! ERROR: could not open relation with OID 27035 which looks quite related :-) > > It's worth mentioning that the portals_p2 test, which happens in the > parallel group previous to where this test is run, also accesses the > onek table successfully. It may be interesting to see exactly what > relation is 27006. sorry but i don't have access to the cluster in question any more (lionfish is quite resource starved and I only enabled to keep failed builds on -HEAD after the last incident ...) > > The test alter_table, which is on the same parallel group as limit (the > failing test), contains these lines: > > ALTER INDEX onek_unique1 RENAME TO tmp_onek_unique1; > ALTER INDEX tmp_onek_unique1 RENAME TO onek_unique1; hmm interesting - lionfish is a slow box(250Mhz MIPS) and particulary low on memory(48MB+140MB swap) so it is quite likely that the parallel regress tests are driving it into swap - maybe some sort of subtile timing issue ? Stefan
Alvaro Herrera <alvherre@commandprompt.com> writes: > Stefan Kaltenbrunner wrote: >> FYI: lionfish just managed to hit that problem again: >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-07-29%2023:30:06 > The test alter_table, which is on the same parallel group as limit (the > failing test), contains these lines: > ALTER INDEX onek_unique1 RENAME TO tmp_onek_unique1; > ALTER INDEX tmp_onek_unique1 RENAME TO onek_unique1; I bet Alvaro's spotted the problem. ALTER INDEX RENAME doesn't seem to take any lock on the index's parent table, only on the index itself. That means that a query on "onek" could be trying to read the pg_class entries for onek's indexes concurrently with someone trying to commit a pg_class update to rename an index. If the query manages to visit the new and old versions of the row in that order, and the commit happens between, *neither* of the versions would look valid. MVCC doesn't save us because this is all SnapshotNow. Not sure what to do about this. Trying to lock the parent table could easily be a cure-worse-than-the-disease, because it would create deadlock risks (we've already locked the index before we could look up and lock the parent). Thoughts? The path of least resistance might just be to not run these tests in parallel. The chance of this issue causing problems in the real world seems small. regards, tom lane
On Sun, Jul 30, 2006 at 11:44:44AM -0400, Tom Lane wrote: > Alvaro Herrera <alvherre@commandprompt.com> writes: > > Stefan Kaltenbrunner wrote: > >> FYI: lionfish just managed to hit that problem again: > >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-07-29%2023:30:06 > > > The test alter_table, which is on the same parallel group as limit (the > > failing test), contains these lines: > > ALTER INDEX onek_unique1 RENAME TO tmp_onek_unique1; > > ALTER INDEX tmp_onek_unique1 RENAME TO onek_unique1; > > I bet Alvaro's spotted the problem. ALTER INDEX RENAME doesn't seem to > take any lock on the index's parent table, only on the index itself. > That means that a query on "onek" could be trying to read the pg_class > entries for onek's indexes concurrently with someone trying to commit > a pg_class update to rename an index. If the query manages to visit > the new and old versions of the row in that order, and the commit > happens between, *neither* of the versions would look valid. MVCC > doesn't save us because this is all SnapshotNow. > > Not sure what to do about this. Trying to lock the parent table could > easily be a cure-worse-than-the-disease, because it would create > deadlock risks (we've already locked the index before we could look up > and lock the parent). Thoughts? > > The path of least resistance might just be to not run these tests in > parallel. The chance of this issue causing problems in the real world > seems small. It doesn't seem that unusual to want to rename an index on a running system, and it certainly doesn't seem like the kind of operation that should pose a problem. So at the very least, we'd need a big fat warning in the docs about how renaming an index could cause other queries in the system to fail, and the error message needs to be improved. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
Jim C. Nasby wrote: > On Sun, Jul 30, 2006 at 11:44:44AM -0400, Tom Lane wrote: >> Alvaro Herrera <alvherre@commandprompt.com> writes: >>> Stefan Kaltenbrunner wrote: >>>> FYI: lionfish just managed to hit that problem again: >>>> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=lionfish&dt=2006-07-29%2023:30:06 >>> The test alter_table, which is on the same parallel group as limit (the >>> failing test), contains these lines: >>> ALTER INDEX onek_unique1 RENAME TO tmp_onek_unique1; >>> ALTER INDEX tmp_onek_unique1 RENAME TO onek_unique1; >> I bet Alvaro's spotted the problem. ALTER INDEX RENAME doesn't seem to >> take any lock on the index's parent table, only on the index itself. >> That means that a query on "onek" could be trying to read the pg_class >> entries for onek's indexes concurrently with someone trying to commit >> a pg_class update to rename an index. If the query manages to visit >> the new and old versions of the row in that order, and the commit >> happens between, *neither* of the versions would look valid. MVCC >> doesn't save us because this is all SnapshotNow. >> >> Not sure what to do about this. Trying to lock the parent table could >> easily be a cure-worse-than-the-disease, because it would create >> deadlock risks (we've already locked the index before we could look up >> and lock the parent). Thoughts? >> >> The path of least resistance might just be to not run these tests in >> parallel. The chance of this issue causing problems in the real world >> seems small. > > It doesn't seem that unusual to want to rename an index on a running > system, and it certainly doesn't seem like the kind of operation that > should pose a problem. So at the very least, we'd need a big fat warning > in the docs about how renaming an index could cause other queries in the > system to fail, and the error message needs to be improved. it is my understanding that Tom is already tackling the underlying issue on a much more general base ... Stefan
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> Jim C. Nasby wrote:
>> On Sun, Jul 30, 2006 at 11:44:44AM -0400, Tom Lane wrote:
>>> The path of least resistance might just be to not run these tests in
>>> parallel. The chance of this issue causing problems in the real world
>>> seems small.
>>
>> It doesn't seem that unusual to want to rename an index on a running
>> system, and it certainly doesn't seem like the kind of operation that
>> should pose a problem. So at the very least, we'd need a big fat warning
>> in the docs about how renaming an index could cause other queries in the
>> system to fail, and the error message needs to be improved.
> it is my understanding that Tom is already tackling the underlying issue
> on a much more general base ...
Done in HEAD, but we might still wish to think about changing the
regression tests in the back branches, else we'll probably continue to
see this failure once in a while ...
regards, tom lane
Tom Lane wrote: >Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: > > >>Jim C. Nasby wrote: >> >> >>>On Sun, Jul 30, 2006 at 11:44:44AM -0400, Tom Lane wrote: >>> >>> >>>>The path of least resistance might just be to not run these tests in >>>>parallel. The chance of this issue causing problems in the real world >>>>seems small. >>>> >>>> >>>It doesn't seem that unusual to want to rename an index on a running >>>system, and it certainly doesn't seem like the kind of operation that >>>should pose a problem. So at the very least, we'd need a big fat warning >>>in the docs about how renaming an index could cause other queries in the >>>system to fail, and the error message needs to be improved. >>> >>> > > > >>it is my understanding that Tom is already tackling the underlying issue >>on a much more general base ... >> >> > >Done in HEAD, but we might still wish to think about changing the >regression tests in the back branches, else we'll probably continue to >see this failure once in a while ... > > > > How sure are we that this is the cause of the problem? The feeling I got was "this is a good guess". If so, do we want to prevent ourselves getting any further clues in case we're wrong? It's also an interesting case of a (low likelihood) bug which is not fixable on any stable branch. cheers andrew
Andrew Dunstan wrote: > Tom Lane wrote: > >> Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: >> >> >>> Jim C. Nasby wrote: >>> >>>> On Sun, Jul 30, 2006 at 11:44:44AM -0400, Tom Lane wrote: >>>> >>>>> The path of least resistance might just be to not run these tests in >>>>> parallel. The chance of this issue causing problems in the real world >>>>> seems small. >>>>> >>>> It doesn't seem that unusual to want to rename an index on a running >>>> system, and it certainly doesn't seem like the kind of operation that >>>> should pose a problem. So at the very least, we'd need a big fat >>>> warning >>>> in the docs about how renaming an index could cause other queries in >>>> the >>>> system to fail, and the error message needs to be improved. >>>> >> >> >> >>> it is my understanding that Tom is already tackling the underlying issue >>> on a much more general base ... >>> >> >> Done in HEAD, but we might still wish to think about changing the >> regression tests in the back branches, else we'll probably continue to >> see this failure once in a while ... >> >> >> >> > > How sure are we that this is the cause of the problem? The feeling I got > was "this is a good guess". If so, do we want to prevent ourselves > getting any further clues in case we're wrong? It's also an interesting > case of a (low likelihood) bug which is not fixable on any stable branch. well I have a lot of trust into tom - though the main issue is that this issue seems to be difficult hard to trigger. afaik only one box (lionfish) ever managed to hit it and even there only 2 times out of several hundred builds - I don't suppose we can come up with a testcase that might be more reliably showing that issue ? Stefan
Stefan Kaltenbrunner wrote: > Andrew Dunstan wrote: > > How sure are we that this is the cause of the problem? The feeling I got > > was "this is a good guess". If so, do we want to prevent ourselves > > getting any further clues in case we're wrong? It's also an interesting > > case of a (low likelihood) bug which is not fixable on any stable branch. > > well I have a lot of trust into tom - though the main issue is that this > issue seems to be difficult hard to trigger. > afaik only one box (lionfish) ever managed to hit it and even there only > 2 times out of several hundred builds - I don't suppose we can come up > with a testcase that might be more reliably showing that issue ? Maybe we could write a suitable test case using Martijn's concurrent testing framework. Or with a pair of custom SQL script running under pgbench, and a separate process sending random SIGSTOP/SIGCONT to backends. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Maybe we could write a suitable test case using Martijn's concurrent
> testing framework.
The trick is to get process A to commit between the times that process B
looks at the new and old versions of the pg_class row (and it has to
happen to do so in that order ... although that's not a bad bet given
the way btree handles equal keys).
I think the reason we've not tracked this down before is that that's a
pretty small window. You could force the problem by stopping process B
with a debugger breakpoint and then letting A do its thing, but short of
something like that you'll never reproduce it with high probability.
As far as Andrew's question goes: I have no doubt that this race
condition is (or now, was) real and could explain Stefan's failure.
It's not impossible that there's some other problem in there, though.
If so we will still see the problem from time to time on HEAD, and
know that we have more work to do. But I don't think that continuing
to see it on the back branches will teach us anything.
regards, tom lane
Tom Lane wrote: > As far as Andrew's question goes: I have no doubt that this race > condition is (or now, was) real and could explain Stefan's failure. > It's not impossible that there's some other problem in there, though. > If so we will still see the problem from time to time on HEAD, and > know that we have more work to do. But I don't think that continuing > to see it on the back branches will teach us anything. > > > Fair enough. cheers andrew
Tom Lane wrote: > Alvaro Herrera <alvherre@commandprompt.com> writes: >> Maybe we could write a suitable test case using Martijn's concurrent >> testing framework. > > The trick is to get process A to commit between the times that process B > looks at the new and old versions of the pg_class row (and it has to > happen to do so in that order ... although that's not a bad bet given > the way btree handles equal keys). > > I think the reason we've not tracked this down before is that that's a > pretty small window. You could force the problem by stopping process B > with a debugger breakpoint and then letting A do its thing, but short of > something like that you'll never reproduce it with high probability. > > As far as Andrew's question goes: I have no doubt that this race > condition is (or now, was) real and could explain Stefan's failure. > It's not impossible that there's some other problem in there, though. > If so we will still see the problem from time to time on HEAD, and > know that we have more work to do. But I don't think that continuing > to see it on the back branches will teach us anything. maybe the following buildfarm report means that we need a new theory :-( http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=sponge&dt=2006-08-16%2021:30:02 Stefan
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: > maybe the following buildfarm report means that we need a new theory :-( > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=sponge&dt=2006-08-16%2021:30:02 Vacuum's always had a race condition: it makes a list of rel OIDs and then tries to vacuum each one. It narrows the window for failure by doing a SearchSysCacheExists test before relation_open, but there's still a window for failure. The rel in question is most likely a temp rel of another backend, because sanity_check is running by itself and so there shouldn't be anything else happening except perhaps some other session's post-disconnect cleanup. Maybe we could put the check for "is this a temp rel of another relation" into the initial list-making step instead of waiting till after relation_open. That doesn't seem to solve the general problem though. regards, tom lane
Tom Lane wrote: > Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes: >> maybe the following buildfarm report means that we need a new theory :-( > >> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=sponge&dt=2006-08-16%2021:30:02 > > Vacuum's always had a race condition: it makes a list of rel OIDs and > then tries to vacuum each one. It narrows the window for failure by > doing a SearchSysCacheExists test before relation_open, but there's > still a window for failure. > > The rel in question is most likely a temp rel of another backend, > because sanity_check is running by itself and so there shouldn't > be anything else happening except perhaps some other session's > post-disconnect cleanup. Maybe we could put the check for "is > this a temp rel of another relation" into the initial list-making > step instead of waiting till after relation_open. That doesn't > seem to solve the general problem though. hmm yeah - missed the VACUUM; part of the regression diff. Still this means we will have to live with (rare) failures once in a while during that test ? Stefan
> Alvaro Herrera <alvherre ( at ) commandprompt ( dot ) com> writes: >> Maybe we could write a suitable test case using Martijn's concurrent >> testing framework. > > The trick is to get process A to commit between the times that process B > looks at the new and old versions of the pg_class row (and it has to > happen to do so in that order ... although that's not a bad bet given > the way btree handles equal keys). > > I think the reason we've not tracked this down before is that that's a > pretty small window. You could force the problem by stopping process B > with a debugger breakpoint and then letting A do its thing, but short of > something like that you'll never reproduce it with high probability. Actually I was already looking into a related issue and have some work here that may help with this. I wanted to test the online index build and to do that I figured you needed to have regression tests like the ones we have now except with multiple database sessions. So I hacked psql to issue queries asynchronously and allow multiple database connections. That way you can switch connections while a blocked or slow transaction is still running and issue queries in other transactions. I thought it was a proof-of-concept kludge but actually it's worked out quite well. There were a few conceptual gotchas but I think I have a reasonable solution for each. The main issue was that any time you issue an asynchronously connection that you expect to block you have a race condition in the test. You can't switch connections and proceed right away or you may actually proceed with the other connection before the first connection's command is received and acted on by the backend. The "right" solution to this would involve altering the backend and the protocol to provide some form of feedback when an asynchronous query had reached various states including when it was blocked. You would have to annotate it with enough information that the client can determine it's actually blocked on the right thing and not just on some uninteresting transient lock too. Instead I just added a command to cause psql to wait for a time. This is nearly as good since all the regression tests run fairly quickly so if you wait even a fraction of a second you can be pretty certain the command has been received and if it were not going to block it would have finished and printed output already. And it was *much* simpler. Also, I think for interactive use we would want a somewhat more sophisticated scheduling of output. It would be nice to print out results as they come in even if we're on another connection. For the regression tests you certainly do not want that since that would introduce unavoidable non-deterministic race conditions in your output files all over the place. The way I've coded it now takes care to print out output only from the "active" database connection and the test cases need to be written to switch connections at each point they want to test for possibly incorrect output. Another issue was that I couldn't come up with a nice set of names for the commands that didn't conflict with the myriad of one-letter commands already in psql. So I just prefixed the all with "c" (connection). I figured when I submitted it I would just let the community hash out the names and take the 2s it would take to change them. The test cases are actually super easy to write and read, at least considering we're talking about concurrent sql sessions here. I think it's far clearer than trying to handle separate scripts and nearly as clear as Martin's proposal from a while back to prepend a connection number on every line. The commands I've added or altered are: \c[onnect][&] [DBNAME|- USER|- HOST|- PORT|-] connect to new database (currently "postgres") if optional & is present open do not close existing connection \cswitch n switch to database connection n\clist list database connections \cdisconnect close current database connection use \cswitch or \connect to select another connection \cnowait issue next query without waiting for results\cwait [n] if any queries are pending wait n seconds for results Also I added %& to the psql prompt format to indicate the current connection. So the tests look like, for example: postgres=# \c& [2] You are now connected to database "postgres". postgres[2]=# begin; BEGIN postgres[2]=# create table foo (a integer); CREATE TABLE postgres[2]=# \cswitch 1 [1] You are now connected to database "postgres" postgres[1]=# select * from foo; ERROR: relation "foo" does not exist postgres[1]=# \cswitch 2 [2] You are now connected to database "postgres" postgres[2]=# commit; COMMIT postgres[2]=# \cswitch 1 [1] You are now connected to database "postgres" postgres[1]=# select * from foo;a --- (0 rows) postgres[1]=# insert into foo values (1); INSERT 0 1 postgres[1]=# begin; BEGIN postgres[1]=# update foo set a = 2; UPDATE 1 postgres[1]=# \cswitch 2 [2] You are now connected to database "postgres" postgres[2]=# select * from foo;a ---1 (1 row) postgres[2]=# \cnowait postgres[2]=# update foo set a = 3; postgres[2]=# \cwait .1 postgres[2]=# \cswitch 1 [1] You are now connected to database "postgres" postgres[1]=# commit; COMMIT postgres[1]=# \cswitch 2 [2] You are now connected to database "postgres" UPDATE 1 postgres[2]=# \clist [1] Connected to database "postgres" [2] Connected to database "postgres" postgres[2]=# \cdisconnect Disconnecting from database (use \connect to reconnect or \cswitch to select another connection) !> \cswitch 1 [1] You are now connected to database "postgres" -- Gregory Stark EnterpriseDB http://www.enterprisedb.com
stark wrote: > Actually I was already looking into a related issue and have some work here > that may help with this. > > I wanted to test the online index build and to do that I figured you needed to > have regression tests like the ones we have now except with multiple database > sessions. So I hacked psql to issue queries asynchronously and allow multiple > database connections. That way you can switch connections while a blocked or > slow transaction is still running and issue queries in other transactions. > > I thought it was a proof-of-concept kludge but actually it's worked out quite > well. There were a few conceptual gotchas but I think I have a reasonable > solution for each. I have had an idea for some time that is actually much simpler -- just launch several backends at once to do different things, and randomly send SIGSTOP and SIGCONT to each. If they keep doing whatever they are doing in infinite loops, and you leave it enough time, it's very likely that you'll get problems if the concurrent locking (or whatever) is not right. The nice thing about this is that it's completely random, i.e. you don't have to introduce individual stop points in the backend (which may themselves hide some bugs). It acts (or at least, I expect it to act) just like the kernel gave execution to another process. The main difference with your approach is that I haven't tried it. -- Alvaro Herrera http://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support
On Thu, Aug 17, 2006 at 04:17:01PM +0100, stark wrote: > I wanted to test the online index build and to do that I figured you needed to > have regression tests like the ones we have now except with multiple database > sessions. So I hacked psql to issue queries asynchronously and allow multiple > database connections. That way you can switch connections while a blocked or > slow transaction is still running and issue queries in other transactions. Wow, that's damn cool! FWIW, one thing I can think of that would be useful is the ability to 'background' a long-running query. I see \cnowait, but having something like & from unix shells would be even easier. It'd also be great to have the equivalent of ^Z so that if you got tired of waiting on a query, you could get back to the psql prompt without killing it. > Also, I think for interactive use we would want a somewhat more sophisticated > scheduling of output. It would be nice to print out results as they come in > even if we're on another connection. For the regression tests you certainly do > not want that since that would introduce unavoidable non-deterministic race > conditions in your output files all over the place. The way I've coded it now > takes care to print out output only from the "active" database connection and > the test cases need to be written to switch connections at each point they > want to test for possibly incorrect output. Thinking in terms of tcsh & co, there's a number of ways to handle this: 1) Output happens real-time 2) Only output from current connection (what you've done) 3) Only output after user input (ie: code that handles output is only run after the user has entered a command). I thinkmost shells operate this way by default. 4) Provide an indication that output has come in from a background connection, but don't provide the actual output. Thiscould be combined with #3. #3 is nice because you won't get interrupted in the middle of entering some long query. #4 could be useful for automated testing, especially if the indicator was routed to another output channel, such as STDERR. > Another issue was that I couldn't come up with a nice set of names for the > commands that didn't conflict with the myriad of one-letter commands already > in psql. So I just prefixed the all with "c" (connection). I figured when I > submitted it I would just let the community hash out the names and take the 2s > it would take to change them. > > The test cases are actually super easy to write and read, at least considering > we're talking about concurrent sql sessions here. I think it's far clearer > than trying to handle separate scripts and nearly as clear as Martin's > proposal from a while back to prepend a connection number on every line. > > The commands I've added or altered are: > > \c[onnect][&] [DBNAME|- USER|- HOST|- PORT|-] > connect to new database (currently "postgres") > if optional & is present open do not close existing connection > \cswitch n > switch to database connection n I can see \1 - \9 as being a handy shortcut. > \clist > list database connections > \cdisconnect > close current database connection > use \cswitch or \connect to select another connection Would ^d have the same effect? -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
On Thu, Aug 17, 2006 at 03:09:30PM -0400, Alvaro Herrera wrote: > stark wrote: > > > Actually I was already looking into a related issue and have some work here > > that may help with this. > > > > I wanted to test the online index build and to do that I figured you needed to > > have regression tests like the ones we have now except with multiple database > > sessions. So I hacked psql to issue queries asynchronously and allow multiple > > database connections. That way you can switch connections while a blocked or > > slow transaction is still running and issue queries in other transactions. > > > > I thought it was a proof-of-concept kludge but actually it's worked out quite > > well. There were a few conceptual gotchas but I think I have a reasonable > > solution for each. > > I have had an idea for some time that is actually much simpler -- just > launch several backends at once to do different things, and randomly > send SIGSTOP and SIGCONT to each. If they keep doing whatever they are > doing in infinite loops, and you leave it enough time, it's very likely > that you'll get problems if the concurrent locking (or whatever) is not > right. This is probably worth doing as well, since it would simulate what an IO-bound system would look like. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
"Jim C. Nasby" <jnasby@pervasive.com> writes:
> On Thu, Aug 17, 2006 at 03:09:30PM -0400, Alvaro Herrera wrote:
>> I have had an idea for some time that is actually much simpler -- just
>> launch several backends at once to do different things, and randomly
>> send SIGSTOP and SIGCONT to each. If they keep doing whatever they are
>> doing in infinite loops, and you leave it enough time, it's very likely
>> that you'll get problems if the concurrent locking (or whatever) is not
>> right.
> This is probably worth doing as well, since it would simulate what an
> IO-bound system would look like.
While that might be useful for testing, it'd absolutely suck for
debugging, because of the difficulty of reproducing a problem :-(
regards, tom lane
Am Donnerstag, 17. August 2006 17:17 schrieb stark: > Instead I just added a command to cause psql to wait for a time. Do we need the full multiple-connection handling command set, or would asynchronous query support and a wait command be enough? -- Peter Eisentraut http://developer.postgresql.org/~petere/
On Fri, Aug 18, 2006 at 02:46:39PM +0200, Peter Eisentraut wrote: > Am Donnerstag, 17. August 2006 17:17 schrieb stark: > > Instead I just added a command to cause psql to wait for a time. > > Do we need the full multiple-connection handling command set, or would > asynchronous query support and a wait command be enough? I am interested in this too. For example the tool I posted a while ago supported only this. It controlled multiple connections and only supported sending async & wait. It is enough to support fairly deterministic scenarios, for example, testing if the locks block on eachother as documented. However, it works less well for non-deterministic testing. Yet, a test-suite has to be deterministic, right? From a client side, is there any testing method better than async and wait? I've wondered about a tool that attached to the backend with gdb and for testing killed the backend when it hit a particular function. By selecting different functions each time, once you'd covered a lot of functions and tested recovery, you could have a good idea if the recovery code works properly. Has anyone seens a tool like that? Have a nice day, -- Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/ > From each according to his ability. To each according to his ability to litigate.
stark wrote: >> Alvaro Herrera <alvherre ( at ) commandprompt ( dot ) com> writes: >> >>> Maybe we could write a suitable test case using Martijn's concurrent >>> testing framework. >>> >> The trick is to get process A to commit between the times that process B >> looks at the new and old versions of the pg_class row (and it has to >> happen to do so in that order ... although that's not a bad bet given >> the way btree handles equal keys). >> >> I think the reason we've not tracked this down before is that that's a >> pretty small window. You could force the problem by stopping process B >> with a debugger breakpoint and then letting A do its thing, but short of >> something like that you'll never reproduce it with high probability. >> > > Actually I was already looking into a related issue and have some work here > that may help with this. > > I wanted to test the online index build and to do that I figured you needed to > have regression tests like the ones we have now except with multiple database > sessions. So I hacked psql to issue queries asynchronously and allow multiple > database connections. That way you can switch connections while a blocked or > slow transaction is still running and issue queries in other transactions. > > I thought it was a proof-of-concept kludge but actually it's worked out quite > well. There were a few conceptual gotchas but I think I have a reasonable > solution for each. > > [snip] Can you please put the patch up somewhere so people can see what's involved? thanks cheers andrew
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
> Tom Lane wrote:
>> Vacuum's always had a race condition: it makes a list of rel OIDs and
>> then tries to vacuum each one. It narrows the window for failure by
>> doing a SearchSysCacheExists test before relation_open, but there's
>> still a window for failure.
> hmm yeah - missed the VACUUM; part of the regression diff.
> Still this means we will have to live with (rare) failures once in a
> while during that test ?
I thought of what seems a pretty simple solution for this: make VACUUM
lock the relation before doing the SearchSysCacheExists, ie instead
of the existing code
if (!SearchSysCacheExists(RELOID, ObjectIdGetDatum(relid),
0,0, 0)) // give up
lmode = vacstmt->full ? AccessExclusiveLock : ShareUpdateExclusiveLock;
onerel = relation_open(relid, lmode);
do
lmode = vacstmt->full ? AccessExclusiveLock : ShareUpdateExclusiveLock;
LockRelationOid(relid, lmode);
if (!SearchSysCacheExists(RELOID, ObjectIdGetDatum(relid),
0,0, 0)) // give up
onerel = relation_open(relid, NoLock);
Once we're holding lock, we can be sure there's not a DROP TABLE in
progress, so there's no race condition anymore. It's OK to take a
lock on the OID of a relation that no longer exists, AFAICS; we'll
just drop it again immediately (the "give up" path includes transaction
exit, so there's not even any extra code needed).
This wasn't possible before the recent adjustments to the relation
locking protocol, but now it looks trivial ... am I missing anything?
Perhaps it is worth folding this test into a "conditional_relation_open"
function that returns NULL instead of failing if the rel no longer
exists. I think there are potential uses in CLUSTER and perhaps REINDEX
as well as VACUUM.
regards, tom lane
Andrew Dunstan <andrew@dunslane.net> writes: > stark wrote: > > > So I hacked psql to issue queries asynchronously and allow multiple > > database connections. That way you can switch connections while a blocked > > or slow transaction is still running and issue queries in other > > transactions. > > [snip] > > Can you please put the patch up somewhere so people can see what's involved? I'll send it to pgsql-patches. -- Gregory Stark EnterpriseDB http://www.enterprisedb.com