Re: Continuing instability in insert-conflict-specconflict test - Mailing list pgsql-hackers

On Sun, Jun 13, 2021 at 04:48:48PM -0400, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > On Tue, Aug 25, 2020 at 12:04:41PM -0400, Tom Lane wrote:
> >> I think what we have to do to salvage this test is to get rid of the
> >> use of NOTICE outputs, and instead have the test functions insert
> >> log records into some table, which we can inspect after the fact
> >> to verify that things happened as we expect.
> 
> > That sounds promising.  Are those messages important for observing server
> > bugs, or are they for debugging/modifying the test itself? If the latter, one
> > could just change the messages to LOG.
> 
> I think they are important, because they show that the things we expect
> to happen occur when we expect them to happen.
> 
> I experimented with replacing the RAISE NOTICEs with INSERTs, and ran
> into two problems:
> 
> 1. You can't do an INSERT in an IMMUTABLE function.  This is easily
> worked around by putting the INSERT in a separate, volatile function.
> That's cheating like mad of course, but so is the rest of the stuff
> this test does in "immutable" functions.
> 
> 2. The inserted events don't become visible from the outside until the
> respective session commits.  This seems like an absolute show-stopper.
> After the fact, we can see that the events happened in the expected
> relative order; but we don't have proof that they happened in the right
> order relative to the actions visible in the test output file.

One could send the inserts over dblink, I suppose.

> > ... Any of the above won't solve things
> > completely, because 3 of the 21 failures have diffs in the pg_locks output:

> * Adjust the test script's functions to emit a NOTICE *after* acquiring
> a lock, not before.

I suspect that particular lock acquisition merely unblocks the processing that
reaches the final lock state expected by the test.  So, ...

> * Annotate permutations with something along the lines of "expect N
> NOTICE outputs before allowing this step to be considered complete",
> which we'd attach to the unlock steps.

... I don't expect this to solve $SUBJECT.  It could be a separately-useful
feature, though.

> This idea is only half baked at present, but maybe there's something
> to work with there.  If it works, maybe we could improve the other
> test cases that are always pseudo-failing in a similar way.  For
> example, in the deadlock tests, annotate steps with "expect step
> Y to finish before step X".

Yeah, a special permutation list entry like PQgetResult(s8) could solve
failures like
http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=anole&dt=2021-06-11%2017%3A13%3A44

Incidentally, I have a different idle patch relevant to deadlock test failures
like that.  Let me see if it has anything useful.