Thread: LISTEN/NOTIFY regression tests

This patch adds some primitive regression tests for the LISTEN,
NOTIFY, and UNLISTEN commands -- there were previously no regression
tests for this functionality.

There are currently two problems/caveats with the patch:

      (1) The regression test can never pass, because the message that
          psql produces when a signal is delivered always changes
          slightly between runs of the regression tests, since the
          backend's PID is not constant:

Asynchronous notification "my_cond" received from server process with
PID 15942.

          How should I fix this? What we'd like to be able to say is
          "ensure that the output of the command matches this regular
          expression", rather than the mere equality comparison (with
          allowances for whitespace, etc.) that we are using right
          now.

      (2) At present, we only test notifications that are delivered to
          our own backend. That means we're not testing the code path
          that delivers notifications to other backends (which is
          probably the one more likely to contain bugs). The tests
          I've written are better than nothing, but it would be nice
          to be able to test this functionality more completely.

          The problem with listening for signals in one backend and
          delivering signals in another backend is that we need some
          way to do synchronization between the backends (for example,
          we don't want to execute a NOTIFY on a signal before the
          other backend has had a chance to LISTEN for it). I was
          thinking of implementing this by acquiring exclusive locks
          on a dummy table, but that is kind of a hack, and has its
          own concurrency problems (for example, we need to ensure the
          CREATE TABLE precedes the LISTEN & NOTIFY tests). Any
          thoughts?

          One possibility is just writing a separate little C
          application that does the LISTEN/NOTIFY testing itself, and
          is invoked by the regression test script. The same technique
          would be useful for doing some testing of VACUUM, for
          example. Does anyone else think this would be useful?

#1 needs to be fixed before this patch can be applied, but #2 will
#probably remain an area for future work.

-Neil

Neil Conway <neilc@samurai.com> writes:
> There are currently two problems/caveats with the patch [...]

I just wanted to check that no one had any input or comments on the
issues described in the original email -- "Speak now, or forever..."
:-)

I'm thinking of solving both problems by doing LISTEN/NOTIFY testing
via a separate small C app that is invoked by the regression test
script, but I'm still open to suggestions.

-Neil

Neil Conway <neilc@samurai.com> writes:
> This patch adds some primitive regression tests for the LISTEN,
> NOTIFY, and UNLISTEN commands -- there were previously no regression
> tests for this functionality.

These cannot usefully be tested with our current regression test
methodology, as you observe in your comments.

>           One possibility is just writing a separate little C
>           application that does the LISTEN/NOTIFY testing itself, and
>           is invoked by the regression test script. The same technique
>           would be useful for doing some testing of VACUUM, for
>           example. Does anyone else think this would be useful?

We have previously speculated about setting up some sort of test harness
program that can drive multiple backends through a coordinated series of
operations.  This would allow realistic testing of LOCK, LISTEN/NOTIFY,
MVCV semantics, etc, so it shouldn't be single-purpose --- it should be
script-driven somehow.  I think we had talked about scripts with
contents like
    [to backend 1]
    some SQL commands...
    [to backend 2]
    some SQL commands...
    [to backend 1]
    some more SQL commands...
but I can't recall if anyone had good ideas about how to describe the
expected output.  If you search the archives you can probably find
the previous discussions --- it was a couple years ago.

            regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:
> We have previously speculated about setting up some sort of test
> harness program that can drive multiple backends through a
> coordinated series of operations.

It would also be useful to test other concurrent properties of the
backend: for example, "all interleavings of the following commands
yield the same result." I agree this would be useful, although I doubt
I'll personally implement it any time soon.

> If you search the archives you can probably find the previous
> discussions --- it was a couple years ago.

I searched a couple different archives of -hackers, but I didn't find
anything that looked relevant. Do you have a useful keyword from the
previous discussion, perhaps?

-Neil

On Thu, 20 Nov 2003 21:31:20 -0500, Neil Conway <neilc@samurai.com>
wrote:
>I searched a couple different archives of -hackers, but I didn't find
>anything that looked relevant.

It's on -patches:
http://archives.postgresql.org/pgsql-patches/2002-08/msg00583.php

Servus
 Manfred