Thread: Test suite fails on non-default configuration

I just built a new jar file from a checkout of CVS HEAD, and (after
fixing up the UNICODE versus UTF8 issue) the unit tests still failed
on me:

runtest:
    [junit] Testsuite: org.postgresql.test.jdbc2.Jdbc2TestSuite
    [junit] Tests run: 293, Failures: 1, Errors: 0, Time elapsed:
35.78 sec
    [junit]
    [junit] Testcase:
testTransactionIsolation(org.postgresql.test.jdbc2.ConnectionTest):
     FAILED
    [junit] expected:<2> but was:<8>
    [junit] junit.framework.AssertionFailedError: expected:<2> but
was:<8>
    [junit]     at
org.postgresql.test.jdbc2.ConnectionTest.testTransactionIsolation(
ConnectionTest.java:210)
    [junit]
    [junit]
    [junit] Test org.postgresql.test.jdbc2.Jdbc2TestSuite FAILED

The cause is that the cluster I was running against has a
non-default postgresql.conf file -- default_transaction_isolation is
set to serializable, which runs afoul of this:

    public void testTransactionIsolation() throws Exception
    {
        con = TestUtil.openDB();

        // PostgreSQL defaults to READ COMMITTED
        assertEquals(Connection.TRANSACTION_READ_COMMITTED,
                     con.getTransactionIsolation());

Should the unit tests really be failing based on configuration
options?

-Kevin

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
>         // PostgreSQL defaults to READ COMMITTED
>         assertEquals(Connection.TRANSACTION_READ_COMMITTED,
>                      con.getTransactionIsolation());

> Should the unit tests really be failing based on configuration
> options?

Well, for a comparison point, you can certainly make the server's
regression tests fail too if you whack the configuration around enough.
I agree that the above seems perhaps overly fragile, but I don't think
you can expect to have a general principle that the JDBC tests should
pass regardless of configuration.  It's more a question of how painful
is it to support any particular nondefault configuration choice, and
does it seem worth it based on real-world usage of the choice.

            regards, tom lane

On Mon, 18 Apr 2011, Tom Lane wrote:

> "Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
>>         // PostgreSQL defaults to READ COMMITTED
>>         assertEquals(Connection.TRANSACTION_READ_COMMITTED,
>>                      con.getTransactionIsolation());
>
>> Should the unit tests really be failing based on configuration
>> options?
>
> Well, for a comparison point, you can certainly make the server's
> regression tests fail too if you whack the configuration around enough.
> I agree that the above seems perhaps overly fragile, but I don't think
> you can expect to have a general principle that the JDBC tests should
> pass regardless of configuration.  It's more a question of how painful
> is it to support any particular nondefault configuration choice, and
> does it seem worth it based on real-world usage of the choice.
>

Exactly.  In this case the tests were assuming too much for no real
purpose.  I've checked in a fix to work with any default isolation level.

Kris Jurka

Kris Jurka <books@ejurka.com> wrote:

> In this case the tests were assuming too much for no real purpose.

Yeah, in that case it didn't seem to be testing much beyond whether
that configuration option had been changed from its default.

> I've checked in a fix to work with any default isolation level.

Thanks!

-Kevin