Re: [HACKERS] TAP tests take a long time - Mailing list pgsql-hackers

From Craig Ringer
Subject Re: [HACKERS] TAP tests take a long time
Date
Msg-id CAMsr+YH9WWg5M3EMg2LHQB_as89o83UjuLhvqrtpGmKLC0goaw@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] TAP tests take a long time  (Andrew Dunstan <andrew.dunstan@2ndquadrant.com>)
Responses Re: [HACKERS] TAP tests take a long time  (Michael Paquier <michael.paquier@gmail.com>)
Re: [HACKERS] TAP tests take a long time  (Andrew Dunstan <andrew.dunstan@2ndquadrant.com>)
List pgsql-hackers


On 12 Apr. 2017 03:14, "Andrew Dunstan" <andrew.dunstan@2ndquadrant.com> wrote:


On 04/11/2017 12:08 PM, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Tue, Apr 11, 2017 at 11:32 AM, Andrew Dunstan
>> <andrew.dunstan@2ndquadrant.com> wrote:
>>> This buildfarm run as you can see takes 33m32s, and the Tap tests take a
>>> combined 19m52s of that time.
>> I don't think it's quite fair to complain about the TAP tests taking a
>> long time to run as a general matter.  Many people here have long
>> wanted a separate test-suite for long running tests that can be run to
>> really check everything possible, and apparently, TAP tests are it.
>> What I think would be more useful is to break down where the time is
>> getting spent.  It may be that some of those tests are not adding
>> value proportionate to their runtime.



Well, you can get a lot of information on timings in crake's latest
builds for example, or nightjar's.

Here's an interesting fact: almost all the time taken up in the scripts
test set seems to be consumed in running initdb over and over.


Is it worth adding an init(cache => 1) option to PostgresNode, where we stash an initdb'd db in tmp_check/  and just do a simple fs copy of it ? 

Even default to caching and allow an option to disable the cached copy.

We're going to need to initdb a lot in the TAP tests. We often need a clean state. Tests also get harder to debug the more you pack into a single test run and more difficult to run individually to test some specific failure. So IMO the best thing is to try to make that repeat initdb as fast as possible.

It reduces our coverage of initdb only incredibly slightly - all that repeat runs will do is help find very uncommon intermittent failures. And we rerun the buildfarm all the time so it's not like there's a shortage of initdb runs anyway.

We should also have a debug --no-fsync option for initdb, or maybe allow it to take -o options to pass to child postgres so we can pass fsync=off .

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [HACKERS] Why does logical replication launcher set application_name?
Next
From: Michael Paquier
Date:
Subject: Re: [HACKERS] TAP tests take a long time