Re: [HACKERS] TAP tests take a long time - Mailing list pgsql-hackers

From Mithun Cy
Subject Re: [HACKERS] TAP tests take a long time
Date
Msg-id CAD__Ouh-qaEb+rD7Uy-4g3xQYOrhPzHs-a_TrFAjiQ5azAW5+w@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] TAP tests take a long time  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: [HACKERS] TAP tests take a long time  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
On Wed, Apr 12, 2017 at 6:24 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> I have looked into the tests and I think we can do some optimization
> without losing much on code coverage.  First is we are doing both
> Vacuum Full and Vacuum on hash_split_heap in the same test after
> executing few statements, it seems to me that we can avoid doing
> Vacuum Full.  Also, I think we can try by reducing the number of
> inserts in hash_split_heap to see if we can achieve the code coverage
> of split bucket code path.  Finally, I am not sure if Reindex Index
> hash_split_index is required for code coverage, but we might need it
> for the sake of testing that operation.
>
> Mithun, as you are the original author of these tests, can you please
> try some of the above optimizations and any others you can think of
> and see if we can reduce the time for hash index tests?

I have tried to optimize the tests maintaining the same coverage we were able to get with it.

Original coverage

Lines:      1466     2341      62.6 %
Functions: 79        101       78.2 %

After test changes

Lines:       1534    2341     65.5 %
Functions: 79         101     78.2 %

changes done
Earlier :

CREATE INDEX hash_split_index on hash_split_heap USING HASH (keycol);        
+ Time: 8.950 ms                                                                
INSERT INTO hash_split_heap SELECT 1 FROM generate_series(1, 70000) a;
+ Time: 3135.098 ms (00:03.135)                                                
VACUUM FULL hash_split_heap;                                                  
+ Time: 2748.146 ms (00:02.748)
REINDEX INDEX hash_split_index;                                              
+ Time: 114.290 ms

Insertion and vacuum full were taking most of the time in tests. I have removed the vacuum full as suggested by you, which is taken care by vacuum statement at the end, And reduced the number of rows inserted to trigger a split, by inserting some records before creating hash index. This way we start with less number of buckets and thus splits will happen much early.

Now :

+ INSERT INTO hash_split_heap SELECT 1 FROM generate_series(1, 500) a;          
+ Time: 2.946 ms                                                                
CREATE INDEX hash_split_index on hash_split_heap USING HASH (keycol);        
! Time: 8.933 ms                                                                
! INSERT INTO hash_split_heap SELECT 1 FROM generate_series(1, 5000) a;        
! Time: 81.674 ms

Also removed REINDEX INDEX as suggested which was not adding for coverage. With this, I think tests should run significantly faster now.

--
Thanks and Regards
Mithun C Y
EnterpriseDB: http://www.enterprisedb.com
Attachment

pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: [HACKERS] Interval for launching the table sync worker
Next
From: Fujii Masao
Date:
Subject: Re: [HACKERS] Tab completion support for ALTER SUBSCRIPTION REFRESH PUBLICATION