Re: [HACKERS] Automatic testing of patches in commit fest - Mailing list pgsql-hackers

From Thomas Munro
Subject Re: [HACKERS] Automatic testing of patches in commit fest
Date
Msg-id CAEepm=21syq-=AbM4npOWd4BWMH8+rAQ9JU1hQv_ruX=b3XThQ@mail.gmail.com
Whole thread Raw
In response to [HACKERS] Automatic testing of patches in commit fest  (Michael Paquier <michael.paquier@gmail.com>)
Responses Re: [HACKERS] Automatic testing of patches in commit fest
List pgsql-hackers
On Mon, Sep 11, 2017 at 11:40 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> Thomas Munro has hacked up a prototype of application testing
> automatically if patches submitted apply and build:
> http://commitfest.cputube.org/
>
> I would recommend have a look at it from time to time if you are a
> patch author (or a reviewer) as any failure may say that your patch
> has rotten over time and needs a rebase. It is good to keep the commit
> fest entries build-clean at least for testers.

I should add: this is a spare-time effort, a work-in-progress and
building on top of a bunch of hairy web scraping, so it may take some
time to perfect.  One known problem at the moment is that the mail
archive (where patches are fetched from) cuts off long threads so it
doesn't manage to find the latest version of (for example) the
Partition-Wise Join patch.  Other problems include getting confused by
incidental material posted in .patch form, patches that depend on
other patches or patches whose apply order is not obvious, or ... etc
etc.  There are also a few other patches missing currently for various
reasons, which I intend to fix, gradually.

That said, it might already be useful for catching bitrot and things
like TAP test failures, contrib regressions and documentation build
errors which (based on the evidence I've seen so far) many submitters
aren't actually running themselves.  YMMV.

I've been having conversations with Magnus and Stephen about what a
more robust version integrated with the CF app might eventually look
like.  The basic idea here is:  Commitfest submissions are analogous
to pull request, which many comparable projects test automatically
using CI technology that is generously made available to open source
projects and well integrated with popular free git hosts.  We should
too!

-- 
Thomas Munro
http://www.enterprisedb.com


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: [HACKERS] Statement timeout behavior in extended queries
Next
From: Thomas Munro
Date:
Subject: Re: [HACKERS] Automatic testing of patches in commit fest