Thread: pgsql: Un-break parallel pg_upgrade.

Un-break parallel pg_upgrade.

Commit b3f840120 changed pg_upgrade so that it'd actually drop and
re-create the template1 and postgres databases in the new cluster.
That works fine, serially.  With the -j option it's not so fine, because
other per-database jobs might be launched while the template1 database is
dropped.  Since they attempt to connect there to start up, kaboom.

This is the cause of the intermittent failures buildfarm member jacana
has been showing for the last month; evidently it is the only BF member
configured to run the pg_upgrade test with parallelism enabled.

Fix by processing template1 separately before we get into the parallel
sub-job launch loop.  (We could alternatively have made the postgres DB
be the special case, but it seems likely that template1 will contain
less stuff and so we lose less parallelism with this choice.)

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/5b570d771b80aadc98755208f8f1b81e9a5eb366

Modified Files
--------------
src/bin/pg_upgrade/pg_upgrade.c | 59 +++++++++++++++++++++++++++++++++--------
1 file changed, 48 insertions(+), 11 deletions(-)

On Sun, Feb 25, 2018 at 11:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Un-break parallel pg_upgrade.
> This is the cause of the intermittent failures buildfarm member jacana
> has been showing for the last month; evidently it is the only BF member
> configured to run the pg_upgrade test with parallelism enabled.

Seeing this I've looked at how jacana does it and configured my animal
(katydid) to set PG_UPGRADE_OPTS to '-j 4'. I did a bunch of test runs
of the tree just before this commit and confirmed that the pg_upgrade
test fails like jacana did, with this commit I saw no failure.

I imagine it would be valuable to better document these kinds of test
options (PG_UPGRADE_OPTS, EXEC_BACKEND, CLOBBER_CACHE_ALWAYS etc.) for
buildfarm owners. That might trigger owners to add some to their
animal and therefore increase diversity and coverage. Without some
docs, non core developers that do run animals will not know about
them. The build farm config script has some docs in the form of
commented out options and some text describing them, would it be worth
adding more options there? PG_UPGRADE_OPTS => '-j 4' would then be a
commented out example in the build_env setting.