Upgrading cluster with thousands of DBs - Mailing list pgsql-admin

From Alejandro Martínez
Subject Upgrading cluster with thousands of DBs
Date
Msg-id CAOx6PCJe-+=Y70fk7c5LcJ5aUqxpnXHtHpP34Sg8hS7nmMV6YQ@mail.gmail.com
Whole thread Raw
Responses Re: Upgrading cluster with thousands of DBs
Re: Upgrading cluster with thousands of DBs
List pgsql-admin
Hi list,

We're working on upgrading from 9.1 to 9.3 some of our database
clusters, while seeking to minimize (write) downtime by making the
upgrades as fast as possible.

However, our clusters have the particularity that they get to host
many different *databases*, as in, ~2000 databases per cluster. Most
of those databases are small (only a table and PostGIS functions), so
it's more a problem of the number of DBs than the space they use. The
total size of the cluster is around 60gb.

I first started using pg_upgrade but, even when enabling
parallelization and hardlinks, as most of the steps (preliminary
check?) don't seem to be parallelized, the upgrade takes ~5 hours as
it goes database-per-database serially on most steps.

I'm experimenting with reducing the time by spawning parallel pg_dump
and pg_restore instances, so far I've reached a 3:15h mark that I
still think can reduce a bit more with some tweaking.

Does anyone have any tips on reducing the time of upgrading such a
kind of a cluster?


Thanks!


pgsql-admin by date:

Previous
From: "Andrew O. Zhukov"
Date:
Subject: Not fatal error
Next
From: Gabriele Bartolini
Date:
Subject: Re: Upgrading cluster with thousands of DBs