Thread: pg_upgrade too slow on vacuum phase

On 5/12/20 5:27 AM, Kouber Saparev wrote:
> I am trying to upgrade PostgreSQL from 9.5 to 12 using pg_upgrade.
> 
> /usr/lib/postgresql/12/bin/pg_upgrade \
>    --old-datadir=/var/lib/postgresql/9.5/main \
>    --new-datadir=/var/lib/postgresql/12/main \
>    --old-bindir=/usr/lib/postgresql/9.5/bin \
>    --new-bindir=/usr/lib/postgresql/12/bin \
>    --old-options '-c config_file=/etc/postgresql/9.5/main/postgresql.conf' \
>    --new-options '-c config_file=/etc/postgresql/12/main/postgresql.conf' \
>    --link \
>    --jobs=16
> 
> It takes 47 minutes for the upgrade to finish (for a 28 GB database). It 
> hangs on two particular steps:
> 
> Analyzing all rows in the new cluster                       ok
> Freezing all rows in the new cluster                        ok

Are you sure?

 From here:

~/src/bin/pg_upgrade/pg_upgrade.c

prepare_new_cluster(void)
{
         /*
          * It would make more sense to freeze after loading the schema, 
but that
          * would cause us to lose the frozenids restored by the load. 
We use
          * --analyze so autovacuum doesn't update statistics later
          */
         prep_status("Analyzing all rows in the new cluster");
         exec_prog(UTILITY_LOG_FILE, NULL, true, true,
                           "\"%s/vacuumdb\" %s --all --analyze %s",
                           new_cluster.bindir, 
cluster_conn_opts(&new_cluster),
                           log_opts.verbose ? "--verbose" : "");
         check_ok();



/* -- NEW -- */
start_postmaster(&new_cluster, true);

check_new_cluster();
report_clusters_compatible();

pg_log(PG_REPORT,
             "\n"
             "Performing Upgrade\n"
             "------------------\n");

prepare_new_cluster();

stop_postmaster(false);

/*
     * Destructive Changes to New Cluster
     */

copy_xact_xlog_xid();

/* New now using xids of the old system */

/* -- NEW -- */
start_postmaster(&new_cluster, true);

prepare_new_globals();

create_new_objects();

stop_postmaster(false);


So the analyze(and freeze) are done before the new cluster are fully 
populated. Is the time being taken maybe for the loading schema/data 
portion?

> 
> Which basically runs:
> 
> vacuumdb --all --analyze
> vacuumdb --all --freeze
> 
> This is where all these 47 minutes are spent, yet I do not understand 
> neither why, nor how I can improve this part. Can I skip it somehow and 
> launch the vacuum manually afterwards?
> 
> Per postgresql.conf, I gave:
> 
> work_mem = 128MB
> maintenance_work_mem = 8GB
> max_parallel_maintenance_workers = 16
> max_parallel_workers = 16
> 
> The server has 44 GB available memory, and 24 cores.
> 
> Do you have any ideas how to speed-up the entire process?
> 
> --
> Kouber Saparev


-- 
Adrian Klaver
adrian.klaver@aklaver.com