Re: pg_upgrade --jobs - Mailing list pgsql-general

From Adrian Klaver
Subject Re: pg_upgrade --jobs
Date
Msg-id 1c0469fb-1878-35ac-8468-78ce47299638@aklaver.com
Whole thread Raw
In response to pg_upgrade --jobs  (senor <frio_cervesa@hotmail.com>)
Responses Re: pg_upgrade --jobs  (senor <frio_cervesa@hotmail.com>)
List pgsql-general
On 4/6/19 11:44 AM, senor wrote:
> The pg_upgrade --jobs option is not passed as an argument when it calls pg_dump. I haven't found anything in docs or
forumsmentioning a reason for not supporting under certain circumstances other than possibly for pre-9.2. The
pg_upgradedocs page states that it allows multiple CPUs to be used for dump and reload of schemas. Some databases I'm
upgradinghave 500,000+ tables and running with a single process is greatly increasing the upgrade time.
 
> 
> I am also using the --link option.
> I have tried "--jobs 20", "--jobs=20", placing this option first and last and many other variations.
> I am upgrading 9.2.4 to 9.6.12 on CentOS 6.
> Varying hardware but all with 32+ CPU cores.
> 
> su - postgres -c "/usr/pgsql-9.6/bin/pg_upgrade --jobs=20 --link \
>         --old-bindir=/usr/pgsql-9.2/bin/ \
>         --new-bindir=/usr/pgsql-9.6/bin/ \
>         --old-datadir=/var/lib/pgsql/9.2/data/ \
>         --new-datadir=/var/lib/pgsql/9.6/data/"
> 
> I feel like there's a simple reason I've missed but this seems pretty straight forward.

https://www.postgresql.org/docs/9.6/app-pgdump.html

"--jobs=njobs

     Run the dump in parallel by dumping njobs tables simultaneously. 
This option reduces the time of the dump but it also increases the load 
on the database server. You can only use this option with the directory
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
output format because this is the only output format where multiple
^^^^^^^^^^^^^
processes can write their data at the same time."


> A secondary plan would be to find instructions for doing the same as "pg_upgrade --link" manually so I can run
"pg_dump--jobs 20".
 
> Any assist is appreciated.
> Thanks,
> S. Cervesa
> 
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



pgsql-general by date:

Previous
From: senor
Date:
Subject: pg_upgrade --jobs
Next
From: senor
Date:
Subject: Re: pg_upgrade --jobs