Thread: multiple sessions to create index in parallel support
Hi,
We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes , with only one connection , the create index automatically run in parallel (table size 40G) , then we started two connections and each session creating index on large table, but at this time, only 1 session parallel creating index, the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 session allowed to parallel index creating at the same time ?
Thanks,
James
"James Pang (chaolpan)" <chaolpan@cisco.com> writes: > We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes, with only one connection , the create index automatically run in parallel (table size 40G) , then we started twoconnections and each session creating index on large table, but at this time, only 1 session parallel creating index,the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 sessionallowed to parallel index creating at the same time ? Check your settings for max_parallel_workers and max_parallel_maintenance_workers. regards, tom lane
Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a large tableshow parallel 8 workers, the other session creating one index in another separate table always in serial. Even increasemax_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the other isdone in serial. James -----Original Message----- From: Tom Lane <tgl@sss.pgh.pa.us> Sent: Tuesday, June 21, 2022 10:30 PM To: James Pang (chaolpan) <chaolpan@cisco.com> Cc: pgsql-admin@lists.postgresql.org Subject: Re: multiple sessions to create index in parallel support "James Pang (chaolpan)" <chaolpan@cisco.com> writes: > We have a large database that just migrated data from Oracle to Postgres(13.4), after data load. We try to create indexes, with only one connection , the create index automatically run in parallel (table size 40G) , then we started twoconnections and each session creating index on large table, but at this time, only 1 session parallel creating index,the other one creating index in serial. There are enough parallel_workers and memory. Is that expected ? only 1 sessionallowed to parallel index creating at the same time ? Check your settings for max_parallel_workers and max_parallel_maintenance_workers. regards, tom lane
"James Pang (chaolpan)" <chaolpan@cisco.com> writes: > Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a largetable show parallel 8 workers, the other session creating one index in another separate table always in serial. Evenincrease max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the otheris done in serial. Hmm, what about max_worker_processes ? That's an independent limit. regards, tom lane
All max_workers_process got increased to 32 before creating index. The 2 session try to create index on totally separatelarge tables(40gb) , either one in 8 parallel workers, but not both. James -----Original Message----- From: Tom Lane <tgl@sss.pgh.pa.us> Sent: Tuesday, June 21, 2022 11:04 PM To: James Pang (chaolpan) <chaolpan@cisco.com> Cc: pgsql-admin@lists.postgresql.org Subject: Re: multiple sessions to create index in parallel support "James Pang (chaolpan)" <chaolpan@cisco.com> writes: > Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a largetable show parallel 8 workers, the other session creating one index in another separate table always in serial. Evenincrease max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the otheris done in serial. Hmm, what about max_worker_processes ? That's an independent limit. regards, tom lane
Sorry for confusion. This time, I add "max_parallel_workers"=32 , "max_parallel_maintenance_workers"=32 , "max_workers_process"=64to postgresql.conf, and restart database. This time, both session runs in parallel now. Last severaltests only got these parameter changes by set max_parallell_workers... in session level , for each session , only1 session parallel creating index. James -----Original Message----- From: James Pang (chaolpan) Sent: Wednesday, June 22, 2022 9:34 AM To: Tom Lane <tgl@sss.pgh.pa.us> Cc: pgsql-admin@lists.postgresql.org Subject: RE: multiple sessions to create index in parallel support All max_workers_process got increased to 32 before creating index. The 2 session try to create index on totally separatelarge tables(40gb) , either one in 8 parallel workers, but not both. James -----Original Message----- From: Tom Lane <tgl@sss.pgh.pa.us> Sent: Tuesday, June 21, 2022 11:04 PM To: James Pang (chaolpan) <chaolpan@cisco.com> Cc: pgsql-admin@lists.postgresql.org Subject: Re: multiple sessions to create index in parallel support "James Pang (chaolpan)" <chaolpan@cisco.com> writes: > Max_parallel_workers and max_parallel_maintenance_workers both equal to 32, one session create one index in a largetable show parallel 8 workers, the other session creating one index in another separate table always in serial. Evenincrease max_parallel_worers and max_parallel_maintenance_workers to 64, still show one is parallel 8 workers, the otheris done in serial. Hmm, what about max_worker_processes ? That's an independent limit. regards, tom lane