Re: To make pg_dump and pg_restore parallel in processing limited number of LOs - Mailing list pgsql-hackers

From Tom Lane
Subject Re: To make pg_dump and pg_restore parallel in processing limited number of LOs
Date
Msg-id 2260523.1747579562@sss.pgh.pa.us
Whole thread Raw
In response to To make pg_dump and pg_restore parallel in processing limited number of LOs  (fkfk000 <fkfk000@126.com>)
List pgsql-hackers
fkfk000 <fkfk000@126.com> writes:
> However, if a user only has a limited number of LOs, like 1k, which seems sensible as LOs should be large. In this
scenario,there would be only 1 process work. Therefore, I'm proposing a change. Instead of using a fixed number to
groupLOs with same owner/ACL pair, we can use a SQL query to distribute each pair into a fixed number of batches. For
eachbatch, it would be assigned an ArchiveEntry. So, the workload for each pair could be distributed into processes
evenif there are only few numbers of LO.  

I do not care for this idea.  I think this behavior will seem
entirely random to most users.  Also, you appear not to be thinking
at all about what will happen with huge numbers (millions) of blobs.
Forcing them all into a relatively small number of TOC entries will
break exactly the same cases that we intended to fix by breaking them
up into multiple TOC entries.

I'd rather do what's speculated in the existing comment:

 * At some point we might want to make this user-controllable, but for now
 * a hard-wired setting will suffice.

That would in particular allow people to split things up as finely
as one blob per TOC entry, which would be useful for selective-restore
purposes.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Mihail Nikalayeu
Date:
Subject: Re: [BUG?] check_exclusion_or_unique_constraint false negative
Next
From: Sami Imseih
Date:
Subject: Re: Possible regression in PG18 beta1