Parallelizing startup with many databases - Mailing list pgsql-hackers

From Babak Ghadiri
Subject Parallelizing startup with many databases
Date
Msg-id CAEkHFKSxYO7Ryv_UNNGSe88zosg0_Q_dop6hXN0oCoYK28EZtA@mail.gmail.com
Whole thread Raw
List pgsql-hackers
Hello,
I hope you are doing well.

In PostgreSQL 16, startup appears to initialize databases sequentially and
primarily uses a single CPU core. In clusters with a very large number of
databases (around 5,000 in our case), this results in noticeably long
startup times after restarts or crash recovery.

I would like to ask:

- Is the largely single-threaded startup behavior a fundamental architectural
  constraint (e.g. catalog dependencies, locking, recovery ordering), or mainly
  an unimplemented optimization?
- Are there any existing discussions, patches, versions (18+) to parallelize parts of startup or otherwise improve startup scalability with many databases?
- Are there any PostgreSQL configuration settings known to dramatically reduce startup time, or is startup performance mostly fixed by architecture in this scenario?

I understand that having many databases in a single cluster is not the most
common or recommended multi-tenant model, but this is an existing system and
I’m trying to better understand the current limits and future direction.

Thank you for your time and insights.

Best regards.

pgsql-hackers by date:

Previous
From: Kirill Reshke
Date:
Subject: Re: Create unique GiST indexes
Next
From: tushar
Date:
Subject: Re: Non-text mode for pg_dumpall