Logical replication initialization time depends dramatically on the publication "schema" size - Mailing list pgsql-bugs

From Crimean (gmail)
Subject Logical replication initialization time depends dramatically on the publication "schema" size
Date
Msg-id CACq0ybaVS8v2ouZfSJuSHX9bvgt=sJkwj=OYTzuv9UVxJN95fw@mail.gmail.com
Whole thread
Responses RE: Logical replication initialization time depends dramatically on the publication "schema" size
List pgsql-bugs

Hello,

I would like to report a scalability issue I observed during logical replication initialization.

The problem is not that replication fails or behaves incorrectly. However, initialization time appears to increase significantly as the number of tables in a publication grows.

While investigating, I noticed the following query being executed:

SELECT DISTINCT       (CASE WHEN (array_length(gpt.attrs, 1) = c.relnatts)             THEN NULL             ELSE gpt.attrs        END)
FROM pg_publication p,     LATERAL pg_get_publication_tables(p.pubname) gpt,     pg_class c
WHERE gpt.relid = 236073  AND c.oid = gpt.relid  AND p.pubname IN ('p1');

According to AWS Performance Insights, this query shows an average latency of more than 130 ms and approximately 113,799 buffer hits per execution when the publication contains around 10,000 tables.

As a result, the subscription initialization rate degrades from roughly 28 tables/second with 500 tables in the publication to about 8 tables/second with 10,000 tables.

In my environment, replication itself works correctly, but initialization time increases substantially as publication size grows. Unfortunately, some of my deployments contain very large schemas, with over 1 million tables on the publisher side, making this behavior a practical problem.

I can provide PostgreSQL version information, execution plans, and reproduction details if needed.

Thank you for your time.

Best regards,
Sergii

pgsql-bugs by date:

Previous
From: Andrey Rachitskiy
Date:
Subject: Re: BUG #19458: OOM killer in jsonb_path_exists_opr (@?) with malformed JSONPath containing non-existent variables
Next
From: Amjad Shahzad
Date:
Subject: Re: BUG #19511: contrib/dblink: NULL dereference in dblink_get_notify() when called without a prior connection