Logical Replication - One table 8 minutes, Identical Two Tables 28 Minutes (Expected 16 Minutes) - Mailing list pgsql-general

From Avi Weinberg
Subject Logical Replication - One table 8 minutes, Identical Two Tables 28 Minutes (Expected 16 Minutes)
Date
Msg-id DB9PR07MB7180E7A4CD39D54A7DE78673CB429@DB9PR07MB7180.eurprd07.prod.outlook.com
Whole thread Raw
List pgsql-general

Hi All,

 

When I have only one table (2.5GB) in logical replication publication it takes 8 minutes to sync to the subscriber when I created a copy of the table and added it to the publication it took 28 minutes.  When I sync three IDENTICAL tables of the same size it take 46 minutes!

I expected the tables to be synched in parallel or at least one after the other, but it turned out the time increase is much larger than just to sync additional table of the same size.  Two tables should be 8*2=16 minutes and not 28 and three tables should be 8*3=24 minutes and not 46.

 

The table has one pk int column and one binary column file_content of type bytea.

 

What might be the reason for this behavior? Can I improve the time it takes to sync several tables so it will be no more than the (time to sync one table) * (number of tables)

 

Doing it for two tables took 28 minutes

2021-12-27 10:32:13.740 UTC [4430] LOG:  logical replication apply worker for subscription "sub_gwn1" has started

2021-12-27 10:32:13.815 UTC [4431] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p1" has started

2021-12-27 10:32:13.827 UTC [4432] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p" has started

2021-12-27 10:57:57.542 UTC [4432] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p" has finished

2021-12-27 11:00:03.806 UTC [4431] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p1" has finished

 

Doing it for three tables took 46 minutes

2021-12-27 11:49:11.931 UTC [4739] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p2" has started

2021-12-27 11:49:11.948 UTC [4740] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p1" has started

2021-12-27 12:23:44.673 UTC [4740] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p1" has finished

2021-12-27 12:23:44.783 UTC [4845] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p" has started

2021-12-27 12:28:53.320 UTC [4739] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p2" has finished

2021-12-27 12:35:49.156 UTC [4845] LOG:  logical replication table synchronization worker for subscription "sub_gwn1", table "p" has finished

 

Thanks!

 

IMPORTANT - This email and any attachments is intended for the above named addressee(s), and may contain information which is confidential or privileged. If you are not the intended recipient, please inform the sender immediately and delete this email: you should not copy or use this e-mail for any purpose nor disclose its contents to any person.

pgsql-general by date:

Previous
From: Pavel Stehule
Date:
Subject: Re: Deprecating plans for PGPASSWORD environment variable as insecure
Next
From: Narendra katlamudi
Date:
Subject: psql command failing with error "undefined symbol: PQhostaddr"