Many logical replication synchronization worker threads kicking off for ONE table - Mailing list pgsql-general

From Dolan, Sean
Subject Many logical replication synchronization worker threads kicking off for ONE table
Date
Msg-id cd6c557caa5c42af95ec1f9cf914cc8a@lmco.com
Whole thread Raw
In response to Re: EXTERNAL: Re: "No Free extents", table using all allocated space but no rows!  (Laurenz Albe <laurenz.albe@cybertec.at>)
List pgsql-general
The initial email below was the end result of something "run-away" in my logical replication.

PostGres 13.6, RedHat 7.9

Database A is the publisher; Database B is the subscriber.    Within Database A are multiple schemas and the
publicationensures that all the schemas and its tables are added.    There is one table in particular that has
1.4millionrows of data.   
 

I create a subscription on Database B and can see in the log:
    LOG: logical replication table synchronization for subscription "sub_to_dbaseA", table "alert_history" has
started.
    CONTEXT:  COPY alert_history line 6668456
    LOG: logical replication table synchronization for subscription "sub_to_dbaseA", table "alert_history" has
started.
    CONTEXT:  COPY alert_history line 5174606
    LOG: logical replication table synchronization for subscription "sub_to_dbaseA", table "alert_history" has
started.
    CONTEXT:  COPY alert_history line 4325283

Normally I would see a line for "finished", but I never do.

I then actively watch the schema/table and do \dt+   and can see that table grow in 2GB increments until I fill up the
entiredrive and run out of room for that schema.
 

I am NOT getting any "checkpoints are occurring too frequently"    As I have updated my WAL size:
    max_wal_size=4GB
    min_wal_size=1GB

Is the system having trouble synching this amount of data in a quick fashion and therefore kicks off more
synchronizationthreads?   Anything I can do to prevent this?
 

Thank you



-----Original Message-----
From: Laurenz Albe <laurenz.albe@cybertec.at> 
Sent: Thursday, March 16, 2023 1:25 PM
To: Dolan, Sean (US N-ISYS Technologies Inc.) <sean.dolan@lmco.com>; pgsql-general@lists.postgresql.org
Subject: EXTERNAL: Re: EXTERNAL: Re: "No Free extents", table using all allocated space but no rows!

On Thu, 2023-03-16 at 13:20 +0000, Dolan, Sean wrote:
> I messed up and confused issues.   The error is :  
> ERROR: Could not extend pg_tblspc/16555/PG_13_20200/xxxx/xxxx  No space left on device
> HINT: Check free disk space
> 
> So the schema is "full" and the offender is this one table.
> I can't TRUNCATE as there needs to be space to perform the action.
> Is there a way to see if there is a transaction on that table like you allude to?

Ah, that's different.

If you don't have enough space to run TRUNCATE, and you don't feel like
extending the disk space, DROP TABLE would be a convenient alternative.

Yours,
Laurenz Albe

pgsql-general by date:

Previous
From: Laurenz Albe
Date:
Subject: Re: EXTERNAL: Re: "No Free extents", table using all allocated space but no rows!
Next
From: jian he
Date:
Subject: src/test/examples/testlibpq2.c where the HAVE_SYS_SELECT_H is defined.