Recommendations on Improving the debezium performance even on medium workload - Mailing list pgsql-general

From Durgamahesh Manne
Subject Recommendations on Improving the debezium performance even on medium workload
Date
Msg-id CAJCZkoKbfzFSfTemPiZN-mBrgy7ob=9gFq-5BnNEbHsH3Fe-dA@mail.gmail.com
Whole thread Raw
List pgsql-general
Hi Team 

How do we improve the debezium performance?
Recommendations at kafka configuration side
Agenda is to minimize the lag during moderate or high work load on db 

default > poll.interval.ms = 500ms 
Recommended value for balanced performance > 1000ms
Recommended value for high throughput > 1000ms to 5000ms

 

 

default > max.batch.size = 2048 allocated > 2560
Recommended value for balanced performance > 500 to 1000
Recommended value for high throughput > 2000 or 5000

 

default > max.queue.size = 8192 allocated > 10000
Recommended value for balanced performance > 7500 (1000 to 10000)
Recommended value for high throughput > 10000

based on kafka load we can optimize debezium connector performance 

 

default > heartbeat.interval.ms = 0 allocated > 5000ms
Recommended value for this > 5000ms to 10000ms


Are these helpful  ?

Need auto connectivity even on connector(slot) failure 

please let me know your recommendations please 


Regards,

Durga Mahesh

pgsql-general by date:

Previous
From: Ron Johnson
Date:
Subject: Re: Better way to process records in bash?
Next
From: Durgamahesh Manne
Date:
Subject: pglogical selective child replication between different partition interval tables