Re: Pulling data from Postgres DB table for every 5 seconds. - Mailing list pgsql-general

From Mark Fletcher
Subject Re: Pulling data from Postgres DB table for every 5 seconds.
Date
Msg-id CAEO096q6VdnZtKO2-4MkSYe29xguh+UiSsMLoD2L7jSQ=nnWFg@mail.gmail.com
Whole thread Raw
In response to Re: Pulling data from Postgres DB table for every 5 seconds.  (github kran <githubkran@gmail.com>)
Responses Re: Pulling data from Postgres DB table for every 5 seconds.  (Martín Marqués <martin@2ndquadrant.com>)
List pgsql-general
On Wed, Jan 9, 2019 at 12:58 PM github kran <githubkran@gmail.com> wrote:

Mark - just curious to know on the logical replication. Do you think I can use it for my use case where i need to publish data to a subscriber when there is a change in the data updated for a row or any new inserts happening on the table. Intention
is to send this data in Json format by collecting this modified data in real time to a subscriber.

From what you've said, it's a great use case for that feature. The one thing to note is that you will have to code up a logical replication client. If I can do it, pretty much anyone can, but it might take some time to get things right. I wrote about some of what I found when developing our client a year ago here: https://wingedpig.com/2017/09/20/streaming-postgres-changes/

We ended up just using the included test output plugin that comes with the postgresql distribution. And we didn't end up streaming to Kafka or anything else first. We just take the data and insert it into our elasticsearch cluster directly as we get it.

Mark

pgsql-general by date:

Previous
From: github kran
Date:
Subject: Re: Pulling data from Postgres DB table for every 5 seconds.
Next
From: Martín Marqués
Date:
Subject: Re: Pulling data from Postgres DB table for every 5 seconds.