Re: CDC feature request - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: CDC feature request
Date
Msg-id CABUevExjjSe3maOx-0bwF=WEATb-+ibQdRcbw1mX=1SGppB_pw@mail.gmail.com
Whole thread Raw
In response to CDC feature request  (Stepan Yankevych <Stepan_Yankevych@epam.com>)
Responses Re: CDC feature request  (Ronan Dunklau <ronan@dunklau.fr>)
List pgsql-hackers


On Thu, Mar 18, 2021 at 2:03 PM Stepan Yankevych <Stepan_Yankevych@epam.com> wrote:

Hi All!

 

Hopefully I’m using correct mail list  

If not please show me right direction 😊

 

I’m quite struggling without native Change Data Capture feature in PostgreSQL.

 

That would be great to implement it, possibly in not so complicated way.

 

Can Logical replication be a little be modified or reused to do not replicate data into destination table as is but to insert each change into “change table” (like in oracle 11 CDC)?

That change table should have at lease few additional columns

  • Operation (I/D/U)
  • txid
  • Commit_time_stamp

If you look at logical decoding, that's basically what you have, isn't it? It won't go into a table, but you can consume it into one if you want. Look at for example wal2json for examples of how to consume it -- but the system is pluggable so you can build your own or use one of the others available plugins.
 
--

pgsql-hackers by date:

Previous
From: Amit Langote
Date:
Subject: Re: A reloption for partitioned tables - parallel_workers
Next
From: Fujii Masao
Date:
Subject: Re: fdatasync performance problem with large number of DB files