Re: Logical archiving - Mailing list pgsql-hackers

From Euler Taveira
Subject Re: Logical archiving
Date
Msg-id CAH503wCiFhtqYg5f8Ogw-j4fEs31EFXt1GH4N5Fs0ZNM+5ATHw@mail.gmail.com
Whole thread Raw
In response to Re: Logical archiving  (Andrey Borodin <x4mmm@yandex-team.ru>)
List pgsql-hackers
On Fri, 4 Dec 2020 at 14:36, Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>
> The most time consuming process is logical decoding, mainly due to long running transactions.
Currently I do not experience problem of high CPU utilisation.

I'm wondering why the LSN isn't moving fast enough for your use case.
 
> In order to minimize your issue, we should improve the logical decoding mechanism.
No, the issue I'm facing comes from the fact that corner cases of failover are not solved properly for logical replication. Timelines, partial segments, archiving along with streaming, starting from arbitrary LSN (within available WAL), rewind, named restore points, cascade replication etc etc. All these nice things are there for WAL and are missing for LR. I'm just trying to find shortest path through this to make CDC(changed data capture) work.

Craig started a thread a few days ago [1] that described some of these issues and possible solutions [2]. The lack of HA with logical replication reduces the number of solutions that could possibly use this technology. Some of the facilities such as logical replication slots and replication origin on failover-candidate subscribers should encourage users to adopt such solutions. 



--
Euler Taveira                 http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: WIP: WAL prefetch (another approach)
Next
From: Alvaro Herrera
Date:
Subject: Re: POC: Better infrastructure for automated testing of concurrency issues