Re: Allow logical replication to copy tables in binary format - Mailing list pgsql-hackers

From Melih Mutlu
Subject Re: Allow logical replication to copy tables in binary format
Date
Msg-id CAGPVpCR15YfE=4hthY6S2OC5G0nj0QM4oE2bV769+FANM_UvwQ@mail.gmail.com
Whole thread Raw
In response to Re: Allow logical replication to copy tables in binary format  ("Euler Taveira" <euler@eulerto.com>)
Responses Re: Allow logical replication to copy tables in binary format
List pgsql-hackers

Euler Taveira <euler@eulerto.com>, 11 Ağu 2022 Per, 16:27 tarihinde şunu yazdı:
On Thu, Aug 11, 2022, at 8:04 AM, Amit Kapila wrote:
On Thu, Aug 11, 2022 at 7:34 AM Euler Taveira <euler@eulerto.com> wrote:
>
> The reason to use text format is that it is error prone. There are restrictions
> while using the binary format. For example, if your schema has different data
> types for a certain column, the copy will fail.
>

Won't such restrictions hold true even during replication?
I expect that the COPY code matches the proto.c code. The point is that table
sync is decoupled from the logical replication. Hence, we should emphasize in
the documentation that the restrictions *also* apply to the initial table
synchronization.

If such restrictions are already the case for replication phase after initial table sync, then it shouldn't prevent us from enabling binary option for table sync. Right?
But yes, it needs to be stated somewhere.

Euler Taveira <euler@eulerto.com>, 11 Ağu 2022 Per, 05:03 tarihinde şunu yazdı
I have a few points about your implementation.

* Are we considering to support prior Postgres versions too? These releases
  support binary mode but it could be an unexpected behavior (initial sync in
  binary mode) for a publisher using 14 or 15 and a subscriber using 16. IMO
  you should only allow it for publisher on 16 or later.

How is any issue that might occur due to version mismatch being handled right now in repliaction after table sync?
What I understand from the documentation is if replication can fail due to using different pg versions, it just fails. So binary option cannot be used. [1]
Do you think that this is more serious for table sync and we need to restrict binary option with different publisher and subscriber versions? But not for replication?

* Docs should say that the binary option also applies to initial table
  synchronization and possibly emphasize some of the restrictions.
* Tests. Are the current tests enough? 014_binary.pl.

You're right on both points. I just wanted to know your opinions on this first. Then the patch will need some tests and proper documentation.  


Best,
Melih
  

pgsql-hackers by date:

Previous
From: "Euler Taveira"
Date:
Subject: Re: Allow logical replication to copy tables in binary format
Next
From: Andrew Dunstan
Date:
Subject: Re: tests and meson - test names and file locations