Re: Binary support for pgoutput plugin - Mailing list pgsql-hackers

From Dave Cramer
Subject Re: Binary support for pgoutput plugin
Date
Msg-id CADK3HHJ7_dfXzOLYLpdw8uNLcAWi0bzFCf3A0yiC5wShqAWphw@mail.gmail.com
Whole thread Raw
In response to Re: Binary support for pgoutput plugin  (David Fetter <david@fetter.org>)
Responses Re: Binary support for pgoutput plugin
List pgsql-hackers

Dave Cramer


On Mon, 3 Jun 2019 at 20:54, David Fetter <david@fetter.org> wrote:
On Mon, Jun 03, 2019 at 10:49:54AM -0400, Dave Cramer wrote:
> Is there a reason why pgoutput sends data in text format? Seems to
> me that sending data in binary would provide a considerable
> performance improvement.

Are you seeing something that suggests that the text output is taking
a lot of time or other resources?

Actually it's on the other end that there is improvement. Parsing text takes much longer for almost everything except ironically text.

To be more transparent there is some desire to use pgoutput for something other than logical replication. Change Data Capture clients such as Debezium have a requirement for a stable plugin which is shipped with core as this is always available in cloud providers offerings. There's no reason that I am aware of that they cannot use pgoutput for this. There's also no reason that I am aware that binary outputs can't be supported. The protocol would have to change slightly and I am working on a POC patch.

Thing is they aren't all written in C so using binary does provide a pretty substantial win on the decoding end.

Dave

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Avoiding hash join batch explosions with extreme skew and weird stats
Next
From: Alvaro Herrera
Date:
Subject: Re: coverage additions