Re: libpq: PQgetCopyData() and allocation overhead - Mailing list pgsql-hackers

From Jeroen Vermeulen
Subject Re: libpq: PQgetCopyData() and allocation overhead
Date
Msg-id CA+zULE5BoZtkYMf9yPGH+BnPEPspXENVCZd_b4s=ayewU6DHZg@mail.gmail.com
Whole thread Raw
In response to Re: libpq: PQgetCopyData() and allocation overhead  (Jeroen Vermeulen <jtvjtv@gmail.com>)
Responses Re: libpq: PQgetCopyData() and allocation overhead
List pgsql-hackers
Update: in the latest iteration, I have an alternative that reduces CPU time by more than half, compared to PQgetCopyData().

And the code is simpler, too, both in the client and in libpq itself.  The one downside is that it breaks with libpq's existing API style.



Jeroen

On Mon, 27 Feb 2023 at 17:08, Jeroen Vermeulen <jtvjtv@gmail.com> wrote:
Done.  Thanks for looking!

Jelte Fennema pointed out that I should probably be using PQExpBuffer for this.  I'll look into that later; this is a proof of concept, not a production-ready API proposal.


Jeroen

On Mon, 27 Feb 2023 at 14:48, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:
On Fri, Feb 10, 2023 at 5:49 PM Jeroen Vermeulen <jtvjtv@gmail.com> wrote:
>
> OK, I've updated the PR with a benchmark (in the main directory).
>
> On this benchmark I'm seeing about a 24% reduction in "user" CPU time, and a 8% reduction in "system" CPU time.  (Almost no reduction in wall-clock time.)

I can help run some logical replication performance benchmarks
tomorrow. Would you mind cleaning the PR and providing the changes
(there are multiple commits in the PR) as a single patch here for the
sake of ease of review and test?

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

pgsql-hackers by date:

Previous
From: gkokolatos@pm.me
Date:
Subject: Re: Add LZ4 compression in pg_dump
Next
From: Melih Mutlu
Date:
Subject: Re: Allow logical replication to copy tables in binary format