Re: Make COPY format extendable: Extract COPY TO format implementations - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: Make COPY format extendable: Extract COPY TO format implementations
Date
Msg-id 10025bac-158c-ffe7-fbec-32b42629121f@dunslane.net
Whole thread Raw
In response to Re: Make COPY format extendable: Extract COPY TO format implementations  (Michael Paquier <michael@paquier.xyz>)
Responses Re: Make COPY format extendable: Extract COPY TO format implementations
List pgsql-hackers
On 2024-01-24 We 03:11, Michael Paquier wrote:
> On Wed, Jan 24, 2024 at 02:49:36PM +0900, Sutou Kouhei wrote:
>> For COPY TO:
>>
>> 0001: This adds CopyToRoutine and use it for text/csv/binary
>> formats. No implementation change. This just move codes.
> 10M without this change:
>
>      format,elapsed time (ms)
>      text,1090.763
>      csv,1136.103
>      binary,1137.141
>
> 10M with this change:
>
>      format,elapsed time (ms)
>      text,1082.654
>      csv,1196.991
>      binary,1069.697
>
> These numbers point out that binary is faster by 6%, csv is slower by
> 5%, while text stays around what looks like noise range.  That's not
> negligible.  Are these numbers reproducible?  If they are, that could
> be a problem for anybody doing bulk-loading of large data sets.  I am
> not sure to understand where the improvement for binary comes from by
> reading the patch, but perhaps perf would tell more for each format?
> The loss with csv could be blamed on the extra manipulations of the
> function pointers, likely.


I don't think that's at all acceptable.

We've spent quite a lot of blood sweat and tears over the years to make 
COPY fast, and we should not sacrifice any of that lightly.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com




pgsql-hackers by date:

Previous
From: Matthias van de Meent
Date:
Subject: Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)
Next
From: Fujii Masao
Date:
Subject: Remove redundant HandleWalWriterInterrupts()