Re: Improvements in Copy From - Mailing list pgsql-hackers

From Surafel Temesgen
Subject Re: Improvements in Copy From
Date
Msg-id CALAY4q_0msaVmW_1k5UOFO4Q4mLfueohE-7m51T2i1xWkVksjg@mail.gmail.com
Whole thread Raw
In response to Re: Improvements in Copy From  (vignesh C <vignesh21@gmail.com>)
Responses Re: Improvements in Copy From  (Kyotaro Horiguchi <horikyota.ntt@gmail.com>)
List pgsql-hackers


On Thu, Sep 10, 2020 at 1:17 PM vignesh C <vignesh21@gmail.com> wrote:

>
> We have a patch for column matching feature [1] that may need a header line to be further processed. Even without that I think it is preferable to process the header line for nothing than adding those checks to the loop, performance-wise.

I had seen that patch, I feel that change to match the header if the
header is specified can be addressed in this patch if that patch gets
committed first or vice versa. We are doing a lot of processing for
the data which we need not do anything. Shouldn't this be skipped if
not required. Similar check is present in NextCopyFromRawFields also
to skip header.

The existing check is unavoidable but we can live better without the checks added by the patch. For very large files the loop may iterate millions of times if it is not in billion and I am sure doing the check that many times will incur noticeable performance degradation than further processing a single line.

regards

Surafel 

pgsql-hackers by date:

Previous
From: "Jonathan S. Katz"
Date:
Subject: Re: PG 13 release notes, first draft
Next
From: Yaroslav
Date:
Subject: Probable documentation errors or improvements