Re: [HACKERS] Inefficiencies in COPY command - Mailing list pgsql-hackers

From Tom Lane
Subject Re: [HACKERS] Inefficiencies in COPY command
Date
Msg-id 11798.934038075@sss.pgh.pa.us
Whole thread Raw
In response to Inefficiencies in COPY command  (Wayne Piekarski <wayne@senet.com.au>)
Responses Re: [HACKERS] Inefficiencies in COPY command
Re: [HACKERS] Inefficiencies in COPY command
List pgsql-hackers
Wayne Piekarski <wayne@senet.com.au> writes:
> So I made some changes to CopyAttributeOut so that it escapes the string
> initially into a temporary buffer (allocated onto the stack) and then
> feeds the whole string to the CopySendData which is a lot more efficient
> because it can blast the whole string in one go, saving about 1/3 to 1/4
> the number of memcpy and so on.

copy.c is pretty much of a hack job to start with, IMHO.  If you can
speed it up without making it even uglier, have at it!  However, it
also has to be portable, and what you've done here:

>  CopyAttributeOut(FILE *fp, char *server_string, char *delim, int is_array)
>  {
>      char       *string;
>      char        c;
> +    char __buffer [(strlen (server_string) * 2) + 4]; /* Use +4 for safety */

is not portable --- variable-sized local arrays are a gcc-ism.  (I use
'em a lot too, but not in code intended for public release...)  Also,
be careful that you don't introduce any assumptions about maximum
field or tuple width; we want to get rid of those, not add more.

> to also make changes to remove all use of sprintf when numbers
> and floats are being converted into strings. ^^^^^^^^^^

While formatting an int is pretty simple, formatting a float is not so
simple.  I'd be leery of replacing sprintf with quick-hack float
conversion code.  OTOH, if someone wanted to go to the trouble of doing
it *right*, using our own code would tend to yield more consistent
results across different OSes, which would be a Good Thing.  I'm not
sure it'd be any faster than the typical sprintf, but it might be worth
doing anyway.

(My idea of *right* is the guaranteed-inverse float<=>ASCII routines
published a few years ago in some SIGPLAN proceedings ... I've got the
proceedings, and I even know approximately where they are, but I don't
feel like excavating for them right now...)
        regards, tom lane


pgsql-hackers by date:

Previous
From: Wayne Piekarski
Date:
Subject: Re: [HACKERS] SIGSEGV on CREATE FUNCTION with plpgsql
Next
From: Don Baccus
Date:
Subject: Re: [HACKERS] 6.5.1, error in pg_dump