Joshua D. Drake wrote:
> Heikki Linnakangas wrote:
>>
>> We read 64 KB at a time, and then CopyReadLineText returns one line
>> at a time from that buffer.
>
> O.k. I am sure I am oversimplifying things but why are we returning
> one line at a time? That reads expensive to me. Just following the
> general, don't do inserts one at a time, do them in batch idea for
> example.
Quite simply because one line corresponds to one record. And yes, I
believe you are oversimplifying, or under several misapprehensions about
what can be done at this level.
>
> I would also question the 64KB at a time. Why not a 1024KB (arbitrary)
> at a time? Is it a resource issue? In the old days when we actually
> had people trying to run postgresql on 128 and 256 megs of ram, o.k.
> but now?
It would be simple enough to change. Try it and see if it actually makes
a difference. All you have to change is the define of RAW_BUF_SIZE.
>
> In reading:
>
> http://doxygen.postgresql.org/backend_2commands_2copy_8c-source.html
>
> It looks like CopyReadAttributesText is used as part of the column
> breakup. It also appears that this happens "before" insert right? So
> if that is the case we are going to pay an additional penalty on the
> data checking.
>
What? I don't understand what you are talking about.
cheers
andrew
