Re: A little COPY speedup - Mailing list pgsql-patches

From Simon Riggs
Subject Re: A little COPY speedup
Date
Msg-id 1172854085.3760.1557.camel@silverbirch.site
Whole thread Raw
In response to Re: A little COPY speedup  (Gregory Stark <stark@enterprisedb.com>)
Responses Re: A little COPY speedup  (Andrew Dunstan <andrew@dunslane.net>)
List pgsql-patches
On Fri, 2007-03-02 at 16:25 +0000, Gregory Stark wrote:
> "Tom Lane" <tgl@sss.pgh.pa.us> writes:
>
> > "Simon Riggs" <simon@2ndquadrant.com> writes:
> >> I'm slightly worried though since that seems to have changed from 8.2,
> >> which I oprofiled over Christmas.
> >
> > If you were testing a case with wider rows than Heikki tested, you'd see
> > less impact --- the cost of the old way was O(N^2) in the number of
> > tuples that fit on a page, so the behavior gets rapidly worse as you get
> > down to smaller tuple sizes.  (Come to think of it, the cmin/cmax
> > collapse would be a factor here too.)
>
> Or larger block sizes of course. A 32kb block would be 16x as bad which starts
> to be pretty serious.

Well, I was only using 8kb blocks.

But I think the message is clear: we need to profile lots of different
combinations. I was using a 2 col table with integer, char(100).

IIRC there are issues with delimiter handling when we have lots of
columns in the input on COPY FROM, and num of cols on COPY TO. I've not
looked at those recently though.

--
  Simon Riggs
  EnterpriseDB   http://www.enterprisedb.com



pgsql-patches by date:

Previous
From: Gregory Stark
Date:
Subject: Re: A little COPY speedup
Next
From: Andrew Dunstan
Date:
Subject: Re: A little COPY speedup