Re: A little COPY speedup - Mailing list pgsql-patches

From Andrew Dunstan
Subject Re: A little COPY speedup
Date
Msg-id 45E70D08.2030200@dunslane.net
Whole thread Raw
In response to A little COPY speedup  (Heikki Linnakangas <heikki@enterprisedb.com>)
List pgsql-patches
Heikki Linnakangas wrote:
> One complaint we've heard from clients trying out EDB or PostgreSQL is
> that loading data is slower than on other DBMSs.
>
> I ran oprofile on a COPY FROM to get an overview of where the CPU time
> is spent. To my amazement, the function at the top of the list was
> PageAddItem with 16% of samples.
>
> On every row, PageAddItem will scan all the line pointers on the
> target page, just to see that they're all in use, and create a new
> line pointer. That adds up, especially with narrow tuples like what I
> used in the test.
>
> Attached is a fix for that. It adds a flag to each heap page that
> indicates that "there isn't any free line pointers on this page, so
> don't bother trying". Heap pages haven't had any heap-specific
> per-page data before, so this patch adds a HeapPageOpaqueData-struct
> that's stored in the special space.
>
> My simple test case of a COPY FROM of 10000000 tuples took 19.6 s
> without the patch, and 17.7 s with the patch applied. Your mileage may
> vary.

What is the speedup with less narrow tuples? 10% improvement is good but
not stellar.

cheers

andrew

pgsql-patches by date:

Previous
From: "Pavan Deolasee"
Date:
Subject: Re: A little COPY speedup
Next
From: Tom Lane
Date:
Subject: Re: A little COPY speedup