Re: Making "COPY partitioned_table FROM" faster - Mailing list pgsql-hackers

From Peter Eisentraut
Subject Re: Making "COPY partitioned_table FROM" faster
Date
Msg-id 2b40468d-6be0-e795-c363-0015bc9fa747@2ndquadrant.com
Whole thread Raw
In response to Re: Making "COPY partitioned_table FROM" faster  (David Rowley <david.rowley@2ndquadrant.com>)
Responses Re: Making "COPY partitioned_table FROM" faster  (David Rowley <david.rowley@2ndquadrant.com>)
List pgsql-hackers
On 20.07.18 16:57, David Rowley wrote:
> One final note:  I'm not entirely convinced we need this adaptive
> code, but it seems easy enough to rip it back out if it's more trouble
> than it's worth. But if the other option is a GUC, then I'd rather
> stick with the adaptive code, it's likely going to do much better than
> a GUC since it can change itself during the copy which will be useful
> when the stream contains a mix small and large sets of consecutive
> tuples which belong to the same partition.

I think some kind of way to switch between the two code paths would be
desirable.  For example, with hash partitioning, it's likely that in
many cases you won't find any adjacent candidates in batches of
significant size.  So then you've just made everything 5% slower.
Unless we can make the multi-insert path itself faster.

The particular heuristic you have chosen seems sensible to me, but we'll
have to see how it holds up in practice.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: [HACKERS] logical decoding of two-phase transactions
Next
From: Andres Freund
Date:
Subject: Re: cached plans and enable_partition_pruning