Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling - Mailing list pgsql-hackers

From Alex K
Subject Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling
Date
Msg-id CADfU8WygFBs5Vv8PhheP0sOfROLaVU1=0PXHmUUO1C_w1iMNdQ@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling  (Alexey Kondratov <kondratov.aleksey@gmail.com>)
Responses Re: [HACKERS] GSOC'17 project introduction: Parallel COPY executionwith errors handling
List pgsql-hackers
> On 16 Jun 2017, at 21:30, Alexey Kondratov <kondratov.aleksey@gmail.com> wrote:

> > On 13 Jun 2017, at 01:44, Peter Geoghegan <pg@bowt.ie> wrote:


> > Speculative insertion has the following special entry points to
> > heapam.c and execIndexing.c, currently only called within
> > nodeModifyTable.c

> > Offhand, it doesn't seem like it would be that hard to teach another
> > heap_insert() caller the same tricks.


> I went through the nodeModifyTable.c code and it seems not to be so
> difficult to do the same inside COPY.

After a more precise look, I have figured out at least one difficulty, COPY
and INSERT follow the different execution paths: INSERT goes through
the Planner, while COPY does not. It leads to the absence of some required
attributes like arbiterIndexes, which are available during INSERT via
PlanState/ModifyTableState. Probably it is possible to get the same in the
COPY, but it is not clear for me how.

Anyway, adding of the 'speculative insertion' into the COPY is worth of a
separated patch; and I would be glad to try implementing it.

In the same time I have prepared a complete working patch with:

- ignoring of the input data formatting errors
- IGNORE_ERRORS parameter in the COPY options
- updated regression tests

Please, find the patch attached or check the web UI diff on GitHub as always:
https://github.com/ololobus/postgres/pull/1/files


Alexey

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachment

pgsql-hackers by date:

Previous
From: Andrew Dunstan
Date:
Subject: Re: [HACKERS] pg_bsd_indent 2.0 is available from git.postgresql.org
Next
From: Robert Haas
Date:
Subject: Re: [HACKERS] Default Partition for Range