Re: An idea for parallelizing COPY within one backend - Mailing list pgsql-hackers

From Tom Lane
Subject Re: An idea for parallelizing COPY within one backend
Date
Msg-id 29436.1204130462@sss.pgh.pa.us
Whole thread Raw
In response to Re: An idea for parallelizing COPY within one backend  (Alvaro Herrera <alvherre@commandprompt.com>)
List pgsql-hackers
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Yeah, but it wouldn't take advantage of, say, the hack to disable WAL
> when the table was created in the same transaction.

In the context of a parallelizing pg_restore this would be fairly easy
to get around.  We could probably teach the thing to combine table
creation and loading steps into one action (transaction) in most cases.
If that couldn't work because of some weird dependency chain, the
data loading transaction could be done as
BEGIN;TRUNCATE table;COPY table FROM stdin;...COMMIT;

which I believe already invokes the no-WAL optimization, and could
certainly be made to do so if not.

Obviously, pg_restore would have to be aware of whether or not it had
created that table in the current run (else it mustn't TRUNCATE),
but it would be tracking more or less exactly that info anyway to handle
dependency ordering.
        regards, tom lane


pgsql-hackers by date:

Previous
From: "Hiroshi Saito"
Date:
Subject: Re: OSSP can be used in the windows environment now!
Next
From: Andrew Dunstan
Date:
Subject: Re: An idea for parallelizing COPY within one backend