Thread: pgsql: pg_restore -jN does not equate "multiple jobs", so partly revert

Log Message:
-----------
pg_restore -jN does not equate "multiple jobs", so partly revert the
previous patch.

Per note from Tom.

Modified Files:
--------------
    pgsql/src/bin/pg_dump:
        pg_restore.c (r1.98 -> r1.99)
        (http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/bin/pg_dump/pg_restore.c?r1=1.98&r2=1.99)

On Tue, 2009-04-14 at 00:06 +0000, Alvaro Herrera wrote:
> Log Message:
> -----------
> pg_restore -jN does not equate "multiple jobs", so partly revert the
> previous patch.

I notice that the release notes talk about "parallel restore", yet the
word parallel is not used in the docs anywhere. The section on restoring
a dump makes no mention of this new facility, nor does the perf tips
section.

Also says that the option doesn't work with --single-transaction, so we
should say that you've added in TRUNCATE to make the COPY go faster (or
haven't you? I know we discussed it).

--
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support

Simon Riggs wrote:
> Also says that the option doesn't work with --single-transaction, so we
> should say that you've added in TRUNCATE to make the COPY go faster (or
> haven't you? I know we discussed it).
>
>

Yes, it does use TRUNCATE.

cheers

andrew

Simon Riggs wrote:
>
> On Tue, 2009-04-14 at 00:06 +0000, Alvaro Herrera wrote:
> > Log Message:
> > -----------
> > pg_restore -jN does not equate "multiple jobs", so partly revert the
> > previous patch.
>
> I notice that the release notes talk about "parallel restore", yet the
> word parallel is not used in the docs anywhere. The section on restoring
> a dump makes no mention of this new facility, nor does the perf tips
> section.
>
> Also says that the option doesn't work with --single-transaction, so we
> should say that you've added in TRUNCATE to make the COPY go faster (or
> haven't you? I know we discussed it).

Good point.  Documentation patch attached and applied.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +
Index: doc/src/sgml/perform.sgml
===================================================================
RCS file: /cvsroot/pgsql/doc/src/sgml/perform.sgml,v
retrieving revision 1.73
diff -c -c -r1.73 perform.sgml
*** doc/src/sgml/perform.sgml    15 Jan 2010 09:18:59 -0000    1.73
--- doc/src/sgml/perform.sgml    26 Feb 2010 02:31:28 -0000
***************
*** 1027,1033 ****
         possibly discarding many hours of processing.  Depending on how
         interrelated the data is, that might seem preferable to manual cleanup,
         or not.  <command>COPY</> commands will run fastest if you use a single
!        transaction and have WAL archiving turned off.
        </para>
       </listitem>
       <listitem>
--- 1027,1036 ----
         possibly discarding many hours of processing.  Depending on how
         interrelated the data is, that might seem preferable to manual cleanup,
         or not.  <command>COPY</> commands will run fastest if you use a single
!        transaction and have WAL archiving turned off.
!        <application>pg_restore</> also has a <option>--jobs</> option
!        which allows concurrent data loading and index creation, and has
!        the performance advantages of doing COPY in a single transaction.
        </para>
       </listitem>
       <listitem>

On Thu, 2010-02-25 at 21:32 -0500, Bruce Momjian wrote:
> Simon Riggs wrote:
> >
> > On Tue, 2009-04-14 at 00:06 +0000, Alvaro Herrera wrote:
> > > Log Message:
> > > -----------
> > > pg_restore -jN does not equate "multiple jobs", so partly revert the
> > > previous patch.
> >
> > I notice that the release notes talk about "parallel restore", yet the
> > word parallel is not used in the docs anywhere. The section on restoring
> > a dump makes no mention of this new facility,

The doc patch you applied does nothing to address these points

> nor does the perf tips
> > section.

Only that one.

> > Also says that the option doesn't work with --single-transaction, so we
> > should say that you've added in TRUNCATE to make the COPY go faster (or
> > haven't you? I know we discussed it).
>
> Good point.  Documentation patch attached and applied.
>
--
 Simon Riggs           www.2ndQuadrant.com

Simon Riggs wrote:
> On Thu, 2010-02-25 at 21:32 -0500, Bruce Momjian wrote:
> > Simon Riggs wrote:
> > >
> > > On Tue, 2009-04-14 at 00:06 +0000, Alvaro Herrera wrote:
> > > > Log Message:
> > > > -----------
> > > > pg_restore -jN does not equate "multiple jobs", so partly revert the
> > > > previous patch.
> > >
> > > I notice that the release notes talk about "parallel restore", yet the
> > > word parallel is not used in the docs anywhere. The section on restoring
> > > a dump makes no mention of this new facility,
>
> The doc patch you applied does nothing to address these points

We don't usually modify past release notes.

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +

On Fri, 2010-02-26 at 13:25 -0500, Bruce Momjian wrote:
> Simon Riggs wrote:
> > On Thu, 2010-02-25 at 21:32 -0500, Bruce Momjian wrote:
> > > Simon Riggs wrote:
> > > >
> > > > On Tue, 2009-04-14 at 00:06 +0000, Alvaro Herrera wrote:
> > > > > Log Message:
> > > > > -----------
> > > > > pg_restore -jN does not equate "multiple jobs", so partly revert the
> > > > > previous patch.
> > > >
> > > > I notice that the release notes talk about "parallel restore", yet the
> > > > word parallel is not used in the docs anywhere. The section on restoring
> > > > a dump makes no mention of this new facility,
> >
> > The doc patch you applied does nothing to address these points
>
> We don't usually modify past release notes.

Cool, but my comment has nothing to do with that. Please reread.

--
 Simon Riggs           www.2ndQuadrant.com

Simon Riggs wrote:
> On Fri, 2010-02-26 at 13:25 -0500, Bruce Momjian wrote:
> > Simon Riggs wrote:
> > > On Thu, 2010-02-25 at 21:32 -0500, Bruce Momjian wrote:
> > > > Simon Riggs wrote:
> > > > >
> > > > > On Tue, 2009-04-14 at 00:06 +0000, Alvaro Herrera wrote:
> > > > > > Log Message:
> > > > > > -----------
> > > > > > pg_restore -jN does not equate "multiple jobs", so partly revert the
> > > > > > previous patch.
> > > > >
> > > > > I notice that the release notes talk about "parallel restore", yet the
> > > > > word parallel is not used in the docs anywhere. The section on restoring
> > > > > a dump makes no mention of this new facility,
> > >
> > > The doc patch you applied does nothing to address these points
> >
> > We don't usually modify past release notes.
>
> Cool, but my comment has nothing to do with that. Please reread.

I don't see any performance mentions in the restore docs so I am unclear
why we should add them there:

    http://developer.postgresql.org/pgdocs/postgres/backup-dump.html#BACKUP-DUMP-RESTORE

We mention all the performance stuff in the performance section:

    http://developer.postgresql.org/pgdocs/postgres/populate.html#POPULATE-PG-DUMP

--
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com
  PG East:  http://www.enterprisedb.com/community/nav-pg-east-2010.do
  + If your life is a hard drive, Christ can be your backup. +