Re: patch for parallel pg_dump - Mailing list pgsql-hackers

From Alvaro Herrera
Subject Re: patch for parallel pg_dump
Date
Msg-id 1332205653-sup-4987@alvh.no-ip.org
Whole thread Raw
In response to Re: patch for parallel pg_dump  (Joachim Wieland <joe@mcknight.de>)
Responses Re: patch for parallel pg_dump
List pgsql-hackers
Excerpts from Joachim Wieland's message of lun mar 19 00:31:47 -0300 2012:
> On Wed, Mar 14, 2012 at 2:02 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> >> I think we should somehow unify both functions, the code is not very
> >> consistent in this respect, it also calls exit_horribly() when it has
> >> AH available. See for example pg_backup_tar.c
> >
> > I think we should get rid of die_horribly(), and instead have arrange
> > to always clean up AH via an on_exit_nicely hook.
>
> Attached is a patch that gets rid of die_horribly().
>
> For the parallel case it maintains an array with as many elements as
> we have worker processes. When the workers start, they enter their Pid
> (or ThreadId) and their ArchiveHandle (AH). The exit handler function
> in a process can then find its own ArchiveHandle by comparing the own
> Pid with all the elements in the array.

Sounds good to me in general ... my only gripe is this: I wonder if it
would be better to have a central routine that knows about both
archive_close_connection and archive_close_connection_parallel -- and
the argument to the callback is a struct that contains both a pointer to
the struct with the connection to be closed, a ParallelState (either of
which can be null), and a flag stating which of the ParallelState/AH is
in use.  That way, you avoid having to reset the callbacks when you
switch from AH to parallel; instead you just clear out the AH
connection, set the ParallelState, and flip the switch.  The general
archive_close_connection checks the flag to know which routine to call.

I mean, what you have probably works fine now, but it doesn't seem very
extensible.

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Memory usage during sorting
Next
From: Josh Berkus
Date:
Subject: Re: Gsoc2012 Idea --- Social Network database schema