Re: [HACKERS] Early locking option to parallel backup - Mailing list pgsql-hackers

From Andres Freund
Subject Re: [HACKERS] Early locking option to parallel backup
Date
Msg-id 20171105230918.whtj3dur2ns2zolu@alap3.anarazel.de
Whole thread Raw
In response to Re: [HACKERS] Early locking option to parallel backup  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: [HACKERS] Early locking option to parallel backup  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: [HACKERS] Early locking option to parallel backup  (Lucas B <lucas75@gmail.com>)
List pgsql-hackers
On 2017-11-05 17:38:39 -0500, Robert Haas wrote:
> On Sun, Nov 5, 2017 at 5:17 AM, Lucas <lucas75@gmail.com> wrote:
> > The patch creates a "--lock-early" option which will make pg_dump to issue
> > shared locks on all tables on the backup TOC on each parallel worker start.
> > That way, the backup has a very small chance of failing. When it does,
> > happen in the first few seconds of the backup job. My backup scripts (not
> > included here) are aware of that and retries the backup in case of failure.
> 
> I wonder why we don't do this already ... and by default.

Well, the current approach afaics requires #relations * 2 locks, whereas
acquiring them in every worker would scale that with the number of
workers.  IIUC the problem here is that even though a lock is already
held by the main backend an independent locker's request will prevent
the on-demand lock by the dump worker from being granted.  It seems to
me the correct fix here would be to somehow avoid the fairness logic in
the parallel dump case - although I don't quite know how to best do so.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: [HACKERS] Early locking option to parallel backup
Next
From: "Tsunakawa, Takayuki"
Date:
Subject: Re: [HACKERS] Statement-level rollback