Re: vacuumdb parallel has a deadlock detected in 9.5.4 - Mailing list pgsql-bugs

From Francisco Olarte
Subject Re: vacuumdb parallel has a deadlock detected in 9.5.4
Date
Msg-id CA+bJJbx8+SKBU=XUE+HxZHysh9226iMfTnA69AznwRTOEGtR7Q@mail.gmail.com
Whole thread Raw
In response to Re: vacuumdb parallel has a deadlock detected in 9.5.4  (Alvaro Herrera <alvherre@2ndquadrant.com>)
Responses Re: vacuumdb parallel has a deadlock detected in 9.5.4  (huang <foggyglass@163.com>)
Re: vacuumdb parallel has a deadlock detected in 9.5.4  (Alvaro Herrera <alvherre@2ndquadrant.com>)
List pgsql-bugs
Hi:

On Wed, Sep 28, 2016 at 10:39 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
>> > There is a error deadlock detected in vacuumdb parallel .
>> > but if I do not use parallel , it has not deadlock detected .
>> man vacuumdb says about -j option:
>>
>> >> Note that using this mode together with the -f (FULL) option might cause
>> >> deadlock failures if certain system catalogs are processed in parallel.
>> so, while I understand it's not pretty, it's well documented.
>
> Yeah.  However I think this behavior is rather unhelpful.  Maybe we
> should try to fix it somehow, but I'm not sure how.  We could say that
> pg_catalog tables can only be processed one at a time, instead of trying
> to paralelize them?  For example: have vacuumdb fill two lists of
> tables, one for pg_catalog and one for tables in other schemas.  Each
> worker chooses a table from the pg_catalog list first if it's not empty
> and there's no other worker doing one of these, otherwise one from the
> other table.
>
> I think that'd fix it, while not destroying paralelizability too badly.

I would propose another behaviour, which I think can solve the problem
without introducing more complex code. Put a couple of flags to vacuum
only catalog tables / non catalog tables ( I believe this can be
served by include/exclude schemas too, which will be even more useful
for other things ). This way one can do a full paralell vacuum of
non-catalog objects followed by a serial one on catalog objects (
which should not be too slow on 'normal' databases ) ( I propose this
order because IIRC normal vacuum updates catalog tables so they get
vacuumed after to tidy them )

Francisco Olarte

pgsql-bugs by date:

Previous
From: Michael Paquier
Date:
Subject: Re: BUG #14243: pg_basebackup failes by a STATUS_DELETE_PENDING file
Next
From: David Rowley
Date:
Subject: Re: BUG #14344: string_agg(DISTINCT ..) crash