Re: parallel pg_restore - WIP patch - Mailing list pgsql-hackers

From Stefan Kaltenbrunner
Subject Re: parallel pg_restore - WIP patch
Date
Msg-id 48E0C217.4070007@kaltenbrunner.cc
Whole thread Raw
In response to Re: parallel pg_restore - WIP patch  (Andrew Dunstan <andrew@dunslane.net>)
Responses Re: parallel pg_restore - WIP patch  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Andrew Dunstan wrote:
> 
> 
> Andrew Dunstan wrote:
>>
>>
>>>
>>> this works better but there is something fishy still - using the same 
>>> dump file I get a proper restore using pg_restore normally. If I 
>>> however use -m for a parallel one I only get parts (in this case only 
>>> 243 of the 709 tables) of the database restored ...
>>>
>>>
>>>
>>
>> Yes, there are several funny things going on, including some stuff 
>> with dependencies. I'll have a new patch tomorrow with luck. Thanks 
>> for testing.
>>
>>
> 
> OK, in this version a whole heap of bugs are fixed, mainly those to do 
> with dependencies and saved state. I get identical row counts in the 
> source and destination now, quite reliably.

this looks much better (for a restore that usually takes 180min I can 
get down to 72min using -m 4) - however especially with higher 
concurrency I'm sometimes running into restore failures due to deadlocks 
happening during constraint restoration (slightly redacted):

pg_restore: [archiver (db)] Error from TOC entry 7765; 2606 1460743180 
FK CONSTRAINT fk_av_relations_av db_owner
pg_restore: [archiver (db)] could not execute query: ERROR:  deadlock 
detected
DETAIL:  Process 18100 waits for AccessExclusiveLock on relation 
1460818342 of database 1460815284; blocked by process 18103.
Process 18103 waits for AccessExclusiveLock on relation 1460818336 of 
database 1460815284; blocked by process 18100.
HINT:  See server log for query details.

ALTER TABLE ONLY foo    ADD CONSTRAINT fk_av_relations_av FOREIGN KEY (vs_id) REFERENCES 
bar ...


pgsql-hackers by date:

Previous
From: pgsql@mohawksoft.com
Date:
Subject: Re: Ad-hoc table type?
Next
From: Tom Lane
Date:
Subject: Re: parallel pg_restore - WIP patch