Re: parallel restore vs. windows - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: parallel restore vs. windows
Date
Msg-id 493ECAE3.6060600@hagander.net
Whole thread Raw
In response to Re: parallel restore vs. windows  (Andrew Dunstan <andrew@dunslane.net>)
Responses Re: parallel restore vs. windows  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Andrew Dunstan wrote:
> 
> 
> Andrew Chernow wrote:
>>>>>
>>>>> Parts of AH need deep cloning, notably the formatData member, which
>>>>> is done in _ReopenArchive().
>>>>>
>>>>
>>>> Is it okay to clone this from within the thread?
>>>
>>> I don't see why not.
>>>
>>
>> Because another thread may be modifying the memory you are trying to
>> clone.  If no one modifies the formatData struct, then why is it being
>> deep copied to begin with.
>>
>>>
>>> The program dies with a nasty dialog box when restoring a dump of the
>>> regression database after the second COPY thread disconnects.
>>
>> Sounds like the friendly and helpful GPF Dialog (General Protection
>> Fault).  This is a core dump which strongly suggests your threads are
>> trampling over one another.  Its possible that a couple threads get
>> fired off but upon the first thread completion, something
>> !(deep_copied) is freed/modified ... bang-bang your dead :o  I tried
>> to find this, but haven't yet.
>>
>> Maybe do a full deep copy in the main thread and comment out any
>> in-thread deep copying.  I wonder if that would work with no other
>> changes.
> 
> I'll try. It's unfortunately not as simple as it sounds, because of the
> way the abstractions are arranged. I can't count the number of times I
> have had to stop and try to clear my head while working on this code.

That's what killed me when I tried to review that stuff and figure it out.

Does that indicate that the abstractions are bad and should be changed,
or just that there's no reasonably way to make the abstractions both
make sense for the internal API itself *and* for being threadsafe?

//Magnus



pgsql-hackers by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: SSL BIO wrappers
Next
From: Tom Lane
Date:
Subject: Re: SSL BIO wrappers