Re: small parallel restore optimization - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: small parallel restore optimization
Date
Msg-id 49B466B7.70207@dunslane.net
Whole thread Raw
In response to Re: small parallel restore optimization  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: small parallel restore optimization  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: small parallel restore optimization  (Alvaro Herrera <alvherre@commandprompt.com>)
List pgsql-hackers

Tom Lane wrote:
>> I've seen a recent error that suggests we are clobbering memory 
>> somewhere in the parallel code, as well as Olivier Prennant's reported 
>> error that suggests the same thing, although I'm blessed if I can see 
>> where it might be. Maybe some more eyeballs on the code would help.
>>     
>
> Can you put together even a weakly reproducible test case?  Something
> that only fails every tenth or hundredth time would still help.
>
>             
>   

I have found the source of the problem I saw. dumputils.c:fmtId() uses a 
static PQExpBuffer which it initialises the first time it's called. This 
gets clobbered by simultaneous calls by Windows threads.

I could just make it auto and set it up on each call, but that could 
result in a non-trivial memory leak ... it's probably called a great 
many times. Or I could provide a parallel version where we pass in a 
PQExpBuffer that we create, one per thread, and is used by anything 
called by the parallel code. That seems like a bit of a potential 
footgun, though.

Has anyone got a better plan?

cheers

andrew


pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: postgresql.conf: patch to have ParseConfigFile report all parsing errors, then bail
Next
From: Tom Lane
Date:
Subject: Re: postgresql.conf: patch to have ParseConfigFile report all parsing errors, then bail