Re: small parallel restore optimization - Mailing list pgsql-hackers

From Andrew Dunstan
Subject Re: small parallel restore optimization
Date
Msg-id 49B474C8.6090000@dunslane.net
Whole thread Raw
In response to Re: small parallel restore optimization  (Alvaro Herrera <alvherre@commandprompt.com>)
Responses Re: small parallel restore optimization  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: small parallel restore optimization  (Magnus Hagander <magnus@hagander.net>)
List pgsql-hackers

Alvaro Herrera wrote:
> Andrew Dunstan wrote:
>
>   
>> I have found the source of the problem I saw. dumputils.c:fmtId() uses a  
>> static PQExpBuffer which it initialises the first time it's called. This  
>> gets clobbered by simultaneous calls by Windows threads.
>>
>> I could just make it auto and set it up on each call, but that could  
>> result in a non-trivial memory leak ... it's probably called a great  
>> many times. Or I could provide a parallel version where we pass in a  
>> PQExpBuffer that we create, one per thread, and is used by anything  
>> called by the parallel code. That seems like a bit of a potential  
>> footgun, though.
>>     
>
> Could you use a different static PQExpBuffer on each thread?
> pthread_getspecific sort of thing, only to be used on Windows.
>   

For MSVC we could declare it with "_declspec(thread)" and it would be 
allocated in thread-local storage, but it looks like this isn't 
supported on Mingw.

> BTW does fmtQualifiedId have the same problem?
>
>   

Yes, but it's not called in threaded code - it's only in pg_dump and 
only pg_restore is threaded.

cheers

andrew


pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: small parallel restore optimization
Next
From: Tom Lane
Date:
Subject: Re: small parallel restore optimization