Re: small parallel restore optimization - Mailing list pgsql-hackers

From Magnus Hagander
Subject Re: small parallel restore optimization
Date
Msg-id 49B4E700.8000106@hagander.net
Whole thread Raw
In response to Re: small parallel restore optimization  (Andrew Dunstan <andrew@dunslane.net>)
Responses Re: small parallel restore optimization  (Andrew Dunstan <andrew@dunslane.net>)
List pgsql-hackers
Andrew Dunstan wrote:
> 
> 
> Alvaro Herrera wrote:
>> Andrew Dunstan wrote:
>>
>>  
>>> I have found the source of the problem I saw. dumputils.c:fmtId()
>>> uses a  static PQExpBuffer which it initialises the first time it's
>>> called. This  gets clobbered by simultaneous calls by Windows threads.
>>>
>>> I could just make it auto and set it up on each call, but that could 
>>> result in a non-trivial memory leak ... it's probably called a great 
>>> many times. Or I could provide a parallel version where we pass in a 
>>> PQExpBuffer that we create, one per thread, and is used by anything 
>>> called by the parallel code. That seems like a bit of a potential 
>>> footgun, though.
>>>     
>>
>> Could you use a different static PQExpBuffer on each thread?
>> pthread_getspecific sort of thing, only to be used on Windows.
>>   
> 
> For MSVC we could declare it with "_declspec(thread)" and it would be
> allocated in thread-local storage, but it looks like this isn't
> supported on Mingw.

Yeah, _declspec(thread) would be the easy way to do it. But you can also
use the TlsAlloc(), TlsSetValue() and friends functions directly. We do
this to use TLS in ecpg.

It requires some coding around, but for ecpg we did a macro that makes
it behave like the posix functions (see
src/interfaces/ecpg/include/ecpg-pthread-win32.h)

//Magnus



pgsql-hackers by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: small parallel restore optimization
Next
From: Alvaro Herrera
Date:
Subject: problem inserting in GIN index