Re: Threading in BGWorkers (!) - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Threading in BGWorkers (!)
Date
Msg-id 20200624014431.aq2y6hwb4ntynume@alap3.anarazel.de
Whole thread Raw
In response to Re: Threading in BGWorkers (!)  (Chapman Flack <chap@anastigmatix.net>)
Responses Re: Threading in BGWorkers (!)  (Chapman Flack <chap@anastigmatix.net>)
Re: Threading in BGWorkers (!)  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-hackers
Hi,

On 2020-06-23 09:19:36 -0400, Chapman Flack wrote:
> 1) It would be nice to be able to ereport from an arbitrary thread. There
>    is already support in core to forward messages from parallel workers:
>    the worker signals the lead process after adding a message to a shm_mq
>    referenced from its ParallelWorkerInfo struct. The signal handler
>    asynchronously sets ParallelMessagePending, which ProcessInterrupts
>    will check at some convenient point and ereport the message.
> 
>    It seems like it would be no sweat for another thread in the same
>    process to add something to an mq (could be the same structure as
>    shm_mq but would not need to really be in shared memory) and do a
>    volatile write of ParallelMessagePending. The rest is already there.
>    Only missing ingredient would be a way for an extension to allocate
>    something like a ParallelWorkerInfo struct good for the life of the
>    backend (the current parallel worker infrastructure makes them all
>    go away at the completion of a parallel query).

I think that's way harder than what you make it sound here. The locking
for shm_mq doesn't really work inside a process. In contrast to the
single threaded case something like a volatile write to
ParallelMessagePending doesn't guarantee much, because there's no
guaranteed memory ordering between threads. And more.


I'm very doubtful this is a good direction to go in. Kinda maybe
somewhat partially converting tiny parts of the backend code into
threadsafe code will leave us with some baroque code.

Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Alvaro Herrera
Date:
Subject: Re: Assertion failure in pg_copy_logical_replication_slot()
Next
From: Chapman Flack
Date:
Subject: Re: Threading in BGWorkers (!)