Re: Parallel threads in query - Mailing list pgsql-hackers

From David Fetter
Subject Re: Parallel threads in query
Date
Msg-id 20181031231137.GH12677@fetter.org
Whole thread Raw
In response to Parallel threads in query  (Darafei "Komяpa" Praliaskouski <me@komzpa.net>)
List pgsql-hackers
On Wed, Oct 31, 2018 at 09:07:43AM -1000, Darafei "Komяpa" Praliaskouski wrote:
> Hi,
> 
> I've tried porting some of PostGIS algorithms to utilize multiple cores via
> OpenMP to return faster.

Great!

> Question is, what's the best policy to allocate cores so we can play nice
> with rest of postgres?

As Tom mentioned, the PostgreSQL backend is not yet threaded and is
not likely to be any time in the next couple of years. There
has been at least one attempt recently to start this work.
https://www.postgresql.org/message-id/9defcb14-a918-13fe-4b80-a0b02ff85527%40postgrespro.ru 

> What I'd like to see is some function that I can call and get a
> number of threads I'm allowed to run, that will also advise rest of
> postgres to not use them, and a function to return the cores back
> (or do it automatically at the end of query). Is there an
> infrastructure for that?

Not really, as above.  In the case of GIS queries, you may be able to
send a large-enough payload of work to make it worthwhile to do some
kind of IPC or even a (short, high-bandwidth, low-latency) network hop
to communicate with a separate OpenMP GIS server.  So long as the
threads don't directly interact with the backend, you could do this
safely.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate


pgsql-hackers by date:

Previous
From: Tomas Vondra
Date:
Subject: Re: Super PathKeys (Allowing sort order through precision lossfunctions)
Next
From: David Rowley
Date:
Subject: Re: Super PathKeys (Allowing sort order through precision loss functions)