Re: Parallel query and temp_file_limit - Mailing list pgsql-hackers

From Robert Haas
Subject Re: Parallel query and temp_file_limit
Date
Msg-id CA+TgmoaPF13TV56ye51xnuc1OkETuh0EoWw_3+ap7Hy6BGv5aw@mail.gmail.com
Whole thread Raw
In response to Re: Parallel query and temp_file_limit  (Peter Geoghegan <pg@heroku.com>)
Responses Re: Parallel query and temp_file_limit  (David Rowley <david.rowley@2ndquadrant.com>)
Re: Parallel query and temp_file_limit  (Peter Geoghegan <pg@heroku.com>)
Re: Parallel query and temp_file_limit  (Peter Geoghegan <pg@heroku.com>)
List pgsql-hackers
On Tue, May 17, 2016 at 6:40 PM, Peter Geoghegan <pg@heroku.com> wrote:
> On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan <pg@heroku.com> wrote:
>> Fundamentally, since temporary_files_size enforcement simply
>> piggy-backs on low-level fd.c file management, without any
>> consideration of what the temp files contain, it'll be hard to be sure
>> that parallel workers will not have issues. I think it'll be far
>> easier to fix the problem then it would be to figure out if it's
>> possible to get away with it.
>
> I'll write a patch to fix the issue, if there is a consensus on a solution.

I think for 9.6 we just have to document this issue.  In the next
release, we could (and might well want to) try to do something more
clever.

What I'm tempted to do is trying to document that, as a point of
policy, parallel query in 9.6 uses up to (workers + 1) times the
resources that a single session might use.  That includes not only CPU
but also things like work_mem and temp file space.  This obviously
isn't ideal, but it's what could be done by the ship date.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Reviewing freeze map code
Next
From: David Rowley
Date:
Subject: Re: Parallel query and temp_file_limit