Re: proposal: Allocate work_mem From Pool - Mailing list pgsql-hackers

From Justin Pryzby
Subject Re: proposal: Allocate work_mem From Pool
Date
Msg-id 20220711043930.GR13040@telsasoft.com
Whole thread Raw
In response to proposal: Allocate work_mem From Pool  ("Joseph D Wagner" <joe@josephdwagner.info>)
Responses RE: proposal: Allocate work_mem From Pool
List pgsql-hackers
On Sun, Jul 10, 2022 at 08:45:38PM -0700, Joseph D Wagner wrote:

> However, that's risky because it's 3GB per operation, not per
> query/connection; it could easily spiral out of control.

This is a well-known deficiency.
I suggest to dig up the old threads to look into.
It's also useful to include links to the prior discussion.

> I think it would be better if work_mem was allocated from a pool of memory

I think this has been proposed before, and the issue/objection with this idea
is probably that query plans will be inconsistent, and end up being
sub-optimal.

work_mem is considered at planning time, but I think you only consider its
application execution.  A query that was planned with the configured work_mem
but can't obtain the expected amount at execution time might perform poorly.
Maybe it should be replanned with lower work_mem, but that would lose the
arms-length relationship between the planner-executor.

Should an expensive query wait a bit to try to get more work_mem?
What do you do if 3 expensive queries are all waiting ?

-- 
Justin



pgsql-hackers by date:

Previous
From: John Naylor
Date:
Subject: Re: [PATCH] Optimize json_lex_string by batching character copying
Next
From: Thomas Munro
Date:
Subject: Re: pgsql: dshash: Add sequential scan support.