Re: Patch: New GUC prepared_statement_limit to limit memory used byprepared statements - Mailing list pgsql-hackers

From Ibrar Ahmed
Subject Re: Patch: New GUC prepared_statement_limit to limit memory used byprepared statements
Date
Msg-id CALtqXTdLY1uHqifLWJzjKkod-X6GO-+JLFUgLPag+pO+xMWEmQ@mail.gmail.com
Whole thread Raw
In response to Patch: New GUC prepared_statement_limit to limit memory used byprepared statements  (Daniel Migowski <dmigowski@ikoffice.de>)
Responses Re: Patch: New GUC prepared_statement_limit to limit memory used byprepared statements  (Daniel Migowski <dmigowski@ikoffice.de>)
List pgsql-hackers


On Sat, Aug 17, 2019 at 6:58 PM Daniel Migowski <dmigowski@ikoffice.de> wrote:
Hello,

attached you find a patch that adds a new GUC:

Quick questions before looking at the patch. 

prepared_statement_limit:

 - Do we have a consensus about the name of GUC? I don't think it is
the right name for that. 

 - Is this a WIP patch or the final patch? Because I can see TODO and non-standard
comments in the patch.

 
         Specifies the maximum amount of memory used in each session to
cache
         parsed-and-rewritten queries and execution plans. This affects
the maximum memory
         a backend threads will reserve when many prepared statements
are used.
         The default value of 0 disables this setting, but it is
recommended to set this
         value to a bit lower than the maximum memory a backend worker
thread should reserve
         permanently.

If the GUC is configured after each save of a CachedPlanSource, or after
creating a CachedPlan from it, the function
EnforcePreparedStatementLimit is called now. It checks the mem usage of
the existing saved CachedPlanSources and invalidates the query_list and
the gplan if available until the memory limit is met again.

CachedPlanSource are removed-and-tailadded in the saved_plan_list
everytime GetCachedPlan is called on them so it can be used as a LRU list.

I also reworked ResetPlanCache, PlanCacheRelCallback and
PlanCacheObjectCallback a bit so when a CachedPlanSource is invalidated
the query_list is not only marked as invalid but it is also fully
released to free memory here.

Regards,
Daniel Migowski

PS@Konstantin: This patch also includes the CachedPlanMemoryUsage
function you like, maybe you like the review the patch for me?


 


--
Ibrar Ahmed

pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: [Proposal] Table-level Transparent Data Encryption (TDE) and KeyManagement Service (KMS)
Next
From: Andres Freund
Date:
Subject: Re: POC: Cleaning up orphaned files using undo logs