Re: OOM on EXPLAIN with lots of nodes - Mailing list pgsql-hackers

From Tom Lane
Subject Re: OOM on EXPLAIN with lots of nodes
Date
Msg-id 11105.1421162380@sss.pgh.pa.us
Whole thread Raw
In response to Re: OOM on EXPLAIN with lots of nodes  (Heikki Linnakangas <hlinnakangas@vmware.com>)
Responses Re: OOM on EXPLAIN with lots of nodes
List pgsql-hackers
Heikki Linnakangas <hlinnakangas@vmware.com> writes:
> On 01/13/2015 02:08 PM, Alexey Bashtanov wrote:
>> By varying the parameters and reading source code I determined that
>> memory usage linearly depends on (plan nodes count)*(overall columns
>> count), thus it quadratically depends on number of tables unionized.
>> 
>> To remove this excessive memory usage I propose
>> to run deparse_context_for_planstate+deparse_expression in a separate
>> memory context and free it after a plan node is generated.

> Hmm, something like the attached? Seems reasonable...

This looks pretty unsafe to me: it assumes, without much justification,
that there is no memory allocated during show_expression() that will be
needed later.

I suspect the O(N^2) consumption comes directly from some workspace
allocated for variable deparsing in ruleutils.c.  A safer fix would
be to do retail pfrees on that.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Safe memory allocation functions
Next
From: Peter Eisentraut
Date:
Subject: Re: [BUGS] BUG #12070: hstore extension: hstore_to_json_loose produces invalid JSON