Re: Excessive memory usage in multi-statement queries w/ partitioning - Mailing list pgsql-hackers

From David Rowley
Subject Re: Excessive memory usage in multi-statement queries w/ partitioning
Date
Msg-id CAKJS1f91ASq6CvmLvVwydGYoVzJh9XGvXntBr1m3z6p3LN7Y=g@mail.gmail.com
Whole thread Raw
In response to Excessive memory usage in multi-statement queries w/ partitioning  (Andreas Seltenreich <andreas.seltenreich@credativ.de>)
List pgsql-hackers
On Thu, 23 May 2019 at 21:19, Julian Schauder
<julian.schauder@credativ.de> wrote:
> > "multiple statements in a single query", did you mean to write
> > session
> > or maybe transaction there?
>
> Maybe the wording isn't perfect. It is required that the querys are
> sent as a single batch. Try the exact bash-script Andreas used for
> updating the parent.

Thanks for explaining.

> > Which version?
>
> Tested including 11.2. Initially found on 11.1. Memory-consumption
> Scales somewhat linearly with existing partitions and ';' delimited
> Querys per single Batch.

Yeah, unfortunately, if the batch contains 100 of those statements
then the planner is going to eat 100 times the memory since it stores
all 100 plans at once.

Since your pruning all but 1 partition then the situation should be
much better for you when you can upgrade to v12. Unfortunately, that's
still about 5 months away.

The best thing you can do for now is going to be either reduce the
number of partitions or reduce the number of statements in the
batch... or install more memory.

-- 
 David Rowley                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services



pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: "long" type is not appropriate for counting tuples
Next
From: Andrew Gierth
Date:
Subject: Re: Patch to fix write after end of array in hashed agg initialization