Re: tuplesort memory usage: grow_memtuples - Mailing list pgsql-hackers

From Tom Lane
Subject Re: tuplesort memory usage: grow_memtuples
Date
Msg-id 14327.1358446943@sss.pgh.pa.us
Whole thread Raw
In response to Re: tuplesort memory usage: grow_memtuples  (Peter Geoghegan <peter@2ndquadrant.com>)
Responses Re: tuplesort memory usage: grow_memtuples
List pgsql-hackers
Peter Geoghegan <peter@2ndquadrant.com> writes:
> On 8 December 2012 14:41, Andres Freund <andres@2ndquadrant.com> wrote:
>> Is anybody planning to work on this? There hasn't been any activity
>> since the beginning of the CF and it doesn't look like there is much
>> work left?

> I took another look at this.

Applied with some changes:

* Use float8 arithmetic to prevent intermediate-result overflow, as I
suggested last night.

* Rearrange the subsequent checks so that they provide bulletproof
clamping behavior on out-of-range newmemtupsize values; this allows
dropping the very ad-hoc range limiting used in the previous patch (and
in what I had last night).

* Fix the check against availMem; it was supposed to be testing that the
increment in the array size was within availMem.  (In the original
coding the increment was always the same as the original size, but not
so much anymore.)

* Allow the array size to grow to the MaxAllocSize limit, instead of
punting if we'd exceed that.  If we're attempting to use as much of
work_mem as we possibly can, I don't see why that should be a reject
case.

* Improve the comments, including the existing one about the availMem
check, since that evidently wasn't clear enough.

* Copy the whole mess into tuplestore.c too.
        regards, tom lane



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: Re: Slave enters in recovery and promotes when WAL stream with master is cut + delay master/slave
Next
From: Josh Berkus
Date:
Subject: Re: [PATCH]Tablesample Submission