BUG #13530: sort receives "unexpected out-of-memory situation during sort" - Mailing list pgsql-bugs

From brent_despain@selinc.com
Subject BUG #13530: sort receives "unexpected out-of-memory situation during sort"
Date
Msg-id 20150731220618.2987.1731@wrigleys.postgresql.org
Whole thread Raw
Responses Re: BUG #13530: sort receives "unexpected out-of-memory situation during sort"  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-bugs
The following bug has been logged on the website:

Bug reference:      13530
Logged by:          Brent DeSpain
Email address:      brent_despain@selinc.com
PostgreSQL version: 9.3.4
Operating system:   Ubuntu
Description:

We are occasionally receiving "unexpected out-of-memory situation during
sort".  This happens mostly when we are running our version of pg_dump.  It
returns the data from each table in primary key order.  The amount of data
in each table is relatively small (< 100k in the larges tables).  The error
is not predictably reproducible.

work_mem is set to 1536kB.  Setting work_mem back to default seems to
decrease the occurrence of the error.

We did not have the problem reported in PostgreSQL 8.3.x.

<< Speculation >>
I have not contributed before to PostgreSQL so take this with a grain of
salt.

I reviewed the code and found that the error comes from
grow_memtuples(Tuplesortstate *state) in src\backend\utils\sort\tuplesort.c
or src\backend\utils\sort\tuplestore.c.

The problem seems to come from the new 9.x algorithm to grow memtupsize by
grow_ratio when doubling memtupsize is no longer possible.

It seems if we get really close to availMem the addition of the chunk header
returned by GetMemoryChunkSpace() pushes us over the edge and LACKMEM()
becomes TRUE.

This is my speculation.

pgsql-bugs by date:

Previous
From: Niall Ross
Date:
Subject: Re: BUG #13518: CancelRequest lacks statement identifier
Next
From: Tom Lane
Date:
Subject: Re: BUG #13530: sort receives "unexpected out-of-memory situation during sort"