Re: work_mem = 900MB but Sort Method: external merge Disk: 304008kB - Mailing list pgsql-general

From Reid Thompson
Subject Re: work_mem = 900MB but Sort Method: external merge Disk: 304008kB
Date
Msg-id 4D63F39A.5040604@ateb.com
Whole thread Raw
In response to Re: work_mem = 900MB but Sort Method: external merge Disk: 304008kB  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
On 02/22/2011 12:06 PM, Tom Lane wrote:
> Reid Thompson <Reid.Thompson@ateb.com> writes:
>> What am I missing that causes this to resort to sorting on disk?
>
> The in-memory space required to sort N tuples can be significantly
> larger than the on-disk space, because the latter representation is
> optimized to be small and the in-memory representation not so much.
> I haven't seen a 3X differential before, but it's not outside the realm
> of reason, especially for narrow rows like these where it's all about
> the overhead.  I suspect if you crank work_mem up still more, you'll see
> it switch over.  It flips to on-disk sort when the in-memory
> representation exceeds the limit ...
>
>             regards, tom lane

ahh, ok; the underlying cpn.value table is 11 GB so I understand how even slightly less optimized representation could
be
significantly larger than ~300MB/900MB

Thanks,
reid

pgsql-general by date:

Previous
From: Vibhor Kumar
Date:
Subject: Re: Reordering a table
Next
From: Tom Lane
Date:
Subject: Re: work_mem = 900MB but Sort Method: external merge Disk: 304008kB