Re: Slow query (planner insisting on using 'external merge' sort type) - Mailing list pgsql-performance

From Mark Kirkwood
Subject Re: Slow query (planner insisting on using 'external merge' sort type)
Date
Msg-id 558B6115.7020209@catalyst.net.nz
Whole thread Raw
In response to Re: Slow query (planner insisting on using 'external merge' sort type)  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
List pgsql-performance
On 24/06/15 09:05, Jim Nasby wrote:
> On 6/19/15 9:57 AM, Ian Pushee wrote:
>>
>>
>> On 6/19/2015 10:47 AM, Andreas Kretschmer wrote:
>>>> Explain Analyze outputs (links as requested):
>>>> Default plan: http://explain.depesz.com/s/ib3k
>>>> Forced index (random_page_cost=1.0): http://explain.depesz.com/s/lYaP
>>>>
>>>> Software/Hardware: PGSql 9.2.1, Windows 8.1, 8GB RAM
>>>> All pgsql settings are at their defaults.
>>> increase work_mem. per session via set work_mem  = 'xxxMB'; or in
>>> postgresql.conf, reload.
>>>
>>>
>>
>> Hi Andreas,
>>
>> The number of rows in the events table isn't constrained, so
>> unfortunately it isn't feasible to set work_mem high enough to allow an
>> in-memory sort. Forcing the planner to use the index works to produce a
>> fast query, so I'm wondering if there is a more general way to getting
>> the planner to take into account that work_mem isn't big enough to fit
>> the query which will result in a MUCH more costly external merge.
>
> What Andreas is saying is the reason the sort is so expensive is because
> it spilled to disk. If you don't have enough memory to do the sort
> in-memory, then you probably don't have enough memory to buffer the
> table either, which means the index scan is going to be a LOT more
> expensive than a sort.
>
> That said, the better your IO system is the lower you need to set
> random_page_cost. With a good raid setup 2.0 is a good starting point,
> and I've run as low as 1.1. I've never run a system on all SSD, but I've
> heard others recommend setting it as low as 1.0 on an all SSD setup.
>
> It's also worth noting that there's some consensus that the optimizer is
> generally too eager to switch from an index scan to a seqscan.


Mind you, this eagerness could be caused by the OP having
effective_cache_size set to the default. This should be changed (set to
a few GB...)!

Cheers

Mark


pgsql-performance by date:

Previous
From: "Sheena, Prabhjot"
Date:
Subject: pgbouncer issue
Next
From: Josh Berkus
Date:
Subject: Does anyone have python code which digests pgbench -r output?