Re: performance of SELECT * much faster than SELECT with large offset - Mailing list pgsql-performance

From Marti Raudsepp
Subject Re: performance of SELECT * much faster than SELECT with large offset
Date
Msg-id CABRT9RC5rqbVYpPvJSFGNXVykLv8O9iG7--qdnxdQWNLwiaNug@mail.gmail.com
Whole thread Raw
In response to Re: performance of SELECT * much faster than SELECT with large offset  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-performance
On Fri, Oct 3, 2014 at 5:39 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Marc Slemko <marcs@znep.com> writes:
>> I ran into this oddity lately that goes against everything I thought I
>> understood and was wondering if anyone had any insight.
>
> SELECT * avoids a projection step ... see ExecAssignScanProjectionInfo.

It would be cool if OFFSET could somehow signal the child nodes "don't
bother constructing the actual tuple". Not sure if that could work in
more complex queries. But this is just one of many performance
problems with large OFFSETs.

Of course you can always work around this using a subquery...
select description from (
  select * from ccrimes offset 5140000 limit 1
) subq;

But most of the time it's better to use scalable paging techniques:
http://use-the-index-luke.com/sql/partial-results/fetch-next-page

Regards,
Marti


pgsql-performance by date:

Previous
From: Tom Lane
Date:
Subject: Re: performance of SELECT * much faster than SELECT with large offset
Next
From: Roberto Grandi
Date:
Subject: Planning for Scalability