Re: EXTERNAL storage and substring on long strings - Mailing list pgsql-performance

From Joe Conway
Subject Re: EXTERNAL storage and substring on long strings
Date
Msg-id 3F297CBA.1080703@joeconway.com
Whole thread Raw
In response to EXTERNAL storage and substring on long strings  (Scott Cain <cain@cshl.org>)
Responses Re: EXTERNAL storage and substring on long strings
Re: EXTERNAL storage and substring on long strings
List pgsql-performance
Scott Cain wrote:
>  Index Scan using feature_pkey on feature  (cost=0.00..3.01 rows=1
> width=153) (actual time=954.13..954.14 rows=1 loops=1)
>    Index Cond: (feature_id = 1)
>  Total runtime: 954.26 msec
> (3 rows)
>
> Whoa!  That's not what I expected, the time to do the query got more
> that twice as long.  So I think, maybe it was just an unlucky section,
> and overall performance will be much better.  So I write a perl script
> to do substring queries over all of my chromosomes at various positions
> and lengths (20,000 queries total).  For comparison, I also ran the same
> script, extracting the chromosomes via sql and doing the substring in
> perl.  Here's what happened:

Hmmm, what happens if you compare with a shorter substring, e.g.:

explain analyze select substring(residues from 1000000 for 2000)
from feature where feature_id=1;

I'm just guessing, but it might be that the extra I/O time to read 20K
of uncompressed text versus the smaller compressed text is enough to
swamp the time saved from not needing to uncompress.

Any other ideas out there?

Joe


pgsql-performance by date:

Previous
From: Scott Cain
Date:
Subject: Re: EXTERNAL storage and substring on long strings
Next
From: Tom Lane
Date:
Subject: Re: EXTERNAL storage and substring on long strings