Re: [HACKERS] Sequential scan speed, mmap, disk i/o - Mailing list pgsql-hackers

From dg@illustra.com (David Gould)
Subject Re: [HACKERS] Sequential scan speed, mmap, disk i/o
Date
Msg-id 9805140727.AA18793@hawk.illustra.com
Whole thread Raw
In response to Re: [HACKERS] Sequential scan speed, mmap, disk i/o  (Bruce Momjian <maillist@candle.pha.pa.us>)
Responses Re: [HACKERS] Sequential scan speed, mmap, disk i/o
List pgsql-hackers
> > Someone was complaining about sequential scan speed, so I decided to run
> > a test.
>
> > wc                            41 sec
> > wc -l                            31 sec
> > dd if=/u/pg/data/base/test/testv of=/dev/null bs=512    32 sec
> > dd if=/u/pg/data/base/test/testv of=/dev/null bs=8k    31 sec
> > dd if=/u/pg/data/base/test/testv of=/dev/null bs=256k    31 sec
> > dd if=/u/pg/data/base/test/testv of=/dev/null bs=1m    30 sec
> > mmap() of file in 8k chunks                99 sec
> > mmap() of file in 8mb chunks                40 sec
> > mmap() of file in 32mb chunks                56 sec
> >
> > PostgreSQL sequential scan                37 sec
>
> Let me add, these times are on a PP200, with SCSI Ultra Barracuda
> drives, BSD/OS 3.1, 64MB RAM.
>
> --
> Bruce Momjian                          |  830 Blythe Avenue
> maillist@candle.pha.pa.us              |  Drexel Hill, Pennsylvania 19026
>   +  If your life is a hard drive,     |  (610) 353-9879(w)
>   +  Christ can be your backup.        |  (610) 853-3000(h)

Very interesting. Is it possible to get the schema, the query, and a
a sample of the data or a generator program for the data? I am quite surprised
to see us do so well, I would have guess that the per row overhead would
have us down far below wc.

Although, on second though, dd has to write the data as well as read it, and
we don't, and wc has to examine every character, where if the "where clause"
applies to only a portion of the row, we don't.

Still, It would be nice to see more info about this test.

Btw, I hope to post tommorrow some hopefully interesting results about the
speed of TAS and S_LOCK...

-dg

David Gould            dg@illustra.com           510.628.3783 or 510.305.9468
Informix Software  (No, really)         300 Lakeside Drive  Oakland, CA 94612
"Of course, someone who knows more about this will correct me if I'm wrong,
 and someone who knows less will correct me if I'm right."
               --David Palmer (palmer@tybalt.caltech.edu)


pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: Re: [HACKERS] Sequential scan speed, mmap, disk i/o
Next
From: Michael Meskes
Date:
Subject: Re: [HACKERS] core dump during initdb