Re: [PERFORM] A Better External Sort? - Mailing list pgsql-hackers

From Josh Berkus
Subject Re: [PERFORM] A Better External Sort?
Date
Msg-id 200510031503.12158.josh@agliodbs.com
Whole thread Raw
In response to Re: [PERFORM] A Better External Sort?  ("Jeffrey W. Baker" <jwbaker@acm.org>)
List pgsql-hackers
Jeffrey,

> I guess database reads are different, but I remain unconvinced that they
> are *fundamentally* different.  After all, a tab-delimited file (my sort
> workload) is a kind of database.

Unfortunately, they are ... because of CPU overheads.   I'm basing what's
"reasonable" for data writes on the rates which other high-end DBs can
make.   From that, 25mb/s or even 40mb/s for sorts should be achievable
but doing 120mb/s would require some kind of breakthrough.

> On a single disk you wouldn't notice, but XFS scales much better when
> you throw disks at it.  I get a 50MB/sec boost from the 24th disk,
> whereas ext3 stops scaling after 16 disks.  For writes both XFS and ext3
> top out around 8 disks, but in this case XFS tops out at 500MB/sec while
> ext3 can't break 350MB/sec.

That would explain it.  I seldom get more than 6 disks (and 2 channels) to
test with.

--
--Josh

Josh Berkus
Aglio Database Solutions
San Francisco

pgsql-hackers by date:

Previous
From: Josh Berkus
Date:
Subject: Re: [PERFORM] A Better External Sort?
Next
From: mark@mark.mielke.cc
Date:
Subject: Re: PG Killed by OOM Condition