Re: [PATCHES] COPY FROM performance improvements - Mailing list pgsql-performance

From Mark Wong
Subject Re: [PATCHES] COPY FROM performance improvements
Date
Msg-id 200507282343.j6SNhEjA013300@smtp.osdl.org
Whole thread Raw
In response to Re: [PATCHES] COPY FROM performance improvements  ("Luke Lonergan" <llonergan@greenplum.com>)
Responses Re: [PATCHES] COPY FROM performance improvements
Re: [PATCHES] COPY FROM performance improvements
List pgsql-performance
On Fri, 22 Jul 2005 12:28:43 -0700
"Luke Lonergan" <llonergan@greenplum.com> wrote:

> Joshua,
>
> On 7/22/05 10:11 AM, "Joshua D. Drake" <jd@commandprompt.com> wrote:
> > The database server is a PE (Power Edge) 6600
> >
> > Database Server IO:
> >
> > [root@master root]# /sbin/hdparm -tT /dev/sda
> >
> > /dev/sda:
> >   Timing buffer-cache reads:   1888 MB in  2.00 seconds = 944.00 MB/sec
> >   Timing buffered disk reads:   32 MB in  3.06 seconds =  10.46 MB/sec
> >
> > Second Database Server IO:
> >
> > [root@pq-slave root]# /sbin/hdparm -tT /dev/sda
> >
> > /dev/sda:
> >   Timing buffer-cache reads:   1816 MB in  2.00 seconds = 908.00 MB/sec
> >   Timing buffered disk reads:   26 MB in  3.11 seconds =   8.36 MB/sec
> > [root@pq-slave root]#
>
> Can you post the "time dd if=/dev/zero of=bigfile bs=8k count=500000"
> results?  Also do the reverse (read the file) with "time dd if=bigfile
> of=/dev/null bs=8k".
>
> I think you are observing what we've known for a while, hardware RAID is
> horribly slow.  We've not found a hardware RAID adapter of this class yet
> that shows reasonable read or write performance.  The Adaptec 2400R or the
> LSI or others have terrible internal I/O compared to raw SCSI with software
> RAID, and even the CPU usage is higher on these cards while doing slower I/O
> than linux SW RAID.
>
> Notably - we've found that the 3Ware RAID controller does a better job than
> the low end SCSI RAID at HW RAID support, and also exports JBOD at high
> speeds.  If you export JBOD on the low end SCSI RAID adapters, the
> performance is also very poor, though generally faster than using HW RAID.

Are there any recommendations for Qlogic controllers on Linux, scsi or
fiber channel?  I might be able to my hands on some.  I have pci-x slots
for AMD, Itanium, or POWER5 if the architecture makes a difference.

Mark

pgsql-performance by date:

Previous
From: Karim Nassar
Date:
Subject: Two queries are better than one?
Next
From: "Joshua D. Drake"
Date:
Subject: Re: [PATCHES] COPY FROM performance improvements