Re: [HACKERS] sort on huge table - Mailing list pgsql-hackers

From Hannu Krosing
Subject Re: [HACKERS] sort on huge table
Date
Msg-id 381E8CDA.D9B3FB43@tm.ee
Whole thread Raw
In response to Re: [HACKERS] sort on huge table  (Tatsuo Ishii <t-ishii@sra.co.jp>)
List pgsql-hackers
Tatsuo Ishii wrote:
> 
> >... I agree this'd be worth trying.  There could be some subtle effect
> >somewhere in RAID5 that's tripping things up.  It'd also be useful if
> >someone could try it on similar RAID hardware with a non-Linux kernel.
> 
> I have compared current with 6.5 using 1000000 tuple-table (243MB) (I
> wanted to try 2GB+ table but 6.5 does not work in this case). The
> result was strange in that current is *faster* than 6.5!
> 
> RAID5
>         current 2:29
>         6.5.2   3:15
> 
> non-RAID
>         current 1:50
>         6.5.2   2:13
> 
> Seems my previous testing was done in wrong way or the behavior of
> sorting might be different if the table size is changed?

Or the behaviour of RAID5 changes at some size. 

I have set up an IBM Netfinity server with specs similar to yours, except 
that it has 1G memory and 5x9GB disks. The RAID controller is IBM ServeRAID.

It seems that when I try to write over 60 MB sequentially, the write 
performance drops from over 50MB/s to under 2MB/s.

Maybe such behaviour would suggest that building an index and traversing 
that could be faster than full sort ?

The same tests on my single Celeron 450 produced ~10MB/s writes 
whatever the size.

> Anyway, here is my test script.
> First, edit Makefile to set DB and number of tuples. Then run type
> make. That's all.

I'll try to run it tonight (in GMT+2 tz). 

Can't run it earlyer as it is a production site and a highly 
visible web-server.

If I have the time I'll even try my index theory.

--------
Hannu


pgsql-hackers by date:

Previous
From: "Andrij Korud"
Date:
Subject: Re: [HACKERS] Get OID of just inserted record
Next
From: "Gene Sokolov"
Date:
Subject: Re: [HACKERS] file descriptors leak?