Re: [HACKERS] sort on huge table - Mailing list pgsql-hackers

From Tatsuo Ishii
Subject Re: [HACKERS] sort on huge table
Date
Msg-id 199911020407.NAA01247@srapc451.sra.co.jp
Whole thread Raw
In response to Re: [HACKERS] sort on huge table  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: [HACKERS] sort on huge table
List pgsql-hackers
>> At 06:24 PM 11/1/99 -0800, Mike Mascari wrote:
>>> I know this is a VERY long shot, but... what were the READ/WRITE ratios
>>> between the old version and the new version? Perhaps the computation
>>> of the checksum (sic) blocks under RAID5 caused the unexpected behavior. 
>
>Good try but no cigar --- we're dealing with a merge algorithm here,
>and it's inherently the same amount of data in and out.  You write
>a block once, you read the same block once later on.  But...
>
>Don Baccus <dhogaza@pacifier.com> writes:
>> RAID 5, not the operating system, might be getting in the way...it
>> would be interesting to test this on a Linux 2.2 kernel without
>> the RAID 5 complication.
>
>... I agree this'd be worth trying.  There could be some subtle effect
>somewhere in RAID5 that's tripping things up.  It'd also be useful if
>someone could try it on similar RAID hardware with a non-Linux kernel.

I have compared current with 6.5 using 1000000 tuple-table (243MB) (I
wanted to try 2GB+ table but 6.5 does not work in this case). The
result was strange in that current is *faster* than 6.5!

RAID5current    2:296.5.2    3:15

non-RAIDcurrent    1:506.5.2    2:13

Seems my previous testing was done in wrong way or the behavior of
sorting might be different if the table size is changed?

Anyway, here is my test script.
First, edit Makefile to set DB and number of tuples. Then run type
make. That's all.
--
Tatsuo Ishii
--------------------------------------------------------------------
begin 644 sort.tar.gz
M'XL(`(-F'C@``^V737/:,!"&N5:_8@MD@@D8VQ@\`R$S!=)V.J3I).TIR<'8
M,H@:F]@BDTQ+?WNU_H#2-L.ED![T<$`?[Z[6R"LM<1CQQH7]E7K,IX7]H&M:
MVS2A`-!LZDW\UJV6AM\I+5,#L#3-,LVV9NIBVA"?`FA[BF>+9<SM"*#`ZRR>
M,O:L;CZ)#A'.H2GM@I3`I9Z]]#G$E',63&+PP@@XC;&CDF$?>DF/?/S\Y=/H
M_%ITQ8XC9+=W,A@(_<1QR.#MZ,T[-*Y?&N3Z_?EH)-J-,0L:\91<7>2=:`YU
MCUQ?#5`ZH8'JD,O^A[P3$F+[?@>B99"$1+)&!V?)JW(E<:R`VG`B:G.JQE,H
M5[+`%=$<]A5"A+8CVNA70:/!`*?2`+&%JRM0#Q.GQ/&I'710=W6AX%!N"M4?
MY*7W=Q<QYG_R,^YOC9WYWVZE^6\8>E/'_->;EBGS_Q"46.#X2Y?":<Q=%JK3
M,T+F-@LJ+.!@1Q.G!LY4_$#5JN@\*.0;`<`I5IMULV:0)WR7X(@'%;2#,]`5
M0#DD"IN'#"<>;O0[!4U7J/86D7#A5<3B-(IJ1=P,/%;@R`4G7#P5:T$B1JDX
M="JLIW79:=!E)R>Y\\Q#\<B]Y<4:2^29>B;4LU-#1#;;Z#<6C\5,O-IR=!OD
MX_@L["AY-NCU0-NX^#UN]5=7J[\\F!L&5$T]K_ZG0R')__PLO/?WLL:N_(>6
ME>6_WFZU+;S_35.3^7\(W"A<`+?'/@6N=TGZ)JP'*@PSO`9<7/"/7+R[+QVN
MY!^SE?_3_:RQ*_]-2\_J_Z:E-S7,?Z-MR/P_!*77ZQI;W'8W4"Y!/:!@P%V7
M3T5UB_LS[/>28AK;6:G<2RY%0OV8;@V7]=RB;!"/$;(0EXHHV&%SQ4!YV"=J
M(ZF3,[/OJ<J!8[SSQ;D#7A3.`2N2X#C14V<:0K&'P(/M+)=S2#I%DO;<,=3K
M^:GUA\6ZJKA?TN@IM^1L3F&]<DQ]ZG"HIDL+)V$D;FX8/P$3?X!B!WPV9QP+
FG32BE]XXB40BD4@D$HE$(I%()!*)1"*12"229_@)GWZ*5``H````
`
end


pgsql-hackers by date:

Previous
From: Bruce Momjian
Date:
Subject: 6.5.3 is ready
Next
From: Lamar Owen
Date:
Subject: Re: [HACKERS] change in name of perl?