Re: [HACKERS] sort on huge table - Mailing list pgsql-hackers

From Tom Lane
Subject Re: [HACKERS] sort on huge table
Date
Msg-id 9486.940105761@sss.pgh.pa.us
Whole thread Raw
In response to Re: [HACKERS] sort on huge table  (Tatsuo Ishii <t-ishii@sra.co.jp>)
Responses Re: [HACKERS] sort on huge table  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: [HACKERS] sort on huge table  (Tatsuo Ishii <t-ishii@sra.co.jp>)
List pgsql-hackers
Tatsuo Ishii <t-ishii@sra.co.jp> writes:
> I have done the 2GB test on current (with your fixes). This time the
> sorting query worked great! I saw lots of temp files, but the total
> disk usage was almost same as before (~10GB). So I assume this is ok.

I have now committed another round of changes that reduce the temp file
size to roughly the volume of data to be sorted.  It also reduces the
number of temp files --- there will be only one per GB of sort data.
If you could try sorting a table larger than 4GB with this code, I'd be
much obliged.  (It *should* work, of course, but I just want to be sure
there are no places that will have integer overflows when the logical
file size exceeds 4GB.)  I'd also be interested in how the speed
compares to the old code on a large table.

Still need to look at the memory-consumption issue ... and CREATE INDEX
hasn't been taught about any of these fixes yet.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [HACKERS] indexable and locale
Next
From: Peter Eisentraut
Date:
Subject: Re: ORACLE COMMENT statement