Re: Compression and on-disk sorting - Mailing list pgsql-hackers

From Jim C. Nasby
Subject Re: Compression and on-disk sorting
Date
Msg-id 20060516173107.GH26212@pervasive.com
Whole thread Raw
In response to Re: Compression and on-disk sorting  (Andrew Dunstan <andrew@dunslane.net>)
Responses Re: Compression and on-disk sorting
Re: Compression and on-disk sorting
Re: Compression and on-disk sorting
List pgsql-hackers
On Tue, May 16, 2006 at 12:27:42PM -0400, Andrew Dunstan wrote:
> Rod Taylor wrote:
> >>I habitually turn off all compression on my Windows boxes, because it's 
> >>a performance hit in my experience. Disk is cheap ...
> >>    
> >
> >Disk storage is cheap. Disk bandwidth or throughput is very expensive.

Hey, that's my line! :P

> Sure, but in my experience using Windows File System compression is not 
> a win here. Presumably if it were an unqualified win they would have it 
> turned on everywhere. The fact that there's an option is a good 
> indication that it isn't in many cases. It is most commonly used for 
> files like executables that are in effect read-only - but that doesn't 
> help us.

The issue with filesystem level compression is that it has to support
things like random access, which isn't needed for on-disk sorting (not
sure about other things like hashing, etc).

In any case, my curiousity is aroused, so I'm currently benchmarking
pgbench on both a compressed and uncompressed $PGDATA/base. I'll also do
some benchmarks with pg_tmp compressed.

Does anyone have time to hack some kind of compression into the on-disk
sort code just to get some benchmark numbers? Unfortunately, doing so is
beyond my meager C abilitiy...
-- 
Jim C. Nasby, Sr. Engineering Consultant      jnasby@pervasive.com
Pervasive Software      http://pervasive.com    work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf       cell: 512-569-9461


pgsql-hackers by date:

Previous
From: "Jim C. Nasby"
Date:
Subject: Re: Compression and on-disk sorting
Next
From: "Jim C. Nasby"
Date:
Subject: Re: [BUGS] BUG #2429: Explain does not report object's schema