Re: Table and Index compression - Mailing list pgsql-hackers

From Sam Mason
Subject Re: Table and Index compression
Date
Msg-id 20090807102937.GA5407@samason.me.uk
Whole thread Raw
In response to Re: Table and Index compression  (Greg Stark <gsstark@mit.edu>)
Responses Re: Table and Index compression  (Greg Stark <gsstark@mit.edu>)
List pgsql-hackers
On Fri, Aug 07, 2009 at 10:33:33AM +0100, Greg Stark wrote:
> 2009/8/7 Pierre Frédéric Caillaud <lists@peufeu.com>:
> > Also, about compressed NTFS : it can give you disk-full errors on read().
> 
> I suspect it's unavoidable for similar reasons to the problems
> Postgres faces. When you issue a read() you have to find space in the
> filesystem cache to hold the data. Some other data has to be evicted.
> If that data doesn't compress as well as it did previously it could
> take more space and cause the disk to become full.
> 
> This also implies that fsync() could generate that error...

If that's indeed how it works it seems like one broken file system and
needs to get its block accounting in order.

When you choose a compression algorithm you know how much space a worst
case compression will take (i.e. lzo takes up to 8% more for a 4kB block
size).  This space should be reserved in case of situations like the
above and the filesystem shouldn't over-commit on this.

Never had to think about this before though so I'm probably missing
something obvious.

--  Sam  http://samason.me.uk/


pgsql-hackers by date:

Previous
From: Bernd Helmle
Date:
Subject: Re: mixed, named notation support
Next
From: Greg Stark
Date:
Subject: Re: Table and Index compression