Block at a time ... - Mailing list pgsql-performance

From Dave Crooke
Subject Block at a time ...
Date
Msg-id ca24673e1003161658g6a0fa30dg2c158db395ac08@mail.gmail.com
Whole thread Raw
Responses Re: Block at a time ...  (Alvaro Herrera <alvherre@commandprompt.com>)
List pgsql-performance
I agree with Tom, any reordering attempt is at best second guessing the filesystem and underlying storage.

However, having the ability to control the extent size would be a worthwhile improvement for systems that walk and chew gum (write to lots of tables) concurrently.

I'm thinking of Oracle's AUTOEXTEND settings for tablespace datafiles .... I think the ideal way to do it for PG would be to make the equivalent configurable in postgresql.conf system wide, and allow specific per-table settings in the SQL metadata, similar to auto-vacuum.

An awesomely simple alternative is to just specify the extension as e.g. 5% of the existing table size .... it starts by adding one block at a time for tiny tables, and once your table is over 20GB, it ends up adding a whole 1GB file and pre-allocating it. Very little wasteage.

Cheers
Dave

On Tue, Mar 16, 2010 at 4:49 PM, Alvaro Herrera <alvherre@commandprompt.com> wrote:
Tom Lane escribió:
> Alvaro Herrera <alvherre@commandprompt.com> writes:
> > Maybe it would make more sense to try to reorder the fsync calls
> > instead.
>
> Reorder to what, though?  You still have the problem that we don't know
> much about the physical layout on-disk.

Well, to block numbers as a first step.

However, this reminds me that sometimes we take the block-at-a-time
extension policy too seriously.  We had a customer that had a
performance problem because they were inserting lots of data to TOAST
tables, causing very frequent extensions.  I kept wondering whether an
allocation policy that allocated several new blocks at a time could be
useful (but I didn't try it).  This would also alleviate fragmentation,
thus helping the physical layout be more similar to logical block
numbers.

--
Alvaro Herrera                                http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

pgsql-performance by date:

Previous
From: Greg Smith
Date:
Subject: Re: shared_buffers advice
Next
From: Alvaro Herrera
Date:
Subject: Re: Block at a time ...