Re: Pg_upgrade speed for many tables - Mailing list pgsql-hackers

From Jeff Davis
Subject Re: Pg_upgrade speed for many tables
Date
Msg-id 1352747558.14335.5.camel@sussancws0025
Whole thread Raw
In response to Re: Pg_upgrade speed for many tables  (Jeff Janes <jeff.janes@gmail.com>)
List pgsql-hackers
On Mon, 2012-11-12 at 10:29 -0800, Jeff Janes wrote:
> When I'm doing a pg_upgrade with thousands of tables, the shutdown
> checkpoint after restoring the dump to the new cluster takes a very
> long time, as the writer drains its operation table by opening and
> individually fsync-ing thousands of files.

This reminds me of the fix I did for initdb to sync the files. I think
we do need to make sure they are sync'd, because ext4 can keep buffers
around for quite a long time without cleaning them.

I ended up using sync_file_range(..., SYNC_FILE_RANGE_WRITE) on linux,
and posix_fadvise(..., POSIX_FADV_DONTNEED) on everything else, and that
made subsequent fsyncs more efficient.

Regards,Jeff Davis





pgsql-hackers by date:

Previous
From: Jeff Janes
Date:
Subject: Re: Pg_upgrade speed for many tables
Next
From: Simon Riggs
Date:
Subject: Re: Proof of concept: standalone backend with full FE/BE protocol