Re: file cloning in pg_upgrade and CREATE DATABASE - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: file cloning in pg_upgrade and CREATE DATABASE
Date
Msg-id 20180326061516.GC2759@paquier.xyz
Whole thread Raw
In response to Re: file cloning in pg_upgrade and CREATE DATABASE  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
Responses Re: file cloning in pg_upgrade and CREATE DATABASE  (Peter Eisentraut <peter.eisentraut@2ndquadrant.com>)
List pgsql-hackers
On Sun, Mar 25, 2018 at 09:33:38PM -0400, Peter Eisentraut wrote:
> On 3/21/18 22:38, Michael Paquier wrote:
>> At least on Linux it is possible to rely on sync_file_range which is
>> called via pg_flush_data, so it seems to me that we ought to roughly
>> keep the loop working on FLUSH_DISTANCE, and replace the calls of
>> read/write by copy_file_range.  copyfile is only able to do a complete
>> file copy, so we would also lose this property as well on Linux.
>
> I have shown earlier in the thread that copy_file_range in one go is
> still better than doing it in pieces.

f8c183a has introduced the optimization that your patch is removing,
which was discussed on this thread:
https://www.postgresql.org/message-id/flat/4B78906A.7020309%40mark.mielke.cc
I am not much into the internals of copy_file_range, but isn't there a
risk to have a large range of blocks copied to discard potentially
useful blocks from the OS cache?  That's what this patch makes me worry
about.  Performance is good, but on a system where the OS cache is
heavily used for a set of hot blocks this could cause performance side
effects that I think we canot neglect.

Another thing is that 71d6d07 allowed a couple of database commands to
be more sensitive to interruptions.  With large databases used as a base
template it seems to me that this would cause the interruptions to be
less responsive.
--
Michael

Attachment

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [HACKERS] AdvanceXLInsertBuffer vs. WAL segment compressibility
Next
From: Michael Paquier
Date:
Subject: Re: ppc64le support in 9.3 branch?