Re: Speed up the removal of WAL files - Mailing list pgsql-hackers

From Michael Paquier
Subject Re: Speed up the removal of WAL files
Date
Msg-id CAB7nPqTyQAyR56T2hZfZVX0jPf_OYC_w0O1c7kUKZkaw-ad-3Q@mail.gmail.com
Whole thread Raw
In response to Re: Speed up the removal of WAL files  (Fujii Masao <masao.fujii@gmail.com>)
Responses RE: Speed up the removal of WAL files  ("Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>)
List pgsql-hackers
On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki
> <tsunakawa.takay@jp.fujitsu.com> wrote:
>> From: Kyotaro HORIGUCHI [mailto:horiguchi.kyotaro@lab.ntt.co.jp]
>>> The orinal code recycles some of the to-be-removed files, but the patch
>>> removes all the victims.  This may impact on performance.
>>
>> Yes, I noticed it after submitting the patch and was wondering what to do.  Thinking simply, I think it would be
justenough to replace durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and sync the pg_wal
directoryonce in RemoveNonParentXlogFiles() and RemoveOldXlogFiles().  This will benefit the failover time when fast
promotionis not performed.  What do you think? 

Note that durable_rename() also flushes the origin file to make sure
that it does not show up again after a crash.

> It seems not good idea to just replace durable_rename() with rename()
> in RemoveOldXlogFiles()->RemoveXlogFiles()->InstallXLogFileSegment().
> Because that change seems to be able to cause the following problem.

If archiving is enabled, RemoveOldXlogFiles() would create a .ready
flag for all segments that have reappeared. Oops, it archived again.
--
Michael


pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [HACKERS] More stats about skipped vacuums
Next
From: Amit Kapila
Date:
Subject: Re: [HACKERS] [POC] Faster processing at Gather node