RE: Speed up the removal of WAL files - Mailing list pgsql-hackers

From Tsunakawa, Takayuki
Subject RE: Speed up the removal of WAL files
Date
Msg-id 0A3221C70F24FB45833433255569204D1F81C82A@G01JPEXMBYT05
Whole thread Raw
In response to Re: Speed up the removal of WAL files  (Michael Paquier <michael.paquier@gmail.com>)
List pgsql-hackers
Thanks, it seems to require a bit more consideration about RemoveOldXLogFiles().  Let me continue this next month.


Regards
Takayuki Tsunakawa


> -----Original Message-----
> From: Michael Paquier [mailto:michael.paquier@gmail.com]
> Sent: Saturday, November 18, 2017 10:37 PM
> To: Fujii Masao
> Cc: Tsunakawa, Takayuki/綱川 貴之; Kyotaro HORIGUCHI;
> pgsql-hackers@postgresql.org
> Subject: Re: Speed up the removal of WAL files
> 
> On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao.fujii@gmail.com> wrote:
> > On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki
> > <tsunakawa.takay@jp.fujitsu.com> wrote:
> >> From: Kyotaro HORIGUCHI [mailto:horiguchi.kyotaro@lab.ntt.co.jp]
> >>> The orinal code recycles some of the to-be-removed files, but the
> >>> patch removes all the victims.  This may impact on performance.
> >>
> >> Yes, I noticed it after submitting the patch and was wondering what to
> do.  Thinking simply, I think it would be just enough to replace
> durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and
> sync the pg_wal directory once in RemoveNonParentXlogFiles() and
> RemoveOldXlogFiles().  This will benefit the failover time when fast
> promotion is not performed.  What do you think?
> 
> Note that durable_rename() also flushes the origin file to make sure that
> it does not show up again after a crash.
> 
> > It seems not good idea to just replace durable_rename() with rename()
> > in RemoveOldXlogFiles()->RemoveXlogFiles()->InstallXLogFileSegment().
> > Because that change seems to be able to cause the following problem.
> 
> If archiving is enabled, RemoveOldXlogFiles() would create a .ready flag
> for all segments that have reappeared. Oops, it archived again.


pgsql-hackers by date:

Previous
From: Tomas Vondra
Date:
Subject: Re: [HACKERS] Custom compression methods
Next
From: Tomas Vondra
Date:
Subject: Re: [HACKERS] GnuTLS support