Re: [HACKERS] Proposal : For Auto-Prewarm. - Mailing list pgsql-hackers

From Mithun Cy
Subject Re: [HACKERS] Proposal : For Auto-Prewarm.
Date
Msg-id CAD__Ouj=epTojkfTBbKgBSmKKiiKEKxUha1FDbbW8kGhUs_nnQ@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Proposal : For Auto-Prewarm.  (Amit Kapila <amit.kapila16@gmail.com>)
List pgsql-hackers
On Thu, Apr 6, 2017 at 4:12 AM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-03-13 18:45:00 +0530, Mithun Cy wrote:
>> I have implemented a similar logic now. The prewarm bgworker will
>> launch a sub-worker per database in the dump file. And, each
>> sub-worker will load its database block info. The sub-workers will be
>> launched only after previous one is finished. All of this will only
>> start if the database has reached a consistent state.
>
> Hm. For replay performance it'd possibly be good to start earlier,
> before reaching consistency.  Is there an issue starting earlier?

Thanks Andres for a detailed review. I will try to address them in my next post. I thought it is important to reply to above comment before that. Earlier patches used to start loading blocks before reaching a consistent state. Then Robert while reviewing found a basic flaw in my approach[1].  The function DropRelFileNodesAllBuffers do not expect others to load the blocks concurrently while it is getting rid of buffered blocks. So has to delay loading until database reaches consistent state so that we can connect to each database and take a relation lock before loading any of theirs blocks.


[1] cannot load blocks without holding relation lock


--
Thanks and Regards
Mithun C Y
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: [HACKERS] SCRAM authentication, take three
Next
From: Heikki Linnakangas
Date:
Subject: Re: [HACKERS] Letting the client choose the protocol to use during aSASL exchange