Re: Proposing pg_hibernate - Mailing list pgsql-hackers
| From | Amit Kapila |
|---|---|
| Subject | Re: Proposing pg_hibernate |
| Date | |
| Msg-id | CAA4eK1LQ9N2-gNazZNb37CWtph79MtYCY4Df1ViHZtgJ2Z5=2w@mail.gmail.com Whole thread Raw |
| In response to | Re: Proposing pg_hibernate (Gurjeet Singh <gurjeet@singh.im>) |
| Responses |
Re: Proposing pg_hibernate
(Gurjeet Singh <gurjeet@singh.im>)
|
| List | pgsql-hackers |
On Wed, Jun 11, 2014 at 7:59 AM, Gurjeet Singh <gurjeet@singh.im> wrote:
> On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
> >> > IIUC on shutdown request, postmaster will send signal to BG Saver
> >> > and BG Saver will save the buffers and then postmaster will send
> >> > signal to checkpointer to shutdown. So before writing Checkpoint
> >> > record, BG Saver can crash (it might have saved half the buffers)
> >>
> >> Case handled as described above.
> >>
> >> > or may BG saver saves buffers, but checkpointer crashes (due to
> >> > power outage or any such thing).
> >>
> >> Checkpointer process' crash seems to be irrelevant to Postgres
> >> Hibernator's workings.
> >
> > Yeap, but if it crashes before writing checkpoint record, it will lead to
> > recovery which is what we are considering.
>
> Good point.
>
> In case of such recovery, the recovery process will read in the blocks
> that were recently modified, and were possibly still in shared-buffers
> when Checkpointer crashed. So after recovery finishes, the
> >> I think you are trying to argue the wording in my claim "save-files
> >> are created only on clean shutdowons; not on a crash or immediate
> >> shutdown", by implying that a crash may occur at any time during and
> >> after the BufferSaver processing. I agree the wording can be improved.
> >
> > Not only wording, but in your above mail Case 2 and 1b would need to
> > load buffer's and perform recovery as well, so we need to decide which
> > one to give preference.
>
> In the cases you mention, 1b and 2, ideally there will be no
> save-files because the server either (1b) was still running, or (2)
> crashed.
>
> If there were any save-files present during the previous startup (the
> one that happened before (1b) hot-backup or (2) crash) of the server,
> they would have been removed by the BlockReaders soon after the
> startup.
I think Block Readers will remove file only after reading and populating
> On Sun, Jun 8, 2014 at 3:24 AM, Amit Kapila <amit.kapila16@gmail.com> wrote:
> >> > IIUC on shutdown request, postmaster will send signal to BG Saver
> >> > and BG Saver will save the buffers and then postmaster will send
> >> > signal to checkpointer to shutdown. So before writing Checkpoint
> >> > record, BG Saver can crash (it might have saved half the buffers)
> >>
> >> Case handled as described above.
> >>
> >> > or may BG saver saves buffers, but checkpointer crashes (due to
> >> > power outage or any such thing).
> >>
> >> Checkpointer process' crash seems to be irrelevant to Postgres
> >> Hibernator's workings.
> >
> > Yeap, but if it crashes before writing checkpoint record, it will lead to
> > recovery which is what we are considering.
>
> Good point.
>
> In case of such recovery, the recovery process will read in the blocks
> that were recently modified, and were possibly still in shared-buffers
> when Checkpointer crashed. So after recovery finishes, the
> BlockReaders will be invoked (because save-files were successfully
> written before the crash), and they would request the same blocks to
> be restored. Most likely, those blocks would already be in
> shared-buffers, hence no cause of concern regarding BlockReaders
> evicting buffers populated by recovery.
> written before the crash), and they would request the same blocks to
> be restored. Most likely, those blocks would already be in
> shared-buffers, hence no cause of concern regarding BlockReaders
> evicting buffers populated by recovery.
Not necessarily because after crash, recovery has to start from
previous checkpoint, so it might not perform operations on same
pages as are saved by buffer saver. Also as the file saved by
buffer saver can be a file which contains only partial list of
buffers which were in shared buffer's, it becomes more likely that
in such cases it can override the buffers populated by recovery.
Now as pg_hibernator doesn't give any preference to usage_count while
saving buffer's, it can also evict the buffers populated by recovery
with some lower used pages of previous run.
> >> I think you are trying to argue the wording in my claim "save-files
> >> are created only on clean shutdowons; not on a crash or immediate
> >> shutdown", by implying that a crash may occur at any time during and
> >> after the BufferSaver processing. I agree the wording can be improved.
> >
> > Not only wording, but in your above mail Case 2 and 1b would need to
> > load buffer's and perform recovery as well, so we need to decide which
> > one to give preference.
>
> In the cases you mention, 1b and 2, ideally there will be no
> save-files because the server either (1b) was still running, or (2)
> crashed.
>
> If there were any save-files present during the previous startup (the
> one that happened before (1b) hot-backup or (2) crash) of the server,
> they would have been removed by the BlockReaders soon after the
> startup.
I think Block Readers will remove file only after reading and populating
buffers from it and that's the reason I mentioned that it can lead to doing
both recovery as well as load buffers based on file saved by buffer
saver.
pgsql-hackers by date: