Re: [HACKERS] Hooks to track changed pages for backup purposes - Mailing list pgsql-hackers

From Andrey Borodin
Subject Re: [HACKERS] Hooks to track changed pages for backup purposes
Date
Msg-id 1E325A48-80D5-4C71-B620-33C8D2EE6868@yandex-team.ru
Whole thread Raw
In response to Re: [HACKERS] Hooks to track changed pages for backup purposes  (Ants Aasma <ants.aasma@eesti.ee>)
List pgsql-hackers
Hi!
Thank you for your interest and experiment results.
> 13 сент. 2017 г., в 15:43, Ants Aasma <ants.aasma@eesti.ee> написал(а):
>
> On Thu, Aug 31, 2017 at 9:02 AM, Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>> When we have accumulated diff blocknumbers for most of segments we can significantly speed up method of WAL
scanning.If we have blocknumbers for all segments we can skip WAL scanning at all. 
>
> Have you measured that the WAL scanning is actually a significant
> issue? As a quick experiment I hacked up pg_waldump to just dump block
> references to stdout in binary format. It scanned 2.8GB of WAL in 3.17
> seconds, outputting 9.3M block refs per second. WAL was generated with
> pgbench, synchronous commit off, using 4 cores for 10 minutes - making
> the ratio of work from generating WAL to parsing it be about 750:1.
>

No, I had not done this measurement myself. Sure, parsing WAL, when it is in RAM, is not very expensive. Though, it can
beeven cheaper before formatting WAL. 
I just want to figure out what is the best place for this, if backuping exec is sharing CPUs with postmaster.

Best regards, Andrey Borodin.



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

pgsql-hackers by date:

Previous
From: Prabhat Sahu
Date:
Subject: Re: [HACKERS] Parallel Hash take II
Next
From: Peter Moser
Date:
Subject: Re: [HACKERS] [PROPOSAL] Temporal query processing with range types