Proposal : For Auto-Prewarm. - Mailing list pgsql-hackers

From Mithun Cy
Subject Proposal : For Auto-Prewarm.
Date
Msg-id CAD__Ougw7Kc+1-cGA8GM0t+FuUHqPYR9Aow6OZX48sbH=V8BWw@mail.gmail.com
Whole thread Raw
Responses Re: Proposal : For Auto-Prewarm.  (Mithun Cy <mithun.cy@enterprisedb.com>)
Re: Proposal : For Auto-Prewarm.  (Jim Nasby <Jim.Nasby@BlueTreble.com>)
Re: Proposal : For Auto-Prewarm.  ("Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>)
Re: [HACKERS] Proposal : For Auto-Prewarm.  (Konstantin Knizhnik <k.knizhnik@postgrespro.ru>)
List pgsql-hackers
# pg_autoprewarm.

This a PostgreSQL contrib module which automatically dump all of the blocknums
present in buffer pool at the time of server shutdown(smart and fast mode only,
to be enhanced to dump at regular interval.) and load these blocks when server restarts.

Design:
------
We have created a BG Worker Auto Pre-warmer which during shutdown dumps all the
blocknum in buffer pool in sorted order.
Format of each entry is <DatabaseId,TableSpaceId,RelationId,Forknum,BlockNum>.
Auto Pre-warmer is started as soon as the postmaster is started we do not wait
for recovery to finish and database to reach a consistent state. If there is a
"dump_file" to load we start loading each block entry to buffer pool until
there is a free buffer. This way we do not replace any new blocks which was
loaded either by recovery process or querying clients. Then it waits until it receives
SIGTERM to dump the block information in buffer pool.

HOW TO USE:
-----------
Build and add the pg_autoprewarm to shared_preload_libraries. Auto Pre-warmer
process automatically do dumping of buffer pool's block info and load them when
restarted.

TO DO:
------
Add functionality to dump based on timer at regular interval.
And some cleanups.
--
Thanks and Regards
Mithun C Y

Attachment

pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: [BUG] pg_basebackup from disconnected standby fails
Next
From: Amit Kapila
Date:
Subject: Re: Speed up Clog Access by increasing CLOG buffers