Re: [BUG] Checkpointer on hot standby runs without looking checkpoint_segments - Mailing list pgsql-hackers

This is new version of the patch.
I replaced GetStandbyFlushRecPtr with GetXLogReplayRecPtr to
check progress of checkpoint following Fujii's sugestion.

The first one is for 9.2dev, and the second is 9.1.3 backported version.

===
By the way, I took a close look around there,

> I agree with it basically. But I've get confused to look into
> GetStandbyFlushRecPtr().
> 
> |   if (XLByteLT(receivePtr, replayPtr))
> |     return XLByteLT(replayPtr, restorePtr) ? restorePtr : replayPtr;
> |   else
> |     return XLByteLT(receivePtr, restorePtr) ? restorePtr : receivePtr;
- receivePtr seems always updated just after syncing received xlog.- replayPtr is updated just BEFORE xlog_redo
operation,and- restorePtr is updated AFTER xlog_redo().- And, replayPtr seems not exceeds receivePtr.
 

These seems quite reasonable. These conditions make following
conditional expression.
    restorePtr <= replayPtr <= receivePtr

But XLByteLT(recievePtr, replayPtr) this should not return true
under the condition above.. Something wrong in my assumption?


Anyway, I understand here that you say the location returned by
GetXLogReplayRecPtr() is always flushed.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

== My e-mail address has been changed since Apr. 1, 2012.
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index c9473f7..c2fafbf 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -491,8 +491,8 @@ CheckpointerMain(void)             * Initialize checkpointer-private variables used during
checkpoint.            */            ckpt_active = true;
 
-            if (!do_restartpoint)
-                ckpt_start_recptr = GetInsertRecPtr();
+            ckpt_start_recptr =
+                do_restartpoint ? GetXLogReplayRecPtr(NULL) : GetInsertRecPtr();            ckpt_start_time = now;
      ckpt_cached_elapsed = 0;
 
@@ -731,28 +731,30 @@ IsCheckpointOnSchedule(double progress)        return false;    /*
-     * Check progress against WAL segments written and checkpoint_segments.
+     * Check progress against WAL segments written, or replayed for
+     * hot standby, and checkpoint_segments.     *     * We compare the current WAL insert location against the
location
-     * computed before calling CreateCheckPoint. The code in XLogInsert that
-     * actually triggers a checkpoint when checkpoint_segments is exceeded
-     * compares against RedoRecptr, so this is not completely accurate.
-     * However, it's good enough for our purposes, we're only calculating an
-     * estimate anyway.
+     * computed before calling CreateCheckPoint. The code in
+     * XLogInsert that actually triggers a checkpoint when
+     * checkpoint_segments is exceeded compares against RedoRecPtr.
+     * Similarly, we consult WAL replay location instead on hot
+     * standbys and XLogPageRead compares it aganst RedoRecPtr, too.
+     * Altough these are not completely accurate, it's good enough for
+     * our purposes, we're only calculating an estimate anyway.     */
-    if (!RecoveryInProgress())
+    
+    recptr =
+        RecoveryInProgress() ? GetXLogReplayRecPtr(NULL) : GetInsertRecPtr();
+    elapsed_xlogs =
+        (((double) (int32) (recptr.xlogid - ckpt_start_recptr.xlogid)) * XLogSegsPerFile +
+         ((double) recptr.xrecoff - (double) ckpt_start_recptr.xrecoff) / XLogSegSize) /
+        CheckPointSegments;
+
+    if (progress < elapsed_xlogs)    {
-        recptr = GetInsertRecPtr();
-        elapsed_xlogs =
-            (((double) (int32) (recptr.xlogid - ckpt_start_recptr.xlogid)) * XLogSegsPerFile +
-             ((double) recptr.xrecoff - (double) ckpt_start_recptr.xrecoff) / XLogSegSize) /
-            CheckPointSegments;
-
-        if (progress < elapsed_xlogs)
-        {
-            ckpt_cached_elapsed = elapsed_xlogs;
-            return false;
-        }
+        ckpt_cached_elapsed = elapsed_xlogs;
+        return false;    }    /*
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 5643ec8..a272866 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -56,6 +56,7 @@#include "pgstat.h"#include "postmaster/bgwriter.h"#include "replication/syncrep.h"
+#include "replication/walreceiver.h"#include "storage/bufmgr.h"#include "storage/fd.h"#include "storage/ipc.h"
@@ -489,8 +490,8 @@ BackgroundWriterMain(void)             * Initialize bgwriter-private variables used during
checkpoint.            */            ckpt_active = true;
 
-            if (!do_restartpoint)
-                ckpt_start_recptr = GetInsertRecPtr();
+            ckpt_start_recptr =    do_restartpoint ?
+                GetXLogReplayRecPtr() : GetInsertRecPtr();            ckpt_start_time = now;
ckpt_cached_elapsed= 0;
 
@@ -764,30 +765,32 @@ IsCheckpointOnSchedule(double progress)        return false;    /*
-     * Check progress against WAL segments written and checkpoint_segments.
+     * Check progress against WAL segments written, or replayed for
+     * hot standby, and checkpoint_segments.     *     * We compare the current WAL insert location against the
location
-     * computed before calling CreateCheckPoint. The code in XLogInsert that
-     * actually triggers a checkpoint when checkpoint_segments is exceeded
-     * compares against RedoRecptr, so this is not completely accurate.
-     * However, it's good enough for our purposes, we're only calculating an
-     * estimate anyway.
+     * computed before calling CreateCheckPoint. The code in
+     * XLogInsert that actually triggers a checkpoint when
+     * checkpoint_segments is exceeded compares against RedoRecPtr.
+     * Similarly, we consult WAL replay location instead on hot
+     * standbys and XLogPageRead compares it aganst RedoRecPtr, too.
+     * Altough these are not completely accurate, it's good enough for
+     * our purposes, we're only calculating an estimate anyway.     */
-    if (!RecoveryInProgress())
+    
+    recptr =
+        RecoveryInProgress() ? GetXLogReplayRecPtr() : GetInsertRecPtr();
+    elapsed_xlogs =
+        (((double) (int32) (recptr.xlogid - ckpt_start_recptr.xlogid)) * XLogSegsPerFile +
+         ((double) recptr.xrecoff - (double) ckpt_start_recptr.xrecoff) / XLogSegSize) /
+        CheckPointSegments;
+    
+    if (progress < elapsed_xlogs)    {
-        recptr = GetInsertRecPtr();
-        elapsed_xlogs =
-            (((double) (int32) (recptr.xlogid - ckpt_start_recptr.xlogid)) * XLogSegsPerFile +
-             ((double) recptr.xrecoff - (double) ckpt_start_recptr.xrecoff) / XLogSegSize) /
-            CheckPointSegments;
-
-        if (progress < elapsed_xlogs)
-        {
-            ckpt_cached_elapsed = elapsed_xlogs;
-            return false;
-        }
+        ckpt_cached_elapsed = elapsed_xlogs;
+        return false;    }
-
+        /*     * Check progress against time elapsed and checkpoint_timeout.     */