Postgres Pro
Downloads
Home   >   mailing lists

Re: checkpointer continuous flushing - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: checkpointer continuous flushing
Date
Msg-id CAA4eK1KHwp5Q3d5Q2+AiDX_oKutjr9OB8LwWPAC8=SLcFwjFCQ@mail.gmail.com
Whole thread Raw
In response to Re: checkpointer continuous flushing  (Fabien COELHO <coelho@cri.ensmp.fr>)
Responses Re: checkpointer continuous flushing  (Fabien COELHO <coelho@cri.ensmp.fr>)
List pgsql-hackers
Tree view
On Wed, Aug 19, 2015 at 12:13 PM, Fabien COELHO <coelho@cri.ensmp.fr> wrote:

Sure, I think what can help here is a testcase/'s (in form of script file
or some other form, to test this behaviour of patch) which you can write
and post here, so that others can use that to get the data and share it.

 Sure... note that I already did that on this thread, without any echo... but I can do it again...

 
Thanks.

I have tried your scripts and found some problem while using avg.py
script.
grep 'progress:' test_medium4_FW_off.out | cut -d' ' -f4 | ./avg.py --limit=10 --length=300
: No such file or directory

I didn't get chance to poke into avg.py script (the command without
avg.py works fine). Python version on the m/c, I planned to test is
Python 2.7.5.

Today while reading the first patch (checkpoint-continuous-flush-10-a),
I have given some thought to below part of patch which I would like
to share with you.

+static int

+NextBufferToWrite(

+ TableSpaceCheckpointStatus *spcStatus, int nb_spaces,

+ int *pspace, int num_to_write, int num_written)

+{

+ int space = *pspace, buf_id = -1, index;

+

+ /*

+ * Select a tablespace depending on the current overall progress.

+ *

+ * The progress ratio of each unfinished tablespace is compared to

+ * the overall progress ratio to find one with is not in advance

+ * (i.e. overall ratio > tablespace ratio,

+ *  i.e. tablespace written/to_write > overall written/to_write



Here, I think above calculation can go for toss if backend or bgwriter
starts writing buffers when checkpoint is in progress.  The tablespace
written parameter won't be able to consider the one's written by backends
or bgwriter.  Now it may not big thing to worry but I find Heikki's version
worth considering, he has not changed the overall idea of this patch, but
the calculations are somewhat simpler and hence less chance of going
wrong. 


With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Peter Eisentraut
Date:
Subject: Re: allowing wal_level change at run time
Next
From: Tom Lane
Date:
Subject: Re: PostgreSQL for VAX on NetBSD/OpenBSD