Re: pg_receivexlog add synchronous mode - Mailing list pgsql-hackers

From Fujii Masao
Subject Re: pg_receivexlog add synchronous mode
Date
Msg-id CAHGQGwFTg7idUnrmwdO-u98kvs2EhXWcBsbk8DnCY+2s7jDVYw@mail.gmail.com
Whole thread Raw
In response to Re: pg_receivexlog add synchronous mode  (<furuyao@pm.nttdata.co.jp>)
Responses Re: pg_receivexlog add synchronous mode  (<furuyao@pm.nttdata.co.jp>)
List pgsql-hackers
On Tue, Jun 10, 2014 at 5:01 PM,  <furuyao@pm.nttdata.co.jp> wrote:
>> No. IIUC walreceiver does flush *less* frequently than what you
>> implemented on pg_receivexlog. Your version of pg_receivexlog tries to
>> do flush every time when it receives one WAL chunk. OTOH, walreceiver
>> does flush only when there is no extra WAL chunk in receive buffer. IOW,
>> after writing WAL chunk, if there is another WAL chunk that walreceiver
>> can receive immediately, it postpones flush later.
>>
>> > However, it seems difficult to apply as same way.
>>
>> Why? ISTM that's not so difficult.
>
> I was not able to understand movement of walreceiver well.
> While walreceiver writes data, do PQconsumeInput() by omitting the select().
> Do flush if the PQgetCopyData has been to return the zero continuously.
> Fixed to the same process using the flag.

You introduced the state machine using the flag "flush_flg" into pg_receivexlog.
That's complicated and would reduce the readability of the source code. I think
that the logic should be simpler like walreceiver's one.

Maybe I found one problematic path as follows:

1. WAL is written and flush_flag is set to 1
2. PQgetCopyData() returns 0 and flush_flg is incremented to 2
3. PQconsumeInput() is executed
4. PQgetCopyData() reads keepalive message
5. After processing keepalive message, PQgetCopyDate() returns 0
6. Since flush_flg is 2, WAL is flushed and flush_flg is reset to 0

But new message can arrive while processing keepalive message. Before
flushing WAL, such new message should be processed.

+        Enables synchronous mode. If replication slot is disabled then
+        this setting is irrelevant.

Why is that irrelevant in that case?

Even when replication slot is not used, thanks to this feature, pg_receivexlog
can flush WAL more proactively and which may improve the durability of WAL
which pg_receivexlog writes.

+    printf(_("  -m, --sync-mode        synchronous mode\n"));

I think that calling this feature "synchronous mode" is confusing.

Regards,

-- 
Fujii Masao



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Re: replication commands and log_statements
Next
From: Kevin Grittner
Date:
Subject: Re: Inaccuracy in VACUUM's tuple count estimates