Re: pg_receivexlog and feedback message - Mailing list pgsql-hackers

On Thu, Jun 7, 2012 at 5:05 AM, Magnus Hagander <magnus@hagander.net> wrote:
> On Wed, Jun 6, 2012 at 8:26 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
>> On Tue, Jun 5, 2012 at 11:44 PM, Magnus Hagander <magnus@hagander.net> wrote:
>>> On Tue, Jun 5, 2012 at 4:42 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
>>>> On Tue, Jun 5, 2012 at 9:53 PM, Magnus Hagander <magnus@hagander.net> wrote:
>>>>> Right now, pg_receivexlog sets:
>>>>>                        replymsg->write = InvalidXLogRecPtr;
>>>>>                        replymsg->flush = InvalidXLogRecPtr;
>>>>>                        replymsg->apply = InvalidXLogRecPtr;
>>>>>
>>>>> when it sends it's status updates.
>>>>>
>>>>> I'm thinking it sohuld set replymsg->write = blockpos instad.
>>>>>
>>>>> Why? That way you can see in pg_stat_replication what has actually
>>>>> been received by pg_receivexlog - not just what we last sent. This can
>>>>> be useful in combination with an archive_command that can block WAL
>>>>> recycling until it has been saved to the standby. And it would be
>>>>> useful as a general monitoring thing as well.
>>>>>
>>>>> I think the original reason was that it shouldn't interefer with
>>>>> synchronous replication - but it does take away a fairly useful
>>>>> usecase...
>>>>
>>>> I think that not only replaymsg->write but also ->flush should be set to
>>>> blockpos in pg_receivexlog. Which allows pg_receivexlog to behave
>>>> as synchronous standby, so we can write WAL to both local and remote
>>>> synchronously. I believe there are some use cases for synchronous
>>>> pg_receivexlog.
>>>
>>> pg_receivexlog doesn't currently fsync() after every write. It only
>>> fsync():s complete files. So we'd need to set ->flush only at the end
>>> of a segment, right?
>>
>> Yes.
>>
>> Currently the status update is sent for each status interval. In sync
>> replication, transaction has to wait for a while even after pg_receivexlog
>> has written or flushed the WAL data.
>>
>> So we should add new option which specifies whether pg_receivexlog
>> sends the status packet back as soon as it writes or flushes the WAL
>> data, like the walreceiver does?
>
> That might be useful, but I think that's 9.3 material at this point.

Fair enough. That's new feature rather than a bugfix.

> But I think we can get the "set the write location" in as a bugfix.

Also "set the flush location"? Sending the flush location back seems
helpful when using pg_receivexlog for WAL archiving purpose. By
seeing the flush location we can ensure that WAL file has been archived
durably (IOW, WAL file has been flushed in remote archive area).