Re: pg_basebackup may fail to send feedbacks. - Mailing list pgsql-hackers

On Fri, Feb 6, 2015 at 3:22 PM, Kyotaro HORIGUCHI
<horiguchi.kyotaro@lab.ntt.co.jp> wrote:
> Sorry, I misunderstood that.
>
>> > At Wed, 4 Feb 2015 19:22:39 +0900, Fujii Masao <masao.fujii@gmail.com> wrote in
<CAHGQGwGudGCMnHZinkd37i+JijDkruEcrea1NCRs1MMtE3rOFQ@mail.gmail.com>
>> >> On Wed, Feb 4, 2015 at 4:58 PM, Kyotaro HORIGUCHI
>> >> <horiguchi.kyotaro@lab.ntt.co.jp> wrote:
>> >> > I'm very sorry for confused report. The problem found in 9.4.0
>> >> > and the diagnosis was mistakenly done on master.
>> >> >
>> >> > 9.4.0 has no problem of feedback delay caused by slow xlog
>> >> > receiving on pg_basebackup mentioned in the previous mail. But
>> >> > the current master still has this problem.
>> >>
>> >> Seems walreceiver has the same problem. No?
>> >
>> > pg_receivexlog.c would have the same problem since it uses the
>> > same function with pg_basebackup.c.
>> >
>> > The correspondent of HandleCopyStream in wansender is
>> > WalReceiverMain, and it doesn't seem to have the same kind of
>> > loop shown below. It seems to surely send feedback per one
>> > record.
>> >
>> > |   r = stream_reader();
>> > |   while (r > 0)
>> > |   {
>> > |      ... wal record processing stuff without sending feedback..
>> > |      r = stream_reader();
>> > |   }
>>
>> WalReceiverMain() has the similar code as follows.
>>
>>     len = walrcv_receive(NAPTIME_PER_CYCLE, &buf);
>>     if (len != 0)
>>     {
>>         for (;;)
>>         {
>>             if (len > 0)
>>             {
>>                 ....
>>                 len = walrcv_receive(0, &buf);
>>             }
>>     }
>
> The loop seems a bit different but eventually the same about this
> discussion.
>
> 408>     len = walrcv_receive(NAPTIME_PER_CYCLE, &buf);
> 409>     if (len != 0)
> 410>     {
> 415>         for (;;)
> 416>         {
> 417>             if (len > 0)
> 418>             {
> 425>                XLogWalRcvProcessMsg(buf[0], &buf[1], len - 1);
> 426>             }
> 427-438>         else {break;}
> 439>             len = walrcv_receive(0, &buf);
> 440>         }
> 441>     }
>
> XLogWalRcvProcessMsg doesn't send feedback when processing
> 'w'=WAL record packet. So the same thing as pg_basebackup and
> pg_receivexlog will occur on walsender.
>
> Exiting the for(;;) loop by TimestampDifferenceExceeds just
> before line 439 is an equivalent measure to I poposed for
> receivelog.c, but calling XLogWalRcvSendHSFeedback(false) there
> is seemingly simpler (but I feel a bit uncomfortable for the
> latter)

I'm concerned about the performance degradation by calling
getimeofday() per a record.

> Or other measures?

Introduce the maximum number of records that we can receive and
process between feedbacks? For example, we can change
XLogWalRcvSendHSFeedback() so that it's called per at least
8 records. I'm not sure what number is good, though...

Regards,

-- 
Fujii Masao