Re: pg_basebackup blocking all queries with horrible performance - Mailing list pgsql-admin

From Fujii Masao
Subject Re: pg_basebackup blocking all queries with horrible performance
Date
Msg-id CAHGQGwHaLwySPXNb+vcEL2Vay1UN36b8NkURLwV54d_4QriHzA@mail.gmail.com
Whole thread Raw
In response to Re: pg_basebackup blocking all queries with horrible performance  (Magnus Hagander <magnus@hagander.net>)
Responses Re: pg_basebackup blocking all queries with horrible performance  (Fujii Masao <masao.fujii@gmail.com>)
Re: pg_basebackup blocking all queries with horrible performance  (Magnus Hagander <magnus@hagander.net>)
List pgsql-admin
On Sun, Jun 10, 2012 at 7:43 PM, Magnus Hagander <magnus@hagander.net> wrote:
> On Sat, Jun 9, 2012 at 2:51 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Fujii Masao <masao.fujii@gmail.com> writes:
>>> This seems a bug. I think we should prevent pg_basebackup from
>>> becoming synchronous standby. Thought?
>>
>> Absolutely.  If we have replication clients that are not actually
>> capable of being standbys, there *must* be a way for the master
>> to know that.
>
> I thought we fixed this already by sending InvalidXlogRecPtr as flush
> location? And that this only applied in 9.2?
>
> Are you saying we picked pg_basebackup *in backup mode* (not log
> streaming) as synchronous standby?

Yes.

> If so then yes, that is
> *definitely* a bug that should be fixed. We should never select a
> connection that's not even streaming log as standby!

Agreed. Attached patch prevents pg_basebackup from becoming sync
standby. Also this patch fixes another problem: currently only walsender
which reaches STREAMING state can become sync walsender. OTOH,
sync walsender thinks that walsender with higher priority will be sync one
whether its state is STREAMING, and switches to potential sync walsender.
So when the standby with higher priority connects to the master, we
might have no sync standby until it reaches the STREAMING state.
To fix this problem, the patch switches walsender's state from sync to
potential *after* walsender with higher priority has reached the
STREAMING state.

We also should not select (1) background stream process forked from
pg_basebackup and (2) pg_receivexlog as sync standby because they
don't send back replication progress. To address this, I'm thinking to
introduce new option "NOSYNC" in "START_REPLICATION" command
as follows, and to change (1) and (2) so that they specify NOSYNC.

    START_REPLICATION XXX/XXX [NOSYNC]

If the standby specifies NOSYNC option, it's never assigned as sync
standby even if its name is in synchronous_standby_names. Thought?

BTW, we are discussing about changing pg_receivexlog so that it sends
back replication progress, in another thread. So if this change will have
been applied, probably we don't need to change pg_receivexlog so that
it uses NOSYNC option.

Regards,

--
Fujii Masao

Attachment

pgsql-admin by date:

Previous
From: Magnus Hagander
Date:
Subject: Re: pg_basebackup blocking all queries with horrible performance
Next
From: Fujii Masao
Date:
Subject: Re: pg_basebackup blocking all queries with horrible performance