Thread: [HACKERS] Why forcing Hot_standby_feedback to be enabled when creating alogical decoding slot on standby
[HACKERS] Why forcing Hot_standby_feedback to be enabled when creating alogical decoding slot on standby
From
sanyam jain
Date:
Hi,
I have created a logical decoding slot on a standby but i haven't enabled Hot_standby_feedback.What are the test cases where this setup will fail?
Thanks,
Sanyam Jain
Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled whencreating a logical decoding slot on standby
From
Michael Paquier
Date:
On Mon, Jun 12, 2017 at 3:16 PM, sanyam jain <sanyamjain22@live.in> wrote: > I have created a logical decoding slot on a standby but i haven't enabled > Hot_standby_feedback.What are the test cases where this setup will fail? hot_standby_feedback needs to be enabled at all times in logical decoding so as the node does not remove rows that are still needed for the decoding, and a VACUUM passing by with a minimal xmin too high would cause inconsistent decoded data. -- Michael
Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled whencreating a logical decoding slot on standby
From
Andres Freund
Date:
Hi, On 2017-06-12 06:16:50 +0000, sanyam jain wrote: > I have created a logical decoding slot on a standby but i haven't enabled Hot_standby_feedback.What are the test caseswhere this setup will fail? Creating logic slots on the standby is not supported... It'll simply currently not work correctly, even if you circumvent the protections (like not being able to create one on a standby). - Andres
Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled whencreating a logical decoding slot on standby
From
sanyam jain
Date:
Isn't XLogRecord carries full information to be decoded in itself?If so a VACCUM should not be a problem in decoding?
Thanks
Sanyam Jain
From: Michael Paquier <michael.paquier@gmail.com>
Sent: Monday, June 12, 2017 6:52:06 AM
To: sanyam jain
Cc: Pg Hackers
Subject: Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled when creating a logical decoding slot on standby
Sent: Monday, June 12, 2017 6:52:06 AM
To: sanyam jain
Cc: Pg Hackers
Subject: Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled when creating a logical decoding slot on standby
On Mon, Jun 12, 2017 at 3:16 PM, sanyam jain <sanyamjain22@live.in> wrote:
> I have created a logical decoding slot on a standby but i haven't enabled
> Hot_standby_feedback.What are the test cases where this setup will fail?
hot_standby_feedback needs to be enabled at all times in logical
decoding so as the node does not remove rows that are still needed for
the decoding, and a VACUUM passing by with a minimal xmin too high
would cause inconsistent decoded data.
--
Michael
> I have created a logical decoding slot on a standby but i haven't enabled
> Hot_standby_feedback.What are the test cases where this setup will fail?
hot_standby_feedback needs to be enabled at all times in logical
decoding so as the node does not remove rows that are still needed for
the decoding, and a VACUUM passing by with a minimal xmin too high
would cause inconsistent decoded data.
--
Michael
Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled whencreating a logical decoding slot on standby
From
Andres Freund
Date:
Hi, On 2017-06-16 06:31:03 +0000, sanyam jain wrote: > Isn't XLogRecord carries full information to be decoded in itself?If so a VACCUM should not be a problem in decoding? First: Please don't full quote emails. Secondly: You've not actually explained what you want to do, nor what your precise question itself is. Nor why Michael or my answer wasn't sufficient. Logicla decoding looks at the catalogs for metadata - otherwise the WAL would need to contain a lot more information and thus be more voluminous - so the records do *not* contain the "full information". Please start at the beginning and explain what you're trying to do (since you apparently did some hacking to get a slot on a standby) and what you're trying to achieve, instead of one-sentence questions. - Andres
Re: [HACKERS] Why forcing Hot_standby_feedback to be enabled whencreating a logical decoding slot on standby
From
sanyam jain
Date:
Hi,
I am trying to create logical decoding slot on a standby.
Enabling Hot_standby_feedback on standby is one of the requirement ; which will delay vacuuming of stale data on Master server.
I am working on a hack to avoid Hot_standby_feedback so that Standby doesn't have any dependency on Master (except receiving WAL).
Hot_standby_feedback restricts Master to do early vacuuming of catalog relation which will be in decoding WAL record using "time travelling snapshot" on a Standby.
The other way to prevent early vacuuming on Standby can be by pausing recovery on Standby when a vacuum record belonging to catalog relation is encountered.
And when walsender process belonging to logical slot on Standby reaches this record it will resume the recovery by executing SetRecoveryPause(false).
To check whether VACUUM record belongs to a catalog relation i simply check if relationID < 10000.
This hack will only work for a single logical slot on standby.
Pausing recovery will increase the size of pg_xlog directory as walreceiver will continue receiving WAL.
Querying standby might result in wrong output.
Sanyam Jain