Thread: Re: doc: Mention clock synchronization recommendation for hot_standby_feedback
Re: doc: Mention clock synchronization recommendation for hot_standby_feedback
From
"Euler Taveira"
Date:
On Thu, Dec 5, 2024, at 6:43 AM, Jakub Wartak wrote:
One of our customers ran into a very odd case, where hot standby feedback backend_xmin propagation stopped working due to major (hours/days) clock time shifts on hypervisor-managed VMs. This happens (and is fully reproducible) e.g. in scenarios where standby connects and its own VM is having time from the future (relative to primary) and then that time goes back to "normal". In such situation "sends hot_standby_feedback xmin" timestamp messages are stopped being transferred, e.g.:
Is it worth a WARNING message if there is a "huge" time difference
between the servers? We already have the reply time in the message so
it is a matter of defining the "huge" interval plus a roundtrip. We also
need to avoid spamming the log.
Your patch looks good to me. Should it be converted into a
<note>...</note>? (See synchronous_standby_names [1] for an example.)
Re: doc: Mention clock synchronization recommendation for hot_standby_feedback
From
Andres Freund
Date:
On 2024-12-05 12:06:41 -0300, Euler Taveira wrote: > Is it worth a WARNING message if there is a "huge" time difference > between the servers? We already have the reply time in the message so > it is a matter of defining the "huge" interval plus a roundtrip. We also > need to avoid spamming the log. IME folks who have huge time differences between the servers are not going to look at the log carefully enough to see such a warning.
Re: doc: Mention clock synchronization recommendation for hot_standby_feedback
From
Jakub Wartak
Date:
Hi Euler!,
On Thu, Dec 5, 2024 at 4:07 PM Euler Taveira <euler@eulerto.com> wrote:
On Thu, Dec 5, 2024, at 6:43 AM, Jakub Wartak wrote:One of our customers ran into a very odd case, where hot standby feedback backend_xmin propagation stopped working due to major (hours/days) clock time shifts on hypervisor-managed VMs. This happens (and is fully reproducible) e.g. in scenarios where standby connects and its own VM is having time from the future (relative to primary) and then that time goes back to "normal". In such situation "sends hot_standby_feedback xmin" timestamp messages are stopped being transferred, e.g.:Is it worth a WARNING message if there is a "huge" time differencebetween the servers? We already have the reply time in the message soit is a matter of defining the "huge" interval plus a roundtrip. We alsoneed to avoid spamming the log.
I'm trying to stay consistent with what the recovery_min_apply_delay did (there is a warning in the docs, but there's no warning in code) and I just wanted the to have pointer in the documentation that if someone is using hot_standby_feedback then he would be at least warned before. Given it is very rare I don't think we need additional code (+ what Andres has noted ).
Your patch looks good to me. Should it be converted into a<note>...</note>? (See synchronous_standby_names [1] for an example.)
Fine for me, but we would have to also convert the recovery_min_apply_delay to do the same, right?