Re: Automatic notification of top transaction IDs - Mailing list pgsql-hackers

The proposed patch is attached.

Best regards,
--
Gurjeet Singh http://gurjeet.singh.im/

On Wed, Jun 30, 2021 at 5:56 PM Gurjeet Singh <gurjeet@singh.im> wrote:
>
> (Re-sending this email, because the Commitfest app mistakenly [3]
> considered previous email [4] to be part of the old thread, whereas it
> should not be considered that way)
>
> I came across this thread [1] to disallow canceling a transaction not
> yet confirmed by a synchronous replica. I think my proposed patch
> might help that case as well, hence adding all involved in that thread
> to BCC, for one-time notification.
>
> As mentioned in that thread, when sending a cancellation signal, the
> client cannot be sure if the cancel signal was honored, and if the
> transaction was cancelled successfully. In the attached patch, the
> backend emits a NotificationResponse containing the current full
> transaction id. It does so only if the relevant GUC is enabled, and
> when the top-transaction is being assigned the ID.
>
> This information can be useful to the client, when:
> i) it wants to cancel a transaction _after_ issuing a COMMIT, and
> ii) it wants to check the status of its transaction that it sent
> COMMIT for, but never received a response (perhaps because the server
> crashed).
>
> Additionally, this information can be useful for middleware, like
> Transaction Processing Monitors, which can now transparently (without
> any change in application code) monitor the status of transactions (by
> watching for the transaction status indicator in the ReadyForQuery
> protocol message). They can use the transaction ID from the
> NotificationResponse to open a watcher, and on seeing either an 'E' or
> 'I' payload in subsequent ReadyForQuery messages, close the watcher.
> On server crash, or other adverse events, they can then use the
> transaction IDs still being watched to check status of those
> transactions, and take appropriate actions, e.g. retry any aborted
> transactions.
>
> We cannot use the elog() mechanism for this notification because it is
> sensitive to the value of client_min_messages. Hence I used the NOTIFY
> infrastructure for this message. I understand that this usage violates
> some expectations as to how NOTIFY messages are supposed to behave
> (see [2] below), but I think these are acceptable violations; open to
> hearing if/why this might not be acceptable, and any possible
> alternatives.
>
> I'm not very familiar with the parallel workers infrastructure, so the
> patch is missing any consideration for those.
>
> Reviews welcome.
>
> [1]: subject was: Re: Disallow cancellation of waiting for synchronous
> replication
> thread: https://www.postgresql.org/message-id/flat/C1F7905E-5DB2-497D-ABCC-E14D4DEE506C%40yandex-team.ru
>
> [2]:
>      At present, NotificationResponse can only be sent outside a
>      transaction, and thus it will not occur in the middle of a
>      command-response series, though it might occur just before ReadyForQuery.
>      It is unwise to design frontend logic that assumes that, however.
>      Good practice is to be able to accept NotificationResponse at any
>      point in the protocol.
>
> [3]:
>
> See Emails section in https://commitfest.postgresql.org/33/3198/
>
> The email [4] is considered a continuation of a previous thread, _and_
> the 'Latest attachment' entry points to a different email, even though
> my email [4] contained a patch.
>
> [4]: https://www.postgresql.org/message-id/CABwTF4VS+HVm11XRE_Yv0vGmG=5kpYdx759RyJEp9F+fiLTU=Q@mail.gmail.com
>
> Best regards,
> --
> Gurjeet Singh http://gurjeet.singh.im/