Expanding on Amit's idea, we found out that there is already a mechanism in code to fully decode prepared transactions prior to a defined LSN where two_phase is enabled using the "two_phase_at" LSN in the slot. Look at ReorderBufferFinishPrepared() on how this is done. This code was not working as expected in our patch because
we were setting two_phase on the slot to true as soon as the alter command was received. This was not the correct way, initially when two_phase is enabled, the two_phase changes to pending state and two_phase option on the slot should only be set to true when two_phase moves from pending to enabled. This will happen once the replication is restarted with two_phase option. Look at code in CreateDecodingContext() on how "two_phase_at" is set in the slot when done this way. So we changed the code to not remotely alter two_phase when toggling from false to true. With this change, now even if there are pending transactions on the publisher when toggling two_phase from false to true, these pending transactions will be fully decoded and sent once the commit prepared is decoded as the pending prepared transactions are prior to the "two_phase_at" LSN. With this patch, now we are able to handle both pending prepared transactions when altering two_phase from true to false as well as false to true.
Attaching the patch for your review and comments. Big thanks to Kuroda-san for also working on the patch.
