repeated decoding of prepared transactions - Mailing list pgsql-hackers

From Markus Wanner
Subject repeated decoding of prepared transactions
Date
Msg-id d0f60d60-133d-bf8d-bd70-47784d8fabf3@enterprisedb.com
Whole thread Raw
Responses Re: repeated decoding of prepared transactions
Re: repeated decoding of prepared transactions
List pgsql-hackers
Amit, Ajin, hackers,

testing logical decoding for two-phase transactions, I stumbled over 
what I first thought is a bug.  But comments seems to indicate this is 
intended behavior.  Could you please clarify or elaborate on the design 
decision?  Or indicate this indeed is a bug?

What puzzled me is that if a decoder is restarted in between the PREPARE 
and the COMMIT PREPARED, it repeats the entire transaction, despite it 
being already sent and potentially prepared on the receiving side.

In terms of `pg_logical_slot_get_changes` (and roughly from the 
prepare.sql test), this looks as follows:

                         data
----------------------------------------------------
  BEGIN
  table public.test_prepared1: INSERT: id[integer]:1
  PREPARE TRANSACTION 'test_prepared#1'
(3 rows)


This is the first delivery of the transaction.  After a restart, it will 
get all of the changes again, though:


                         data
----------------------------------------------------
  BEGIN
  table public.test_prepared1: INSERT: id[integer]:1
  PREPARE TRANSACTION 'test_prepared#1'
  COMMIT PREPARED 'test_prepared#1'
(4 rows)


I did not expect this, as any receiver that wants to have decoded 2PC is 
likely supporting some kind of two-phase commits itself.  And would 
therefore prepare the transaction upon its first reception.  Potentially 
receiving it a second time would require complicated filtering on every 
prepared transaction.

Furthermore, this clearly and unnecessarily holds back the restart LSN. 
Meaning even just a single prepared transaction can block advancing the 
restart LSN.  In most cases, these are short lived.  But on the other 
hand, there may be an arbitrary amount of other transactions in between 
a PREPARE and the corresponding COMMIT PREPARED in the WAL.  Not being 
able to advance over a prepared transaction seems like a bad thing in 
such a case.

I fail to see where this repetition would ever be useful.  Is there any 
reason for the current implementation that I'm missing or can this be 
corrected?  Thanks for elaborating.

Regards

Markus



pgsql-hackers by date:

Previous
From: "Tang, Haiying"
Date:
Subject: RE: Parallel INSERT (INTO ... SELECT ...)
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: Is Recovery actually paused?