Re: An improvement of ProcessTwoPhaseBuffer logic - Mailing list pgsql-hackers
From | Vitaly Davydov |
---|---|
Subject | Re: An improvement of ProcessTwoPhaseBuffer logic |
Date | |
Msg-id | 96951-679a1880-17-e255290@36905887 Whole thread Raw |
In response to | An improvement of ProcessTwoPhaseBuffer logic ("Vitaly Davydov" <v.davydov@postgrespro.ru>) |
List | pgsql-hackers |
It seems, there are much deeper problems with twophase transactions as I thought. I'm interested in fixing twophase transactions,because I support a solution which actively uses twophase transactions. I'm interested to get more deeply intothe twophase functionality. Below, I just want to clarify some ideas behind twophase transactions. I appreciate, if youcomment my point of view, or just ignore this email if you find it too naive and boring. Two phase files are created after checkpoint to keep twophase states on disk after WAL truncation. For transactions, thatare inside the checkpoint horizon, we do not create two phase files because the states are currently stored in the WAL. Based on the thesis above, I guess, we have to read only those twophase files which are related to the transactions beforethe latest checkpoint. Its full xid should be lesser than TransamVariables->nextXid (which is the same as ControlFile->checkPointCopy.nextXidat the moment of StartupXLOG -> restoreTwoPhaseData call). The files with greater (orequal) full xids, should be ignored and removed. That's all what we need in restoreTwoPhaseData, I believe. In the current implementation, such check is applied in restoreTwoPhaseData -> ProcessTwoPhaseBuffer but after checking forxid in CLOG. I'm not sure, why we check CLOG here. Once we truncate the WAL on checkpoint and save twophase states intopg_twophase, these files must store states of real transactions from past. I mean, if someone creates a stub file withfull xid < TransamVariables->nextXid, we have no means (except CLOG ?) to check that this file belongs to a real transactionfrom past. CLOG check seems to be a weak attempt to deal with it. At this point, I'm not sure that CLOG may containstates for all full xids of existing twophase files. I guess, we should call restoreTwoPhaseData at start of recoverybut we shouldn't check CLOG at this stage. May be it is reasonable to check some not so old xids which are greaterthan the current CLOG horizon, but I'm not sure how CLOG segments are managed and how the horizon is moving. There is another question about the loading order of twophase files but I think it doesn't matter in which order we loadthese files. But I would prefer to load it in full xid ascending order. On Tuesday, January 28, 2025 08:02 MSK, Michael Paquier <michael@paquier.xyz> wrote: > Noah's > proposal at [1] is much closer to the long-term picture that would > look adapted. > - The CLOG lookups that can happen in ProcessTwoPhaseBuffer() during > recovery while a consistent state is not reached are still possible > (planning to start a different thread about this specific issue). > > [1]: https://www.postgresql.org/message-id/20250116205254.65.nmisch@google.com Agree, thank you, but my simple patch with some adjustments and swapping of checks in ProcessTwoPhaseBuffer may be back-ported.It doesn't fix all the problems but may help to fix the problem with twophase files related to broken latestWAL segments. With best regards, Vitaly
pgsql-hackers by date: