Re: POC: Cleaning up orphaned files using undo logs - Mailing list pgsql-hackers

From Antonin Houska
Subject Re: POC: Cleaning up orphaned files using undo logs
Date
Msg-id 70410.1632115633@antos
Whole thread Raw
In response to Re: POC: Cleaning up orphaned files using undo logs  (Amit Kapila <amit.kapila16@gmail.com>)
Responses Re: POC: Cleaning up orphaned files using undo logs
List pgsql-hackers
Amit Kapila <amit.kapila16@gmail.com> wrote:

> On Thu, Sep 9, 2021 at 8:33 PM Antonin Houska <ah@cybertec.at> wrote:

> > The problem of the temporary undo log is that it's loaded into local buffers
> > and that backend can exit w/o flushing local buffers to disk, and thus we are
> > not guaranteed to find enough information when trying to discard the undo log
> > the backend wrote. I'm thinking about the following solutions:
> >
> > 1. Let the backend manage temporary undo log on its own (even the slot
> >    metadata would stay outside the shared memory, and in particular the
> >    insertion pointer could start from 1 for each session) and remove the
> >    segment files at the same moment the temporary relations are removed.
> >
> >    However, by moving the temporary undo slots away from the shared memory,
> >    computation of oldestFullXidHavingUndo (see the PROC_HDR structure) would
> >    be affected. It might seem that a transaction which only writes undo log
> >    for temporary relations does not need to affect oldestFullXidHavingUndo,
> >    but it needs to be analyzed thoroughly. Since oldestFullXidHavingUndo
> >    prevents transactions to be truncated from the CLOG too early, I wonder if
> >    the following is possible (This scenario is only applicable to the zheap
> >    storage engine [1], which is not included in this patch, but should already
> >    be considered.):
> >
> >    A transaction creates a temporary table, does some (many) changes and then
> >    gets rolled back. The undo records are being applied and it takes some
> >    time. Since XID of the transaction did not affect oldestFullXidHavingUndo,
> >    the XID can disappear from the CLOG due to truncation.
> >
>
> By above do you mean to say that in zheap code, we don't consider XIDs
> that operate on temp table/undo for oldestFullXidHavingUndo?

I was referring to the code

        /* We can't process temporary undo logs. */
        if (log->meta.persistence == UNDO_TEMP)
            continue;

in undodiscard.c:UndoDiscard().

>
> > However zundo.c in
> >    [1] indicates that the transaction status *is* checked during undo
> >    execution, so we might have a problem.
> >
>
> It would be easier to follow if you can tell which exact code are you
> referring here?

In meant the call of TransactionIdDidCommit() in
zundo.c:zheap_exec_pending_rollback().

--
Antonin Houska
Web: https://www.cybertec-postgresql.com



pgsql-hackers by date:

Previous
From: Corey Huinker
Date:
Subject: Re: WIP: System Versioned Temporal Table
Next
From: Amul Sul
Date:
Subject: Re: Deduplicate code updating ControleFile's DBState.