Thread: Re: [HACKERS] Subtransaction commits and Hot Standby

Re: [HACKERS] Subtransaction commits and Hot Standby

From
Simon Riggs
Date:
On Thu, 2008-09-18 at 15:59 +0100, Simon Riggs wrote:
> On Tue, 2008-09-16 at 10:11 -0400, Alvaro Herrera wrote:
>
> > I wonder if the improved clog API required to mark multiple
> > transactions as committed at once would be also useful to
> > TransactionIdCommitTree which is used in regular transaction commit.
>
> I've hacked together this concept patch (WIP).
>
> Not fully tested yet, but it gives a flavour of the API rearrangements
> required for atomic clog updates. It passes make check, but that's not
> saying enough for a serious review yet. I expect to pick this up again
> next week.

I've tested this some more and am much happier with it now.
Also added README details; there are no user interface or behaviour
changes.

The patch removes the need for RecordSubTransactionCommit() which

* reduces response times of subtransaction queries because we are able
to apply these changes in batches at commit time. This requires a
batch-style API that now works atomically, so there is much change in
transam.c

* reduces the path length for visibility tests for all users viewing
concurrent subtransaction activity since we are much less likely to
waste time following a long trail to an uncommitted higher-level
transaction

* removes the need for additional WAL logging to implement
subtransaction commits for Hot Standby

So half the patch is refactoring, half rearranging of clog access
functions to support batched-access.

An early review would greatly help my work on Hot Standby. Thanks.

--
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support

Attachment

Re: [HACKERS] Subtransaction commits and Hot Standby

From
Simon Riggs
Date:
On Tue, 2008-09-23 at 22:47 +0100, Simon Riggs wrote:

> I've tested this some more and am much happier with it now.

The concept is fine, but I've found a coding bug in further testing.
Please wait now for new version before review.

--
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support


Re: [HACKERS] Subtransaction commits and Hot Standby

From
Simon Riggs
Date:
On Wed, 2008-09-24 at 13:48 +0100, Simon Riggs wrote:
> On Tue, 2008-09-23 at 22:47 +0100, Simon Riggs wrote:
>
> > I've tested this some more and am much happier with it now.
>
> The concept is fine, but I've found a coding bug in further testing.
> Please wait now for new version before review.

OK, spent long time testing various batching scenarios for this using a
custom test harness to simulate various spreads of xids in transaction
trees. All looks fine now.

The main work is done in new clog.c functions:
TransactionIdSetTreeStatus() which sets whole tree atomically by calling
TransactionIdSetPageStatus(), which in turn calls
TransactionIdSetStatusBit() for each xid status change.

TransactionIdSetPageStatus() performs locking and handles write_ok
problem, as did code it replaces. TransactionIdSetPageStatus() is called
theoretical minimum number of times for any transaction tree.

Patch slightly fumbles diff-ing new and replacement code, so there are
two chunks that appear to show I'm removing locking. I'm not!!

Everything else is just API changes.

--
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support

Attachment