Re: table versioning approach (not auditing) - Mailing list pgsql-general

From Nick Guenther
Subject Re: table versioning approach (not auditing)
Date
Msg-id 12a2d484-657d-4129-aaa0-67b4c369f640@email.android.com
Whole thread Raw
In response to Re: table versioning approach (not auditing)  (Jonathan Vanasco <postgres@2xlp.com>)
Responses Re: table versioning approach (not auditing)  (Jonathan Vanasco <postgres@2xlp.com>)
List pgsql-general

On September 29, 2014 11:08:55 AM EDT, Jonathan Vanasco <postgres@2xlp.com> wrote:
>
>- use a "transaction" log.  every write session gets logged into the
>transaction table (serial, timestamp, user_id).  all updates to the
>recorded tables include the transaction's serial.  then there is a
>"transactions" table, that is just "transaction_serial ,  object_id ,
>object_action".

A newbie tangent question: how do you access the transaction serial? Is it txid_current() as listed in
http://www.postgresql.org/docs/9.3/static/functions-info.html?

And how do you actually make use of that information? I know from Bruce Momjians's excellent MVCC talk
<http://momjian.us/main/writings/pgsql/mvcc.pdf>that postgres internally has a secret txid column on each row; can you
somehowquery on the secret column? And does your implementation worry about multiple timelines?  

My use case is dynamically allocated replication. Broadly, my algorithm is that for each client
1) download a full copy of the current table
2) keep the connection open and send deltas (which are just inserts and deletes, for me)

I need 2 to begin *as if immediately* after 1.  txids sound like they are exactly what I need but without knowing how
tohandle them, I fudged it by opening a query for 1 and for 2 immediately after each other so that they should be
pluggedto the same txid but before reading them.  There's definitely a race condition that will show under load,
though.I think the correct algorithm is: 

1) ask the current txid X
2) start buffering deltas with txid > X
3) download the table as of X
4) download the buffer of deltas and listen for future ones
--


pgsql-general by date:

Previous
From: Jonathan Vanasco
Date:
Subject: Re: table versioning approach (not auditing)
Next
From: Andy Colson
Date:
Subject: PG 9.3 Switch streaming to wal shipping