Re: Allow "snapshot too old" error, to prevent bloat - Mailing list pgsql-hackers

From Greg Stark
Subject Re: Allow "snapshot too old" error, to prevent bloat
Date
Msg-id CAM-w4HMfDgPGEB77jU=R1m0orPAk4bNNqpqXGQ3Rk+steq0=fw@mail.gmail.com
Whole thread Raw
In response to Re: Allow "snapshot too old" error, to prevent bloat  (Tom Lane <tgl@sss.pgh.pa.us>)
Responses Re: Allow "snapshot too old" error, to prevent bloat  (Kevin Grittner <kgrittn@ymail.com>)
List pgsql-hackers

On Sun, Feb 15, 2015 at 8:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
There might be something in that, but again it's not much like this patch.
The key point I think we're both making is that nondeterministic failures
are bad, especially when you're talking about long-running, expensive-to-
retry transactions.

But I think Kevin would agree with you there. That's why he's interested in having the errors not occur if you don't read from the volatile tables. Ie, your reporting query reading from read-only tables would be guaranteed to succeed even if some other table had had some rows vacuumed away.

I'm not sure that's really going to work because of things like hint bit updates or hot pruning. That is, say your table is read-only now but last week some of the records were updated. You might reasonably expect those rows to be safe and indeed the rows themselves will still be in your report. But the earlier versions of the rows could still be sitting on some pages invisible to every query but not vacuumable quite yet and then suddenly they get vacuumed away and the LSN on the page gets bumped.

Fwiw I would strongly suggest that instead of having a number of transactions to have a time difference. I haven't been following the "commit timestamp" thread but I'm hoping that patch has the infrastructure needed to look up an lsn and find out the timestamp and vice versa. So when you take a snapshot you could look up the effective cut-off LSN based on the time difference specified in the GUC. As a DBA I would find it infinitely more comforting specifying "old_snapshot_threshold=4h" than specifying some arbitrary large number of transactions and hoping I calculated the transaction rate reasonably accurately.


--
greg

pgsql-hackers by date:

Previous
From: Peter Geoghegan
Date:
Subject: INSERT ... ON CONFLICT UPDATE and logical decoding
Next
From: Stephen Frost
Date:
Subject: Re: Allow "snapshot too old" error, to prevent bloat