Thread: Testing WAL replay by comparing before and after images again

I rerun my little testing tool that compares buffer page contents after
every modification, in master and in WAL replay. Previously discussed
here: http://www.postgresql.org/message-id/5357B582.7060707@vmware.com.
Here's an updated version of my original hack, for current git master.
(Michael posted less-hacky versions of that, but unfortunately I haven't
gotten around to review his stuff.)

I did not find any new bugs. There were a couple of false positives
however. Firstly, the post-processing tool needed to be taught that BRIN
pages can have the PD_HAS_FREE_LINES flag set, and ignore that (like it
does for heap and other indexam pages).

Another issue was with the new speculative insertions. Replaying a
speculative insertion record sets the tuple's CTID to point to itself,
like in a regular insertion. But in the original system, the CTID is set
to a special speculative insertion token. The tool flagged up that
difference.

I propose the attached patch
(mark-speculative-insertions-in-replay.patch) to fix that in the replay
routine. This is not required for correctness, but helps this tool, and
seems like a good idea for debugging purposes anyway.

Any objections?

- Heikki

On 09/04/2015 09:30 PM, Simon Riggs wrote:
> On 4 September 2015 at 13:45, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
>> Another issue was with the new speculative insertions. Replaying a
>> speculative insertion record sets the tuple's CTID to point to itself, like
>> in a regular insertion. But in the original system, the CTID is set to a
>> special speculative insertion token. The tool flagged up that difference.
>>
>> I propose the attached patch (mark-speculative-insertions-in-replay.patch)
>> to fix that in the replay routine. This is not required for correctness,
>> but helps this tool, and seems like a good idea for debugging purposes
>> anyway.
>
> ISTM that the WAL record should include the speculative insertion token, so
> that replay can set it correctly.

I view this the same as command IDs. We don't restore the original 
command ID of a tuple at WAL replay either, because it's irrelevant for 
recovery and hot standby.

> That way we can always re-check that the later update matches the
> speculative insertion token we expect, in all cases.

Hmm, I guess that would give a tiny bit of extra sanity checking at 
replay. Doesn't really seem worth the trouble and extra WAL volume to me.

> In any case, the assumption that we are replaying all changes in single
> threaded mode is not appropriate for use with logical replication.

No such assumption here AFAICS.

- Heikki