Re: data corruption hazard in reorderbuffer.c - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: data corruption hazard in reorderbuffer.c
Date
Msg-id d8302df8-e37b-a518-1e95-925eeb086af5@enterprisedb.com
Whole thread Raw
In response to Re: data corruption hazard in reorderbuffer.c  (Mark Dilger <mark.dilger@enterprisedb.com>)
Responses Re: data corruption hazard in reorderbuffer.c  (Mark Dilger <mark.dilger@enterprisedb.com>)
List pgsql-hackers
Hi,

I think it's mostly futile to list all the possible issues this might 
have caused - if you skip arbitrary decoded changes, that can trigger 
pretty much any bug in reorder buffer. But those bugs can be triggered 
by various other issues, of course.

It's hard to say what was the cause, but the "logic" bugs are probably 
permanent, while the issues triggered by I/O probably disappear after a 
restart?

That being said, I agree this seems like an issue and we should not 
ignore I/O errors. I'd bet other places using transient files (like 
sorting or hashagg spilling to disk) has the same issue, although in 
that case the impact is likely limited to a single query.

I wonder if sync before the close is an appropriate solution, though. It 
seems rather expensive, and those files are meant to be "temporary" 
(i.e. we don't keep them over restart). So maybe we could ensure the 
consistency is a cheaper way - perhaps tracking some sort of checksum 
for each file, or something like that?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: John Naylor
Date:
Subject: Re: speed up verifying UTF-8
Next
From: Mark Dilger
Date:
Subject: Re: data corruption hazard in reorderbuffer.c