Re: beta testing version - Mailing list pgsql-hackers

On Fri, Dec 01, 2000 at 12:00:12AM -0400, The Hermit Hacker wrote:
> On Thu, 30 Nov 2000, Nathan Myers wrote:
> > On Thu, Nov 30, 2000 at 07:02:01PM -0400, The Hermit Hacker wrote:
> > > v7.1 should improve crash recovery ...
> > > ... with the WAL stuff that Vadim is producing, you'll be able to
> > > recover up until the point that the power cable was pulled out of 
> > > the wall.
> > 
> > Please do not propagate falsehoods like the above.  It creates
> > unsatisfiable expectations, and leads people to fail to take
> > proper precautions and recovery procedures.  
> > 
> > After a power outage on an active database, you may have corruption
> > at low levels of the system, and unless you have enormous redundancy
> > (and actually use it to verify everything) the corruption may go 
> > undetected and result in (subtly) wrong answers at any future time.
> > 
> > The logging in 7.1 protects transactions against many sources of 
> > database crash, but not necessarily against OS crash, and certainly
> > not against power failure.  (You might get lucky, or you might just 
> > think you were lucky.)  This is the same as for most databases; an
> > embedded database that talks directly to the hardware might be able
> > to do better.  
> 
> We're talking about transaction logging here ... nothing gets written
> to it until completed ... if I take a "known to be clean" backup from
> the night before, restore that and then run through the transaction
> logs, my data should be clean, unless my tape itself is corrupt. If
> the power goes off half way through a write to the log, then that
> transaction wouldn't be marked as completed and won't roll into the
> restore ...

Sorry, wrong.  First, the only way that your backups could have any
relationship with the transaction logs is if they are copies of the
raw table files with the database shut down, rather than the normal 
"snapshot" backup.  

Second, the transaction log is not, as has been noted far too frequently
for Vince's comfort, really written atomically.  The OS has promised
to write it atomically, and given the opportunity, it will.  If you pull 
the plug, all promises are broken.

> if a disk goes corrupt, I'd expect that the redo log would possibly
> have a problem with corruption .. but if I pull the plug, unless I've
> somehow damaged the disk, I would expect my redo log to be clean
> *and*, unless Vadim totally messed something up, if there is any
> corruption in the redo log, I'd expect that restoring from it would
> generate from red flags ...

You have great expectations, but nobody has done the work to satisfy
them, so when you pull the plug, I'd expect that you will be left 
in the dark, alone and helpless.

Vadim has done an excellent job on what he set out to do: optimize
transaction processing.  Designing and implementing a factor-of-twenty 
speed improvement on a professional-quality database engine demanded
great effort and expertise.  To complain that he hasn't also done 
a lot of other stuff would be petty.

Nathan Myers
ncm@zembu.com