Thread: TOAST table (or index?) corruption on PITR replica

We have a customer running a very high-velocity database.  We have set
up a PITR replica that's restoring correctly using JD's PITRtools and
pg_standby.

However, upon stopping the recovery and starting it for production (for
"switchover") it promptly gets into these problems:

14895 2009-02-28 22:19:08 EST kangaroo app 49a9fe2e.3a2f ERROR: missing chunk number 0 for toast value 93557123
14895 2009-02-28 22:19:08 EST kangaroo app 49a9fe2e.3a2f STATEMENT: update sometable set stop_status=1,
stop_time=current_timestamp,stop_method=1 where unit_id='227245' and email='foobar@example.com'
 

14903 2009-02-28 22:22:02 EST kangaroo app 49a9fe2e.3a37 ERROR: index "pg_toast_42299_index" contains unexpected zero
pageat block 23768
 
14903 2009-02-28 22:22:02 EST kangaroo app 49a9fe2e.3a37 HINT: Please REINDEX it.

There were actually several different instances of the first error
involving different toast values.

The servers are running 8.3.5.

Does this ring a bell for anyone?

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera <alvherre@commandprompt.com> writes:

> 14903 2009-02-28 22:22:02 EST kangaroo app 49a9fe2e.3a37 ERROR: index
> "pg_toast_42299_index" contains unexpected zero page at block 23768
>
> There were actually several different instances of the first error
> involving different toast values.

Do you know how big this table was when the backup was taken? Were these
blocks present then or were they added since then? My suspicion would lie with
the relation extension code where there have been bugs before, but I don't see
any found recently.

--  Gregory Stark EnterpriseDB          http://www.enterprisedb.com Ask me about EnterpriseDB's PostGIS support!

Alvaro Herrera píše v pá 06. 03. 2009 v 10:32 -0300:

> 
> The servers are running 8.3.5.
> 
> Does this ring a bell for anyone?

look here:

http://archives.postgresql.org/pgsql-bugs/2008-11/msg00113.php

It look likes that there could be problem with btree wal.
Zdenek