Re: drive failre, corrupt data... - Mailing list pgsql-general

raid rebuilt...
ran fsck

PARTIALLY TRUNCATED INODE I=612353
SALVAGE? yes

INCORRECT BLOCK COUNT I=612353 (544 should be 416)
CORRECT? yes

PARTIALLY TRUNCATED INODE I=612389
SALVAGE? yes

INCORRECT BLOCK COUNT I=612389 (544 should be 416)
CORRECT? yes

INCORRECT BLOCK COUNT I=730298 (676448 should be 675520)
CORRECT? yes

root@back-app-1# find /db -inum 612353
/db/pg_clog/0952

root@back-app-1# find /db -inum 612389
/db/pg_clog/0951

root@back-app-1# find /db -inum 730298
/db/base/1093090/1212223

hmmm...wanted to see what the third one was so I

test=# select oid, relname from pg_class order by oid;

ERROR:  could not access status of transaction 2485385834
DETAIL:  could not open file "pg_clog/0942": No such file or directory

So....am I screwed here...just I just re-init-db and restore the entire kit and kaboodle from scratch?

Jeff Amiel <becauseimjeff@yahoo.com> wrote:

Had a drive failure on a raid 5 array of a backup box that a couple of postgres databases sit on.  One of the databases is a slony subscriber to a production database and the other is a test-environment database. 

The drive was offline...brought it back online, hoping it would start a rebuild...which it didn't. Almost immediately started getting errors from slony

could not access status of transaction 2463273456
could not open file "pg_clog/0937": No such file or directory
...
etc.

Looks like the subscriber database had some issues (at least with one specific table).  In addition, trying to access to the other (test) database yielded an error accessing pg_namespace.

So....reseated the drive which started a rebuild.  I stopped postgres.   When the rebuild isdone (or if it fails, I will replace the drive), I will restart postgres and see what happens.

Question...should I just re-initdb and restore databases from
 backup?  Should I have done something differently once I noticed the failure?  I've had drive failures before on this box and either rebuilt the array or replaced the drive with no postgres issues (although the amount of traffic was much less than now)

Any help would be appreciated.

---

Don't pick lemons.
See all the new 2007 cars at Yahoo! Autos.

---

Any questions? Get answers on any topic at Yahoo! Answers. Try it now.