Thread: Invalid page header

Invalid page header

From
"Ed L."
Date:
I have a server with 20 pgsql clusters on it.  I am seeing a slew of errors
like the following across 5 of these clusters, all running 7.3.4 on Linux
with kernel 2.4.18-4bigmem:

    ERROR:  Invalid page header in block N of R

N and R vary among block numbers and relation names, respectively.  Looking
at the archives, it sounds like a possible hardware issue, and we're
investigating that.  In the meantime, every query that attempts to access
those blocks errors out.

Taking the server down or losing data is very expensive, so I'm looking for
some short-term band-aids to re-enable full production access and recover
data while we sort out the hardware issues.  When the corrupted block is
within a user index, I've been able to fix the corruption by dropping and
recreating the index.  Not sure what I can do for user tables yet.
Preparing to run e2fsck/badblocks after I'm convinced there's nothing more
to salvage.

Is there anything I can do for invalid page header in a block of
pg_depend_reference_index?

Other suggestions?

TIA.

Ed



Re: Invalid page header

From
Martijn van Oosterhout
Date:
Looks bad. Have you got backups. Seriously!

REINDEX works on system indexes but you have to drop to single user mode in
postgres to do it. Check out the -P option in the manpage.

Good luck!

Hope this helps,

On Thu, Dec 04, 2003 at 12:54:07PM -0700, Ed L. wrote:
> I have a server with 20 pgsql clusters on it.  I am seeing a slew of errors
> like the following across 5 of these clusters, all running 7.3.4 on Linux
> with kernel 2.4.18-4bigmem:
>
>     ERROR:  Invalid page header in block N of R
>
> N and R vary among block numbers and relation names, respectively.  Looking
> at the archives, it sounds like a possible hardware issue, and we're
> investigating that.  In the meantime, every query that attempts to access
> those blocks errors out.
>
> Taking the server down or losing data is very expensive, so I'm looking for
> some short-term band-aids to re-enable full production access and recover
> data while we sort out the hardware issues.  When the corrupted block is
> within a user index, I've been able to fix the corruption by dropping and
> recreating the index.  Not sure what I can do for user tables yet.
> Preparing to run e2fsck/badblocks after I'm convinced there's nothing more
> to salvage.
>
> Is there anything I can do for invalid page header in a block of
> pg_depend_reference_index?
>
> Other suggestions?
>
> TIA.
>
> Ed
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> "All that is needed for the forces of evil to triumph is for enough good
> men to do nothing." - Edmond Burke
> "The penalty good people pay for not being interested in politics is to be
> governed by people worse than themselves." - Plato

Attachment