pgsql 8.3.3 "Invalid page header" leads to "FATAL: segment too big" - Mailing list pgsql-admin

From Achilleas Mantzios
Subject pgsql 8.3.3 "Invalid page header" leads to "FATAL: segment too big"
Date
Msg-id 201004221516.08729.achill@matrix.gatewaynet.com
Whole thread Raw
Responses Re: pgsql 8.3.3 "Invalid page header" leads to "FATAL: segment too big"
Re: pgsql 8.3.3 "Invalid page header" leads to "FATAL: segment too big"
List pgsql-admin
Hello,
i have this serious problem in one of our remote vessels. (comm is done by minicom to the remote satelite modem)
I think that this server was under some sort of constant resets or hardware failures.
Initially,i had this problem:
ERROR:  invalid page header in block 672720 of relation "pg_toast_125716009"

This toast table corresponds to a table named "mail_message",
                                Table "public.mail_message"
  Column   |       Type        |                         Modifiers
-----------+-------------------+-----------------------------------------------------------
 msgno     | mail_msgno_domain | not null default nextval('mail_msgno_sequence'::regclass)
 msgsource | bytea             |
Indexes:
    "mail_message_key" PRIMARY KEY, btree (msgno)

(obviously the TOAST table serves the msgsource varlena) the contents of which is not of vital importance.
I tried, REINDEXING, with no success, and after that, i tried resetting the said block on disk as per this
suggestion by Tom here: http://old.nabble.com/invalid-page-header-td11981154.html

i found the oid of the table:
SELECT tableoid,1 from pg_toast.pg_toast_125716009 order by chunk_id LIMIT 1
 tableoid  | ?column?
-----------+----------
 125716013 |        1

(and just to verify)
SELECT relname from pg_class where oid=125716013;
      relname
--------------------
 pg_toast_125716009

Then i did: (as i said i do not need the contents of msgsource - yet the contents of msgno are vital)
dd if=/dev/zero of=/usr/local/var/lib/pgsql/data/base/125714957/125716013 seek=672720 bs=8192 count=1

However, after that, unfortunately i get constant postgresql server restarts with:
FATAL:  segment too big
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.

Is there anything i can do to savage the situation?

(one of) the hard part here is that i dont have neither physical nor network access to the server
(only ultra expensive unreliable satellite comms)

Thanks for any hints...

--
Achilleas Mantzios

pgsql-admin by date:

Previous
From: Scott Marlowe
Date:
Subject: Re: ALTER TABLE ADD COLUMN Hangs
Next
From: Cédric Villemain
Date:
Subject: Re: pgsql 8.3.3 "Invalid page header" leads to "FATAL: segment too big"