Re: Replication terminated due to PANIC - Mailing list pgsql-general

From Adarsh Sharma
Subject Re: Replication terminated due to PANIC
Date
Msg-id CAGx-QqLQOf8AJzbSr+JUKM8bfWz9AQ4MEWrTHcyB8UgGF0xV5A@mail.gmail.com
Whole thread Raw
In response to Re: Replication terminated due to PANIC  (Sergey Konoplev <gray.ru@gmail.com>)
Responses Re: Replication terminated due to PANIC
List pgsql-general
Thanks Sergey for such a quick response, but i dont think this is some patch problem because we have other DB servers also running fine on same version and message is also different :

host= PANIC: _bt_restore_page: cannot add item to page

And the whole day replication is working fine but at midnight when log rotates it shows belows msg :

2013-04-24 00:00:00 UTC [26989]: [4945032-1] user= db= host= LOG: checkpoint starting: time
2013-04-24 00:00:00 UTC [26989]: [4945033-1] user= db= host= ERROR: could not open file "global/14078": No such file or directory
2013-04-24 00:00:00 UTC [26989]: [4945034-1] user= db= host= CONTEXT: writing block 0 of relation global/14078
2013-04-24 00:00:00 UTC [26989]: [4945035-1] user= db= host= WARNING: could not write block 0 of global/14078
2013-04-24 00:00:00 UTC [26989]: [4945036-1] user= db= host= DETAIL: Multiple failures --- write error might be permanent.
Looks like some index corruption.
 
Thanks





On Thu, Apr 25, 2013 at 8:14 AM, Sergey Konoplev <gray.ru@gmail.com> wrote:
On Wed, Apr 24, 2013 at 5:05 PM, Adarsh Sharma <eddy.adarsh@gmail.com> wrote:
> I have a Postgresql 9.2 instance running on a CentOS6.3 box.Yesterday i
> setup a hot standby by using pgbasebackup. Today i got the below  alert from
> standby box :
>
> [1] (from line 412,723)
> 2013-04-24 23:07:18 UTC [13445]: [6-1] user= db= host= PANIC:
> _bt_restore_page: cannot add item to page
>
> When i check, the replication is terminated due to slave DB shutdown. From
> the logs i can see below messages :-

I am not sure that it is your situation but take a look at this thread:

http://www.postgresql.org/message-id/CAL_0b1t=WuM6roO8dki=w8DhH8P8whhohbPjReymmQUrOcNT2A@mail.gmail.com

There is a patch by Andres Freund in the end of the discussion. Three
weeks have passed after I installed the patched version and it looks
like the patch fixed my issue.

>
> 2013-04-24 23:17:16 UTC [26989]: [5360083-1] user= db= host= ERROR:  could
> not open file "global/14078": No such file or directory
> 2013-04-24 23:17:16 UTC [26989]: [5360084-1] user= db= host= CONTEXT:
> writing block 0 of relation global/14078
> 2013-04-24 23:17:16 UTC [26989]: [5360085-1] user= db= host= WARNING:  could
> not write block 0 of global/14078
> 2013-04-24 23:17:16 UTC [26989]: [5360086-1] user= db= host= DETAIL:
> Multiple failures --- write error might be permanent.
>
> I checked in global directory of master, the directory 14078 doesn't exist.
>
> Anyone has faced above issue ?
>
> Thanks



--
Kind regards,
Sergey Konoplev
Database and Software Consultant

Profile: http://www.linkedin.com/in/grayhemp
Phone: USA +1 (415) 867-9984, Russia +7 (901) 903-0499, +7 (988) 888-1979
Skype: gray-hemp
Jabber: gray.ru@gmail.com

pgsql-general by date:

Previous
From: Tom Lane
Date:
Subject: Re: DISTINCT ON changes sort order
Next
From: jesse.waters@gmail.com
Date:
Subject: Re: pgdump error "Could not open file pg_clog/0B8E: No such file or directory"