Thread: Replication stopped on 9.0.2 after making change to conf file

Replication stopped on 9.0.2 after making change to conf file

From

akp geek

Date:

08 March 2013, 03:28:45

Hi all -

Recently made change on our primary database

default_text_search_config = 'pg_catalog.simple' . After that the replication is stopped. Can you please help me ? how to fix the issue. I am sure I made the change on the slave also.

How can I start the replication and catch up the data. Thanks for your time.

Regards

Re: Replication stopped on 9.0.2 after making change to conf file

From

Scott Marlowe

Date:

08 March 2013, 05:25:43

On Thu, Mar 7, 2013 at 5:28 PM, akp geek <akpgeek@gmail.com> wrote:
> Hi all -
>
>                Recently made change on our primary database
>
>               default_text_search_config = 'pg_catalog.simple' . After that
> the replication is stopped. Can you please help me ? how to fix the issue. I
> am sure I made the change on the slave also.
>
> How can I start the replication and catch up the data. Thanks for your time.

What are you seeing in your slony and / or postgresql logs, if anything?

Re: Replication stopped on 9.0.2 after making change to conf file

From

akp geek

Date:

08 March 2013, 19:53:57

I got it fixed.

What I did was

$ psql -c "SELECT pg_start_backup('label', true)"
$ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid
$ psql -c "SELECT pg_stop_backup()"

It took a while a to catch up the data.

One question I have , are there any scripts to monitor the status of the replciation. so that I can be little proactive

Regards

On Thu, Mar 7, 2013 at 9:25 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:

On Thu, Mar 7, 2013 at 5:28 PM, akp geek <akpgeek@gmail.com> wrote:
> Hi all -
>
> Recently made change on our primary database
>
> default_text_search_config = 'pg_catalog.simple' . After that
> the replication is stopped. Can you please help me ? how to fix the issue. I
> am sure I made the change on the slave also.
>
> How can I start the replication and catch up the data. Thanks for your time.

What are you seeing in your slony and / or postgresql logs, if anything?

Re: Replication stopped on 9.0.2 after making change to conf file

From

John Laing

Date:

08 March 2013, 20:06:21

I'm not sure about the existence of any standard scripts, but we have a pair of checks running periodically on the backup server.

This shouldn't return anything:

tail -3 /var/log/postgresql/postgresql-9.1-main.log | grep FATAL

And this should return something:

ps -u postgres -o cmd | grep "postgres: wal receiver process streaming"

These have worked very reliably for many months.

-John

On Fri, Mar 8, 2013 at 11:53 AM, akp geek <akpgeek@gmail.com> wrote:

I got it fixed.

What I did was
$ psql -c "SELECT pg_start_backup('label', true)"
$ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid
$ psql -c "SELECT pg_stop_backup()"
It took a while a to catch up the data.
One question I have , are there any scripts to monitor the status of the replciation. so that I can be little proactive
Regards
On Thu, Mar 7, 2013 at 9:25 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
On Thu, Mar 7, 2013 at 5:28 PM, akp geek <akpgeek@gmail.com> wrote:
> Hi all -
>
> Recently made change on our primary database
>
> default_text_search_config = 'pg_catalog.simple' . After that
> the replication is stopped. Can you please help me ? how to fix the issue. I
> am sure I made the change on the slave also.
>
> How can I start the replication and catch up the data. Thanks for your time.

What are you seeing in your slony and / or postgresql logs, if anything?

Re: Replication stopped on 9.0.2 after making change to conf file

From

akp geek

Date:

09 March 2013, 08:53:04

Thanks a lot. I started the replication. It became very slow. It is taking long time to sync the masters data onto slave. Is there a way to find what's causing the issue?

Regards

On Fri, Mar 8, 2013 at 12:06 PM, John Laing <john.laing@gmail.com> wrote:

I'm not sure about the existence of any standard scripts, but we have a pair of checks running periodically on the backup server.

This shouldn't return anything:
tail -3 /var/log/postgresql/postgresql-9.1-main.log | grep FATAL

And this should return something:
ps -u postgres -o cmd | grep "postgres: wal receiver process streaming"

These have worked very reliably for many months.

-John
On Fri, Mar 8, 2013 at 11:53 AM, akp geek <akpgeek@gmail.com> wrote:
I got it fixed.

What I did was
$ psql -c "SELECT pg_start_backup('label', true)"
$ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid
$ psql -c "SELECT pg_stop_backup()"
It took a while a to catch up the data.
One question I have , are there any scripts to monitor the status of the replciation. so that I can be little proactive
Regards
On Thu, Mar 7, 2013 at 9:25 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
On Thu, Mar 7, 2013 at 5:28 PM, akp geek <akpgeek@gmail.com> wrote:
> Hi all -
>
> Recently made change on our primary database
>
> default_text_search_config = 'pg_catalog.simple' . After that
> the replication is stopped. Can you please help me ? how to fix the issue. I
> am sure I made the change on the slave also.
>
> How can I start the replication and catch up the data. Thanks for your time.

What are you seeing in your slony and / or postgresql logs, if anything?

Re: Replication stopped on 9.0.2 after making change to conf file

From

akp geek

Date:

09 March 2013, 10:06:40

What I noticed is...

our streaming stopped on march 5th. I restarted the process today. However the new data is syncing correctly. But the data for these dates between 5th and today is not syncing. Is there some thing wrong that I did.

here is what I did.

1.. created base backup

2. Took the data directory on to the slave

3. stopped the base backup on master

4. started the db on slave

Appreciate your help.

Regards

On Sat, Mar 9, 2013 at 12:52 AM, akp geek <akpgeek@gmail.com> wrote:

Thanks a lot. I started the replication. It became very slow. It is taking long time to sync the masters data onto slave. Is there a way to find what's causing the issue?

Regards
On Fri, Mar 8, 2013 at 12:06 PM, John Laing <john.laing@gmail.com> wrote:
I'm not sure about the existence of any standard scripts, but we have a pair of checks running periodically on the backup server.

This shouldn't return anything:
tail -3 /var/log/postgresql/postgresql-9.1-main.log | grep FATAL

And this should return something:
ps -u postgres -o cmd | grep "postgres: wal receiver process streaming"

These have worked very reliably for many months.

-John
On Fri, Mar 8, 2013 at 11:53 AM, akp geek <akpgeek@gmail.com> wrote:
I got it fixed.

What I did was
$ psql -c "SELECT pg_start_backup('label', true)"
$ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude postmaster.pid
$ psql -c "SELECT pg_stop_backup()"
It took a while a to catch up the data.
One question I have , are there any scripts to monitor the status of the replciation. so that I can be little proactive
Regards
On Thu, Mar 7, 2013 at 9:25 PM, Scott Marlowe <scott.marlowe@gmail.com> wrote:
On Thu, Mar 7, 2013 at 5:28 PM, akp geek <akpgeek@gmail.com> wrote:
> Hi all -
>
> Recently made change on our primary database
>
> default_text_search_config = 'pg_catalog.simple' . After that
> the replication is stopped. Can you please help me ? how to fix the issue. I
> am sure I made the change on the slave also.
>
> How can I start the replication and catch up the data. Thanks for your time.

What are you seeing in your slony and / or postgresql logs, if anything?

Re: Replication stopped on 9.0.2 after making change to conf file

From

Lonni J Friedman

Date:

09 March 2013, 19:14:12

It sounds like all you did was setup the slave from scratch with a
fresh base backup, without understanding or debugging what caused
everything to break.  Clearly whatever was wrong on March 5 is still
wrong, and nothing has been fixed.  The first step in debugging this
problem is to look at and/or post the log content (from both the
master & slave) from the time when this stopped working (march 5).

On Fri, Mar 8, 2013 at 11:06 PM, akp geek <akpgeek@gmail.com> wrote:
> What I noticed is...
>
> our streaming stopped on march 5th.   I restarted the process today.
> However the new data is syncing correctly. But the data for these  dates
> between 5th and today is not syncing.  Is there some thing wrong that I did.
>
> here is what I did.
>
> 1.. created base backup
> 2.  Took the data directory on to the slave
> 3.  stopped the base backup on master
> 4. started the db on slave
>
> Appreciate your help.
>
> Regards
>
>
>
> On Sat, Mar 9, 2013 at 12:52 AM, akp geek <akpgeek@gmail.com> wrote:
>>
>> Thanks a lot. I started the replication. It became very slow. It is taking
>> long time to sync the masters data onto slave. Is there a way to find what's
>> causing the issue?
>>
>> Regards
>>
>>
>>
>> On Fri, Mar 8, 2013 at 12:06 PM, John Laing <john.laing@gmail.com> wrote:
>>>
>>> I'm not sure about the existence of any standard scripts, but we have a
>>> pair of checks running periodically on the backup server.
>>>
>>> This shouldn't return anything:
>>> tail -3 /var/log/postgresql/postgresql-9.1-main.log | grep FATAL
>>>
>>> And this should return something:
>>> ps -u postgres -o cmd | grep "postgres: wal receiver process   streaming"
>>>
>>> These have worked very reliably for many months.
>>>
>>> -John
>>>
>>>
>>> On Fri, Mar 8, 2013 at 11:53 AM, akp geek <akpgeek@gmail.com> wrote:
>>>>
>>>> I got it fixed.
>>>>
>>>> What I did was
>>>>
>>>> $ psql -c "SELECT pg_start_backup('label', true)"
>>>> $ rsync -a ${PGDATA}/ standby:/srv/pgsql/standby/ --exclude
>>>> postmaster.pid
>>>> $ psql -c "SELECT pg_stop_backup()"
>>>>
>>>> It took a while a to catch up the data.
>>>>
>>>>
>>>> One question I have , are there any scripts to monitor the status of the
>>>> replciation. so that I can be little proactive
>>>>
>>>>
>>>> Regards
>>>>
>>>>
>>>>
>>>> On Thu, Mar 7, 2013 at 9:25 PM, Scott Marlowe <scott.marlowe@gmail.com>
>>>> wrote:
>>>>>
>>>>> On Thu, Mar 7, 2013 at 5:28 PM, akp geek <akpgeek@gmail.com> wrote:
>>>>> > Hi all -
>>>>> >
>>>>> >                Recently made change on our primary database
>>>>> >
>>>>> >               default_text_search_config = 'pg_catalog.simple' .
>>>>> > After that
>>>>> > the replication is stopped. Can you please help me ? how to fix the
>>>>> > issue. I
>>>>> > am sure I made the change on the slave also.
>>>>> >
>>>>> > How can I start the replication and catch up the data. Thanks for
>>>>> > your time.
>>>>>
>>>>> What are you seeing in your slony and / or postgresql logs, if
>>>>> anything?