Thread: power failure....

power failure....

From
Tom Allison
Date:
How do I restart the following with some level of sanity?

2007-11-30 19:35:20 EST LOG:  could not load root certificate file
"root.crt": no SSL error reported
2007-11-30 19:35:20 EST DETAIL:  Will not verify client certificates.
2007-11-30 19:35:20 EST LOG:  database system was interrupted at
2007-11-27 09:08:44 EST
2007-11-30 19:35:20 EST LOG:  record with zero length at 0/40808BA0
2007-11-30 19:35:20 EST LOG:  invalid primary checkpoint record
2007-11-30 19:35:20 EST LOG:  record with zero length at 0/407C9628
2007-11-30 19:35:20 EST LOG:  invalid secondary checkpoint record
2007-11-30 19:35:20 EST PANIC:  could not locate a valid checkpoint
record
2007-11-30 19:35:20 EST LOG:  startup process (PID 8755) was
terminated by signal 6
2007-11-30 19:35:20 EST LOG:  aborting startup due to startup process
failure


postgres version 8.2 sitting on a Reiser FS on RAID1....

I'm not finding much on the web other than bad news...


Re: power failure....

From
Tom Lane
Date:
Tom Allison <tom@tacocat.net> writes:
> 2007-11-30 19:35:20 EST PANIC:  could not locate a valid checkpoint
> record

Ugh :-(.  pg_resetxlog should get you back into the database, but it's
anybody's guess whether and how badly the contents will be corrupted.
I would recommend trying a dump/reload and also doing any manual
consistency crosschecks you can think of.

> postgres version 8.2 sitting on a Reiser FS on RAID1....

Better take a second look at your disk hardware configuration.  At least
in theory, this Can't Happen if your disk hardware is handling fsync
honestly.

            regards, tom lane

Re: power failure....

From
Tom Allison
Date:
Tom Lane wrote:
> Tom Allison <tom@tacocat.net> writes:
>> 2007-11-30 19:35:20 EST PANIC:  could not locate a valid checkpoint
>> record
>
> Ugh :-(.  pg_resetxlog should get you back into the database, but it's
> anybody's guess whether and how badly the contents will be corrupted.
> I would recommend trying a dump/reload and also doing any manual
> consistency crosschecks you can think of.
>
>> postgres version 8.2 sitting on a Reiser FS on RAID1....
>
> Better take a second look at your disk hardware configuration.  At least
> in theory, this Can't Happen if your disk hardware is handling fsync
> honestly.
>
>             regards, tom lane


Cool, thanks.

I wonder what I'm supposed to do with my debian installation since there doesn't
seem to be any such thing as pg_resetxlog.  Or is it hiding somewhere?

Re: power failure....

From
Tom Allison
Date:
Tom Lane wrote:
> Tom Allison <tom@tacocat.net> writes:
>> 2007-11-30 19:35:20 EST PANIC:  could not locate a valid checkpoint
>> record
>
> Ugh :-(.  pg_resetxlog should get you back into the database, but it's
> anybody's guess whether and how badly the contents will be corrupted.
> I would recommend trying a dump/reload and also doing any manual
> consistency crosschecks you can think of.
>
>> postgres version 8.2 sitting on a Reiser FS on RAID1....
>
> Better take a second look at your disk hardware configuration.  At least
> in theory, this Can't Happen if your disk hardware is handling fsync
> honestly.
>
>             regards, tom lane


I tracked it down and did a reset.
I only have one large table right now.

And now I've decided to start using fsync=on!!!  :)
I don't think I want to "test" this just yet...

Re: power failure....

From
"Joshua D. Drake"
Date:
On Sat, 01 Dec 2007 15:22:38 -0500
Tom Allison <tom@tacocat.net> wrote:
    regards, tom lane
>
>
> Cool, thanks.
>
> I wonder what I'm supposed to do with my debian installation since
> there doesn't seem to be any such thing as pg_resetxlog.  Or is it
> hiding somewhere?

I don't recall if it is in contrib or not.. try?:

apt-file is your friend:

apt-file search pg_resetxlog

/usr/lib/postgresql/8.1/bin/pg_resetxlog


Joshua D. Drake

>
> ---------------------------(end of
> broadcast)--------------------------- TIP 5: don't forget to increase
> your free space map settings
>

Attachment

Re: power failure....

From
"Joshua D. Drake"
Date:
On Sat, 01 Dec 2007 15:32:50 -0500
Tom Allison <tom@tacocat.net> wrote:

> I tracked it down and did a reset.
> I only have one large table right now.
>
> And now I've decided to start using fsync=on!!!  :)

change wal_sync_method to open_sync and fsync=on isn't nearly as bad as
it sounds.

joshua drake

> I don't think I want to "test" this just yet...
>
> ---------------------------(end of
> broadcast)--------------------------- TIP 6: explain analyze is your
> friend
>

Attachment

Re: power failure....

From
Martin Marques
Date:
Tom Allison escribió:
>
> I wonder what I'm supposed to do with my debian installation since there
> doesn't seem to be any such thing as pg_resetxlog.  Or is it hiding
> somewhere?

On one debian I have:

/usr/lib/postgresql/8.1/bin/pg_resetxlog
/usr/lib/postgresql/8.2/bin/pg_resetxlog


Re: power failure....

From
Martin Marques
Date:
Joshua D. Drake escribió:
> On Sat, 01 Dec 2007 15:22:38 -0500
>
>> I wonder what I'm supposed to do with my debian installation since
>> there doesn't seem to be any such thing as pg_resetxlog.  Or is it
>> hiding somewhere?
>
> I don't recall if it is in contrib or not.. try?:
>
> apt-file is your friend:
>
> apt-file search pg_resetxlog
>
> /usr/lib/postgresql/8.1/bin/pg_resetxlog

$ dlocate /usr/lib/postgresql/8.2/bin/pg_resetxlog
postgresql-8.2: /usr/lib/postgresql/8.2/bin/pg_resetxlog


Re: power failure....

From
Greg Smith
Date:
On Sat, 1 Dec 2007, Joshua D. Drake wrote:

> change wal_sync_method to open_sync and fsync=on isn't nearly as bad as
> it sounds.

Just be warned that there's been one report that some Linux versions have
bugs that make open_sync problematic:

http://archives.postgresql.org/pgsql-hackers/2007-10/msg01310.php

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Re: power failure....

From
Tom Allison
Date:
I was able to get things up and running OK.
Don't have any WAL that I'm aware of, but it managed to have another
power failure hours later.

I seems that the UPS is more POS than UPS.  I think the battery is dead.




On Dec 2, 2007, at 3:52 AM, Greg Smith wrote:

> On Sat, 1 Dec 2007, Joshua D. Drake wrote:
>
>> change wal_sync_method to open_sync and fsync=on isn't nearly as
>> bad as
>> it sounds.
>
> Just be warned that there's been one report that some Linux
> versions have bugs that make open_sync problematic:
>
> http://archives.postgresql.org/pgsql-hackers/2007-10/msg01310.php
>
> --
> * Greg Smith gsmith@gregsmith.com http://www.gregsmith.com
> Baltimore, MD