Corrupt database? 8.1/FreeBSD6.0 - Mailing list pgsql-general

From Jeff Amiel
Subject Corrupt database? 8.1/FreeBSD6.0
Date
Msg-id 599454.19005.qm@web60814.mail.yahoo.com
Whole thread Raw
Responses Re: Corrupt database? 8.1/FreeBSD6.0  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
"PostgreSQL 8.1.2 on i386-portbld-freebsd6.0, compiled by GCC cc (GCC) 3.4.4 [FreeBSD] 20050518"

Started seeing this in the logs this afternoon...scaring the begeezus out of me.

Jan 11 19:20:19 prod-app-1 postgres[1752]: [5-1]  1752 ERROR:  could not access status of transaction 1924023481
Jan 11 19:20:19 prod-app-1 postgres[1752]: [5-2]  1752 DETAIL:  could not open file "pg_clog/072A": No such file or directory
Jan 11 19:24:35 prod-app-1 postgres[4094]: [5-1]  4094 ERROR:  could not access status of transaction 1924023481
Jan 11 19:24:35 prod-app-1 postgres[4094]: [5-2]  4094 DETAIL:  could not open file "pg_clog/072A": No such file or directory
Jan 11 19:28:35 prod-app-1 postgres[6728]: [5-1]  6728 ERROR:  could not access status of transaction 1924023481
Jan 11 19:28:35 prod-app-1 postgres[6728]: [5-2]  6728 DETAIL:  could not open file "pg_clog/072A": No such file or directory

I could find nothing any transaction refereced in pg_locks...nor could I find a file with that designation (was always the same transaction id)

ls -l /db/pg_clog/
total 984
-rw-------  1 pgsql  wheel  262144 Jan 11 09:55 07CF
-rw-------  1 pgsql  wheel  262144 Jan 11 13:45 07D0
-rw-------  1 pgsql  wheel  262144 Jan 11 17:00 07D1
-rw-------  1 pgsql  wheel  172032 Jan 11 20:39 07D2


At first I thought it was related to a constantly running batch process which I halted, and sure enough, the problem seemed to go away.  If I restarted, the problem returned.

I then tried select * from on most of the tables used by that process (at least the last days worth) and found no obvious issues or errors.
About that time, I noticed that the errors were all IMMEDIATELY preceeded by an autovacuum of template0.  So the logs actually looked like this:

Jan 11 19:20:19 prod-app-1 postgres[1752]: [4-1]  1752 LOG:  autovacuum: processing database "template0"
Jan 11 19:20:19 prod-app-1 postgres[1752]: [5-1]  1752 ERROR:  could not access status of transaction 1924023481
Jan 11 19:20:19 prod-app-1 postgres[1752]: [5-2]  1752 DETAIL:  could not open file "pg_clog/072A": No such file or directory
Jan 11 19:24:35 prod-app-1 postgres[4094]: [4-1]  4094 LOG:  autovacuum: processing database "template0"
Jan 11 19:24:35 prod-app-1 postgres[4094]: [5-1]  4094 ERROR:  could not access status of transaction 1924023481
Jan 11 19:24:35 prod-app-1 postgres[4094]: [5-2]  4094 DETAIL:  could not open file "pg_clog/072A": No such file or directory
Jan 11 19:28:35 prod-app-1 postgres[6728]: [4-1]  6728 LOG:  autovacuum: processing database "template0"
Jan 11 19:28:35 prod-app-1 postgres[6728]: [5-1]  6728 ERROR:  could not access status of transaction 1924023481
Jan 11 19:28:35 prod-app-1 postgres[6728]: [5-2]  6728 DETAIL:  could not open file "pg_clog/072A": No such file or directory

We've downloaded/compiled pg_filedump, but are stumped as to what relation (or even what database) to start with.

I restarted the batch process that I thought was the culprit and the issue has 'gone away'.  I'm sure there is some corruption somewhere (from reading various similar posts), but not sure where to start.

Any help would be appreciated.




Access over 1 million songs - Yahoo! Music Unlimited.

pgsql-general by date:

Previous
From: Chris
Date:
Subject: Re: documentation vs reality: template databases
Next
From: Jeff Amiel
Date:
Subject: Re: Corrupt database? 8.1/FreeBSD6.0