Re: PITR Backups - Mailing list pgsql-performance

Thanks for the pointers to a) make it readable and b) log min messages

I didn't however keep the snapshots around. I could try and re-set
this scenario up. I was in the middle of doing some data migration
with Netapp and wanted to just 'test' it to make sure it was sane.

If you guys would like me to try to 'break' it again and keep the db
around for further testing let me know.

Regards,
Dan Gorman


On Jun 25, 2007, at 9:34 AM, Tom Lane wrote:

> Dan Gorman <dgorman@hi5.com> writes:
>> Jun 21 00:39:43 sfmedstorageha001 postgres[3506]: [9-1] 2007-06-21
>> 00:39:43 PDTLOG:  redo done at 71/99870670
>> Jun 21 00:39:43 sfmedstorageha001 postgres[3506]: [10-1] 2007-06-21
>> 00:39:43 PDTWARNING:  page 28905 of relation 1663/16384/76718 was
>> uninitialized
>> ... lots of these ...
>> Jun 21 00:39:43 sfmedstorageha001 postgres[3506]: [40-1] 2007-06-21
>> 00:39:43 PDTPANIC:  WAL contains references to invalid pages
>
> (BTW, you'll find putting a space at the end of log_line_prefix
> does wonders for log readability.)
>
> Reformatting and sorting, we have
>
> WARNING: page 3535207 of relation 1663/16384/33190 was uninitialized
> WARNING: page 3535208 of relation 1663/16384/33190 did not exist
> WARNING: page 3535209 of relation 1663/16384/33190 did not exist
>
> WARNING: page 800226 of relation 1663/16384/33204 did not exist
>
> WARNING: page 17306 of relation 1663/16384/76710 did not exist
>
> WARNING: page 13626 of relation 1663/16384/76716 did not exist
>
> WARNING: page 28900 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28902 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28903 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28904 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28905 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28906 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28907 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28908 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28909 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28910 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28911 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28912 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28913 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28914 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28915 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28916 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28917 of relation 1663/16384/76718 was uninitialized
> WARNING: page 28918 of relation 1663/16384/76718 was uninitialized
>
> WARNING: page 26706 of relation 1663/16384/76719 was uninitialized
> WARNING: page 26708 of relation 1663/16384/76719 was uninitialized
> WARNING: page 26710 of relation 1663/16384/76719 was uninitialized
> WARNING: page 26711 of relation 1663/16384/76719 was uninitialized
> WARNING: page 26712 of relation 1663/16384/76719 was uninitialized
> WARNING: page 26713 of relation 1663/16384/76719 was uninitialized
>
> So the problems were pretty localized, probably at the ends of these
> files.  Can you go back to the source database and check which
> tables these are --- match the last number cited in each line
> against pg_class.relfilenode?  Are they tables or indexes, and
> about how big are they?
>
> A possible explanation is we stopped scanning WAL before reaching
> records that truncated or dropped these tables.  But it's not clear
> why.
> Could we see the last few log lines before the "redo done" one?
>
>             regards, tom lane
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq