Re: Dump/Reload pg_statistic to cut time from pg_upgrade? - Mailing list pgsql-admin

From Jerry Sievers
Subject Re: Dump/Reload pg_statistic to cut time from pg_upgrade?
Date
Msg-id 87r4f65sdm.fsf@comcast.net
Whole thread Raw
In response to Re: Dump/Reload pg_statistic to cut time from pg_upgrade?  (Kevin Grittner <kgrittn@ymail.com>)
Responses Re: Dump/Reload pg_statistic to cut time from pg_upgrade?  (Tom Lane <tgl@sss.pgh.pa.us>)
Re: Dump/Reload pg_statistic to cut time from pg_upgrade?  (Bruce Momjian <bruce@momjian.us>)
List pgsql-admin
Kevin Grittner <kgrittn@ymail.com> writes:

> Jerry Sievers <gsievers19@comcast.net> wrote:
>
>> Planning to pg_upgrade some large (3TB) clusters using hard link
>> method.  Run time for the upgrade itself takes around 5 minutes.
>> Nice!!  Origin version 8.4 and destination version 9.1.
>>
>> Unfortunately the post-upgrade analyze of the entire cluster is going
>> to take a minimum of 1.5 hours running several threads to analyze all
>> tables.  This was measured in an R&D environment.
>>
>> Got to thinking, what if we make a SAN snapshot a few hours prior to
>> upgrade time, upgrade that... then analyze it and then dump the stats
>> table?
>
>> Has anyone else attempted anything similar?  Any feedback is
>> appreciated.
>
> I certainly understand the motivation, and it may be a good option
> if you test carefully beforehand.
>
> What I have done in a similar situation, to minimize down time, is
> to run a database ANALYZE with a very small target.  I forget the
> particular value we used -- it may have been 3.  Then ran an
> ANALYZE with the normal default target on a few key tables
> (identified in testing to be the ones eating most of the scan time
> with no statistics), and let users in.  The database ANALYZE with
> the normal default target was done while normal production hit the
> database, without too much of a performance hit.  With this
> technique we were able to let users in with near-normal performance
> with 10 or 15 minutes of down time rather than hours.

Thanks Kevin!  In fact, I've conceived of this solution route already
and may have to resort to it.  We do run with
default_statistics_target set fairly high at 1000 (legacy setting
here) without my knowing for certain that it's required across the
board (and most likely is not).

Curious though if it's known that the pg_statistic table can't be
reloded from it's own pg_dump due to that error that I highlighted in
the original post.

Agree that being able to manually load pg_statistic is of questionable
usefulness though perhaps viable under the right circumstances.

> --
> Kevin Grittner
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

--
Jerry Sievers
e: jerry.sievers@comcast.net
p: 312.241.7800


pgsql-admin by date:

Previous
From: Kevin Grittner
Date:
Subject: Re: Dump/Reload pg_statistic to cut time from pg_upgrade?
Next
From: Tom Lane
Date:
Subject: Re: Dump/Reload pg_statistic to cut time from pg_upgrade?