Thread: 7.4 <-> 8.0

All,

Couple questions from to a recent upgrade from 7.4.8 to 8.0.3.  With
servers for both versions running on the same machine,

   pg_dumpall -p 5432 -c | psql -p 5433 template1

appears to have migrated the databases smoothly.  At least I didn't
notice any errors in the process, and all the data appears to be there.

The one troubling thing is the new 8.0 databases take about half the
disk space as the 7.4 databases.  Similar dump | psql to recreate the
databases on other machines running 7.4.7 and 7.4.8 show the same thing
- the fresh copies take half the space of the old.  What might be
happening with the old db, so it takes so much more space?  Could half
the data be missing?

What's the best way to move data from 8.0 to 7.4?  The 8.0 pg_dump
writes a dump that doesn't restore with 7.4 psql, and 7.4 pg_dump
doesn't seem to handle an 8.0 database.  At least I haven't found the
right switches...

Thanks,
warren

--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.

--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.

Guido,

Thanks for the pointers.  I have routinely vacuumed the db, but not
using full.

By itself, the "vacuum full" run overnight did not free the space I was
expecting - du still showed the old database taking about 12 Gb disk,
and the fresh databases taking about 6.5 Gb.  Running "reindex" on one
large table freed ~5 Gb, so the old db now takes 7 Gb.  Reindexing the
remaining tables should get the databases to the same size.

Would you suggest scripting the vacuum full / reindex process to run
periodically (once a week or so)?  Right now disk space on this machine
is not an issue, but vacuuming and reindexing occasionally should keep
the db from consuming too much excess space.

Thanks again,
warren

> still thinking on this topic, read about the REINDEX command, that
> will recreate your indexes, per table or per database.
>
> Regards,
> Guido.
>
>
> On 9/15/05, Guido Barosio <gbarosio@gmail.com> wrote:
>         Hi Warren
>
>         On the space issue, seems at a first sight that the old db
>         needs a vacuum full to free some room.
>         Try to identify your biggest tables, isolate the vacuum
>         against them, and then escalate to the whole db, with a vacuum
>         full.
>
>         Thinking that you should double check the vacuum
>         documentation, to understand better which are the effects of
>         the different vacuum modes. This will clear out your doubts on
>         why such amount of space is being allocated, but not being
>         used.
>
>         My two cents there.
>
>         Best wishes,
>         Guido.
>
>         On 9/15/05, Warren Snelling <snelling@email.marc.usda.gov>
>         wrote:
>                 All,
>
>                 Couple questions from to a recent upgrade from 7.4.8
>                 to 8.0.3.  With
>                 servers for both versions running on the same
>                 machine,
>
>                   pg_dumpall -p 5432 -c | psql -p 5433 template1
>
>                 appears to have migrated the databases smoothly.  At
>                 least I didn't
>                 notice any errors in the process, and all the data
>                 appears to be there.
>
>                 The one troubling thing is the new 8.0 databases take
>                 about half the
>                 disk space as the 7.4 databases.  Similar dump | psql
>                 to recreate the
>                 databases on other machines running 7.4.7 and 7.4.8
>                 show the same thing
>                 - the fresh copies take half the space of the
>                 old.  What might be
>                 happening with the old db, so it takes so much more
>                 space?  Could half
>                 the data be missing?
>
>                 What's the best way to move data from 8.0 to 7.4?  The
>                 8.0 pg_dump
>                 writes a dump that doesn't restore with 7.4 psql, and
>                 7.4 pg_dump
>                 doesn't seem to handle an 8.0 database.  At least I
>                 haven't found the
>                 right switches...
>
>                 Thanks,
>                 warren
>
>
>                 ---------------------------(end of
>                 broadcast)---------------------------
>                 TIP 2: Don't 'kill -9' the postmaster
>
>
>
>
>         --
>         "Adopting the position that you are smarter than an automatic
>         optimization algorithm is generally a good way to achieve less
>         performance, not more" - Tom Lane.
>
>
>
> --
> "Adopting the position that you are smarter than an automatic
> optimization algorithm is generally a good way to achieve less
> performance, not more" - Tom Lane.

Hi Warren,

Actually, in my particular case (old pgsql version, and a big db), reindex and vacuum full are extremely expensive in operation costs,
and I am running them myself on planned conditions. But that's my case. Scripting this, with a cron hand,
would be ok, and I guess that is something you will prolly find out there.

The reindex process is something to watch at, and not to leave running during the night, without some bird eye on it,
at least on my scenery.

Best wishes,
Guido.





--
"Adopting the position that you are smarter than an automatic
optimization algorithm is generally a good way to achieve less
performance, not more" - Tom Lane.

On Fri, 2005-09-16 at 10:57, Warren Snelling wrote:
> Guido,
>
> Thanks for the pointers.  I have routinely vacuumed the db, but not
> using full.
>
> By itself, the "vacuum full" run overnight did not free the space I was
> expecting - du still showed the old database taking about 12 Gb disk,
> and the fresh databases taking about 6.5 Gb.  Running "reindex" on one
> large table freed ~5 Gb, so the old db now takes 7 Gb.  Reindexing the
> remaining tables should get the databases to the same size.
>
> Would you suggest scripting the vacuum full / reindex process to run
> periodically (once a week or so)?  Right now disk space on this machine
> is not an issue, but vacuuming and reindexing occasionally should keep
> the db from consuming too much excess space.

Assuming that you run regular vacuums often enough, and your fsm
settings are high enough to hold on to all the references to the deleted
tuples, you shouldn't need to run either full vacuums or reindexes.
There are some usage patterns that can result in indexes with very
sparsely populated blocks, such as when you create indexes to the
numbers 1 to 1000, then delete 95% or so of those references, and all
the new ones will be >1000, then repeat, increasing the number all the
time.  But that type of situation is the only that might require
occasional reindexing.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Warren Snelling wrote:

>
>
> Would you suggest scripting the vacuum full / reindex process to
> run periodically (once a week or so)? Right now disk space on this
> machine is not an issue, but vacuuming and reindexing occasionally
> should keep the db from consuming too much excess space.
>
    7.4 has a problem with vacuuming indexes I seam to remember. I
think thats this is one of the fixes in 8.0. Apart from anything a
large index is going to slow your database down as the database has to
scan past junk everytime it does an index lookup.

Peter Childs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDKvMKJWgkOaNnHdMRAsBEAKCv3ZWdy1a4+qQG0U9Ov85TPCembQCfazPH
QVPj5xEDPSumy1Z4FUSTgf4=
=As1l
-----END PGP SIGNATURE-----


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

               http://archives.postgresql.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Warren Snelling wrote:

>
>
> Would you suggest scripting the vacuum full / reindex process to
> run periodically (once a week or so)? Right now disk space on this
> machine is not an issue, but vacuuming and reindexing occasionally
> should keep the db from consuming too much excess space.
>
    7.4 has a problem with vacuuming indexes I seam to remember. I
think thats this is one of the fixes in 8.0. Apart from anything a
large index is going to slow your database down as the database has to
scan past junk everytime it does an index lookup.

Peter Childs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDKvMKJWgkOaNnHdMRAsBEAKCv3ZWdy1a4+qQG0U9Ov85TPCembQCfazPH
QVPj5xEDPSumy1Z4FUSTgf4=
=As1l
-----END PGP SIGNATURE-----


---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

               http://archives.postgresql.org