Thread: Upgrading: So now you tell me!!?!?

Upgrading: So now you tell me!!?!?

From
Hacksaw
Date:
I had an older installation of postgresql, 7.1.3. I got down the packages for
7.3.2. I looked at a few of the files in the directories where the packages
were kept, and seeing no particular warnings, I ran rpm -Uvh postgresql*,
which ran without a hitch, until I restarted the postmaster. Then it informed
me that I should have dumped the damned database before I upgraded the
binaries.

This is like seeing the "Bridge Out" sign at the bottom of the river.

Why didn't the preinstall scripts dump the database? Why doesn't the FAQ or
the README in the download directory mention the required procedure?

If there is going to be a conflict that might require having the old binaries
around, why do the new ones replace the old ones?
--
In the creative leap, history waits outside.
http://www.hacksaw.org -- http://www.privatecircus.com -- KB1FVD


Re: Upgrading: So now you tell me!!?!?

From
"David Olbersen"
Date:
Hacksaw,

I feel your pain but can't share your experience.

The FreeBSD port of PostgreSQL says in nice big (err, monospaced) letters:

YOU SHOULD BACKUP YOUR DATA BEFORE PROCEEDING!

I'm pretty sure the normal (compile, install) instructions will let you know to back things up.

Yet another example of why RPMs are evil?

--------------------------
David Olbersen
iGuard Engineer
11415 West Bernardo Court
San Diego, CA 92127
1-858-676-2277 x2152


> -----Original Message-----
> From: Hacksaw [mailto:hacksaw@hacksaw.org]
> Sent: Thursday, March 27, 2003 10:17 AM
> To: pgsql-admin@postgresql.org
> Subject: [ADMIN] Upgrading: So now you tell me!!?!?
>
>
> I had an older installation of postgresql, 7.1.3. I got down
> the packages for
> 7.3.2. I looked at a few of the files in the directories
> where the packages
> were kept, and seeing no particular warnings, I ran rpm -Uvh
> postgresql*,
> which ran without a hitch, until I restarted the postmaster.
> Then it informed
> me that I should have dumped the damned database before I
> upgraded the
> binaries.
>
> This is like seeing the "Bridge Out" sign at the bottom of the river.
>
> Why didn't the preinstall scripts dump the database? Why
> doesn't the FAQ or
> the README in the download directory mention the required procedure?
>
> If there is going to be a conflict that might require having
> the old binaries
> around, why do the new ones replace the old ones?
> --
> In the creative leap, history waits outside.
> http://www.hacksaw.org -- http://www.privatecircus.com -- KB1FVD
>
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 6: Have you searched our list archives?
>
> http://archives.postgresql.org
>


Re: Upgrading: So now you tell me!!?!?

From
Lamar Owen
Date:
On Thursday 27 March 2003 13:30, David Olbersen wrote:
> I'm pretty sure the normal (compile, install) instructions will let you
> know to back things up.

> Yet another example of why RPMs are evil?

So that would make me, the RPM maintainer, evil, right?

If PostgreSQL was easier to upgrade it wouldn't be a problem.

And it's not RPM-specific.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: Upgrading: So now you tell me!!?!?

From
Hacksaw
Date:
> Yet another example of why RPMs are evil?

Errr... no? The only evil of RPM's is their less than delightful handling of
package dependencies.

RPM's have preinstall and postinstall scripts which allow you to do everything
needed to get ready. I've even had them be interactive.

In this case, a README.upgrade should have been in the directory where I got
the packages. It should detail precisely what I need to do before changing the
binaries. The RPM's should check the database directory, and see if the files
are in the right format, and abort the upgrade if something needs to be done
first, and the script can't do it itself.

But mainly, the information I need to know should be presented *before* I need
to act on it.

It's not a great big deal, I don't have thousands of employees dead in the
water or anything. Exactly one person is affected: me.

I'm hoping this will remain the case.

Someone at the very least needs to add a README.upgrade to every directory
where *any* packages reside, at the very least saying to make sure you run
pg_dumpall before upgrading the box, and pointing out any other files they
need to look at to complete the upgrade.

Even better, the file should be labeled README.you_will_hate_life_if_you_upgrad
e_before_reading_this.
--
Necessity is a measure of aim.
http://www.hacksaw.org -- http://www.privatecircus.com -- KB1FVD


Re: Upgrading: So now you tell me!!?!?

From
Lamar Owen
Date:
On Thursday 27 March 2003 13:17, Hacksaw wrote:
> I had an older installation of postgresql, 7.1.3. I got down the packages
> for 7.3.2. I looked at a few of the files in the directories where the
> packages were kept, and seeing no particular warnings, I ran rpm -Uvh
> postgresql*, which ran without a hitch, until I restarted the postmaster.
> Then it informed me that I should have dumped the damned database before I
> upgraded the binaries.

This has been true for over four years.  It is an interaction between
PostgreSQL's insistence on dump/restore between major versions and RPM's
insistence that no user input or output can occur during an rpm invocation.
Complain to Jeff Johnson at Red Hat about the RPM issue, and complain to
pgsql-hackers@postgresql.org about the lack of a smooth upgrade.

Find the 7.1.3 packages you had and reinstall them using rpm -Uvh
--oldpackage.  Then do a proper dump.

> This is like seeing the "Bridge Out" sign at the bottom of the river.

It isn't a good idea to blindly upgrade any package ever.  Regardless of the
normal ease.  Things break -- witness the situation with the current Wine and
the Red Hat 8.0 glibc security update.  Be prepared to back out the upgrade
by having the old packages available.

> Why didn't the preinstall scripts dump the database? Why doesn't the FAQ or
> the README in the download directory mention the required procedure?

The preinstall scripts run in a very limited environment.  There are things
they can't do; reliably dumping a database is one of the things that they are
very poor at doing.  As to the README, well, that would be my fault.  There
_is_ a README.rpm-dist file in the RPM itself, and I had previously been
putting that file in the download directory.  I did not do this for this
release.  Although, it didn't make much difference when it was in the
download directory; I still got all kinds of grief.

> If there is going to be a conflict that might require having the old
> binaries around, why do the new ones replace the old ones?

Argh.  The newest RPM's have a specific conflicts: clause against the old
version.  It is supposed to work (it worked once here, but I can't
reproduce).  Apparently RPM has a bug in the processing of the conflicts
clause where the package conflicts with previous versions of itself.

I have been in your shoes.  It is not a pleasant place to be.  But it's not
something the RPM distribution can fix, since it is endemic to PostgreSQL
itself to require a dump/initdb/restore cycle that by its nature conflicts
with the RPM upgrade scenario.  I've tried to kludge around it; the cure was
worse than the disease.  This is why I got into the mostly thankless position
of RPM maintainer.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11


Re: Upgrading: So now you tell me!!?!?

From
Hacksaw
Date:
No worries, I do know what to do in essence. One question: is there anything
*special* I need to do? I notice a mention of needing to hand edit the
dumpfiles if upgrading from 7.2 to 7.3. Is this the case here as well? Where
is the documentation on that?

Let me say that despite the problems, I support your efforts. Being a
sys-admin, I understand that mistakes are made, and I'm cool with it. I mainly
wanted to help make sure someone with less experience and more at stake
doesn't get burned.

Is there a clear marker of file format version in the database files? My
suggestion would be to have the pre-install script test for that, and abort if
it's the wrong version. Then you have the README.upgrade say something like
"After you run pg_dumpall, move /var/lib/pqsql to /var/lib/pgsql.$OLD_VERSION_N
UMBER.

That way, the preinstall will find nothing, and be able to  do initdb, and if
you are feeling really snarky, offer to try a restore for them.

--
Trust music.
http://www.hacksaw.org -- http://www.privatecircus.com -- KB1FVD


Re: Upgrading: So now you tell me!!?!?

From
Andrew Biagioni
Date:
My 2 cents:  I'm somewhat new to PostgreSQL and sysadmin'ing in general,
but I haven't had problems with installs/upgrades.

The reason:  I haven't used the RPMs, I went the "hard" route and did
builds and installs.  This, in turn, forced me to read the
install/upgrade documentation (very well written, kudos to the authors
-- very good section on system config, too) which I would never have
done if I had done an RPM-style upgrade.

Given the complexity of a DB system (especially if used for heavy work)
and the clarity of the docs, I strongly recommend anyone to scan through
them at least once...


http://www.us.postgresql.org/users-lounge/docs/7.3/postgres/admin.html

       Andrew

Hacksaw wrote:

>No worries, I do know what to do in essence. One question: is there anything
>*special* I need to do? I notice a mention of needing to hand edit the
>dumpfiles if upgrading from 7.2 to 7.3. Is this the case here as well? Where
>is the documentation on that?
>
>Let me say that despite the problems, I support your efforts. Being a
>sys-admin, I understand that mistakes are made, and I'm cool with it. I mainly
>wanted to help make sure someone with less experience and more at stake
>doesn't get burned.
>
>Is there a clear marker of file format version in the database files? My
>suggestion would be to have the pre-install script test for that, and abort if
>it's the wrong version. Then you have the README.upgrade say something like
>"After you run pg_dumpall, move /var/lib/pqsql to /var/lib/pgsql.$OLD_VERSION_N
>UMBER.
>
>That way, the preinstall will find nothing, and be able to  do initdb, and if
>you are feeling really snarky, offer to try a restore for them.
>
>
>


Re: Upgrading: So now you tell me!!?!?

From
"Steve Wolfe"
Date:
> I had an older installation of postgresql, 7.1.3. I got down the
packages for
> 7.3.2. I looked at a few of the files in the directories where the
packages
> were kept, and seeing no particular warnings, I ran rpm -Uvh
postgresql*,
> which ran without a hitch, until I restarted the postmaster. Then it
informed
> me that I should have dumped the damned database before I upgraded the
> binaries.
>
> This is like seeing the "Bridge Out" sign at the bottom of the river.
>
> Why didn't the preinstall scripts dump the database? Why doesn't the FAQ
or
> the README in the download directory mention the required procedure?

 That's the nature of the beast with RPM's, or any other package manager,
for that matter.  When you allow someone else to do the work and make the
decisions for you, you're not always guaranteed that their decisions are
going to be what you want.

  Compiling PG (and most Unix software) from source really isn't that
hard.  Some utilities rarely, if ever need to be updated, and when they
do, using RPM's is just fine.  But for things like PG, I personally feel
that you do yourself a great disservice by not compiling it yourself.  Of
course, YMMV.

steve


Re: Upgrading: So now you tell me!!?!?

From
Hacksaw
Date:
> The reason:  I haven't used the RPMs, I went the "hard" route and did
> builds and installs.  This, in turn, forced me to read the
> install/upgrade documentation (very well written, kudos to the authors

Well, I've been a Unix geek for 14 years, and a professional sysadmin for
about 6 years. I don't find the configure; make; make install route very
interesting anymore. I like package managers a great deal.

In any case, the problem is one with the RPM's, and it won't be solved by
telling everyone to compile and install by hand. RPM's are extremely important
from an advocacy point of view. If you can say "Click and install, and five
minutes later you'll have a working relational database going", people will be
interested.

So, I'm all for Lamar's work. I'm just sorry I didn't spot his address before
I posted about this publicly.
--
Artistry repeats the unrepeatable.
http://www.hacksaw.org -- http://www.privatecircus.com -- KB1FVD


Re: Upgrading: So now you tell me!!?!?

From
Hacksaw
Date:
>  That's the nature of the beast with RPM's, or any other package manager,
> for that matter.  When you allow someone else to do the work and make the
> decisions for you, you're not always guaranteed that their decisions are
> going to be what you want.

Of course not. However, RPM's can be set up to prevent disaster, and this is
what I am trying to arrange.

>   Compiling PG (and most Unix software) from source really isn't that
> hard.  Some utilities rarely, if ever need to be updated, and when they
> do, using RPM's is just fine.  But for things like PG, I personally feel
> that you do yourself a great disservice by not compiling it yourself.  Of
> course, YMMV.

Unless there are compile time variables that need to be set there is no real
advantage to compiling on your own, save for maybe getting a small speed up
because you are compiling for a particular CPU.

I'd be surprised if there were any other real advantage. This is a pretty
mature product, and not one that requires a great deal of knowledge about the
system it's going to be run on, like a package requiring a GL graphics card
and sound.

Everything after that is going to be buffer sizes and other small tweaks that
can be done to the machine or on the command line.

--
In the creative leap, history waits outside.
http://www.hacksaw.org -- http://www.privatecircus.com -- KB1FVD


Re: Upgrading: So now you tell me!!?!?

From
Gary Dunn
Date:
"Steve Wolfe" <nw@codon.com> wrote:

> I had an older installation of postgresql, 7.1.3. I got down the
packages for
> 7.3.2. I looked at a few of the files in the directories where the
packages
> were kept, and seeing no particular warnings, I ran rpm -Uvh
postgresql*,
> which ran without a hitch, until I restarted the postmaster. Then it
informed
> me that I should have dumped the damned database before I upgraded the
> binaries.
>
> This is like seeing the "Bridge Out" sign at the bottom of the river.
>
> Why didn't the preinstall scripts dump the database? Why doesn't the FAQ
or
> the README in the download directory mention the required procedure?

I'm in the same boat. The thread that followed went off into the merits of
rpms and building from source ... always fun, but not what I need right
now. My question is, how do I recover? I have a bunch of really nice data
sitting in a database I can no longer access. Is there a conversion
procedure? Do I build the previous version? How do I do that?

I am running FreeBSD 4.6.2 and installed from the ports collection using
portupgrade. The "problem" with portupgrade in this situation is that it
fully automates the build and install process. Even if the "make" command
warned about the need to dump data (and maybe it did) there was no chance
to do anything before "make install." From now on I'll exclude postgresql
from the portupgrade procedure.

Except for this one glitch I have been very happy with postgresql. Live
and learn!

--

           _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/
          _/                                     _/
         _/  Gary Dunn                          _/
        _/  Open Slate Project                 _/
       _/  http://openslate.sourceforge.net/  _/
      _/  http://www.aloha.com/~knowtree/    _/
     _/  Honolulu                           _/
    _/  registered Linux user #273809      _/
   _/                                     _/
  _/  This tagline is umop apisdn.       _/
 _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/


Re: Upgrading: So now you tell me!!?!?

From
Lamar Owen
Date:
On Friday 11 April 2003 01:30, Gary Dunn wrote:
> I'm in the same boat. The thread that followed went off into the merits of
> rpms and building from source ... always fun, but not what I need right
> now. My question is, how do I recover? I have a bunch of really nice data
> sitting in a database I can no longer access. Is there a conversion
> procedure? Do I build the previous version? How do I do that?

> I am running FreeBSD 4.6.2 and installed from the ports collection using
> portupgrade. The "problem" with portupgrade in this situation is that it
> fully automates the build and install process. Even if the "make" command
> warned about the need to dump data (and maybe it did) there was no chance
> to do anything before "make install." From now on I'll exclude postgresql
> from the portupgrade procedure.

:-O

So portupgrade does the same dance that an 'evil' RPM upgrade would...

The FreeBSD ports maintainer haunts this list (or HACKERS at least); he may be
able to help you with this.  I feel your pain (if I didn't, I'd be ROFL right
now because finally it's not just an evil RPM (or DEB) problem -- but I'm not
laughing because it is a serious issue).  It is a very serious issue, and one
that should not be addressed by packaging.  See my post in GENERAL and
HACKERS about a Red Hat Linux 9 upgrade for the full rant.

Unfortunately I forget the FreeBSD maintainer's name right now.  He should be
able to tell you how to downgrade using their system.  If all else fails, you
can get the older version's tarball and install it manually. (After removing
the new version.)  If you can, wait a few days (or directly contact the
maintainer) to se if the FreeBSD ports maintainer replies here.

If not, download the tarball from ftp.postgresql.org (not knowing what version
you had, I can't be mor specific) and install it using the standard
installation instructions.  Just don't do an initdb.
--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11