Re: 7.1 Release Date - Mailing list pgsql-general

Brook Milligan wrote:

>    We NEED this 'pg_upgrade'-on-steroids program that simply Does The Right
>    Thing.

>    I think it's high time that the dump/initdb/restore cycle needs to be
>    retired as a normal upgrading step.

> YOU (i.e., people relying on the RH stuff to do everything at once)
> may need such a thing, but it seems like you are overstating the case
> just a bit.  If this project gets adopted by core developers, it would
> seem to conflict drastically with the goal of developing the core
> functionality.  Thus, it's not quite "high time" for this.

Does a dump/restore from 6.5.3 to 7.1.0 properly handle large objectcs
yet?  (I know Philip Warner is working on it -- but that is NOT going to
help the person running and old version wanting to upgrade).  I would
dare say that there are more users of PostgreSQL running on RedHat than
all other platforms combined.

That's fine -- if PostgreSQL doesn't want to cater to newbies who simply
want it to work, then someone else will cater to them.  Personally, I
believe the 'newbie niche' is one of many niches that PostgreSQL fills
very effectively -- until the hapless newbie upgrades his OS and trashes
his database in the process.  Then he goes and gets someone else's
database and badmouths PostgreSQL. (as to those other niches, I benched
my OpenACS installation yesterday at 10.5 pages per second -- where each
page involved 7-10 SQL queries -- with a concurrent load of 50
connections.  PostgreSQL's speed and scalability are major benefits --
it's relative ease of installation and administration are other major
benefits).

Education is nice -- but, tell me, first of all, how is the newbie to
find it?  Release notes that don't get put on the disk until it's
already too late to do a proper dump/restore?  Sure, old hands at
PostgreSQL know the drill -- I know to uncheck PostgreSQL during OS
upgrades.  But, even that doesn't help if the new version of the OS
can't run the old version's binaries.

This is not the first time I've mentioned this -- nor is it the first
time it has been called into question.  This upgrading issue is already
wearing thin at RedHat (or didn't you notice Trond's message) -- it
would not surprise me in the least to see PostgreSQL dropped from the
RedHat distribution in favor of InterBase or MySQL if this issue isn't
fixed for 7.1.  Sure, it's their loss -- unless you actually want
PostgreSQL to be more popular, which I would like.  Even if RedHat drops
PostgreSQL, I'm likely to remain with it -- at least until InterBase's
AOLserver driver is up to par, and OpenACS is fully ported over to
InterBase.  Well, even then I'll likely remain with PostgreSQL, as it
works, I know it (relatively well), and the development community is
great to work with.

> first place before trashing their system.  If the person lacks such a
> clue, the solution is education (e.g., make the analogy explicit, show
> the tools required, make pg_dump more robust, ...) not redirecting the
> precious resources of core developers to duplicate the database system
> in a standalone program for upgrades.

No one outside the PostgreSQL developer community understands why it is
such an issue to require dump/restore at _every_single_ minor update --
ooops, sorry, major update where their minor is our major.  Or, to put
it differently -- mysql doesn't have this problem.  Sure, mysql has
plenty of problems, but this isn't one of them.

Did you also miss where I'm willing to do the legwork myself?  I'm to
that point of aggravation over this -- but, then again, I get the 100+
emails a week about the RPM set, and I get the ire of newbies who are
dumbfounded that they have to be _that_careful_ during updates. Maybe I
_am_ a little too vehement over this -- but, I am not alone.  I know
Trond shares my frustration -- amongst others.

Just how long would such a program take to write, anyway?  Probably not
nearly as long as you might suspect, as all such a program is is a
translator, taking input in one format and rewriting it to another
format. You just have to know what to translate and how to translate --
there are details of course (such as pg_log handling), but the basics
are already coded in the existing backends of the many versions. There's
no SQL parsing or executing to deal with -- just reading in one format
and writing in another.

In fact, you would only need to support upgrades from 9 versions (1.01,
1.09, 6.0, 6.1, 6.2, 6.3, 6.4, 6.5,7.0) to make this work -- and some of
those versions have the same binary format (am I right on that, Tom,
Bruce, Thomas, or Vadim?).  IIRC, the binary format changed at 6.5 -- so
you basically have pre-6.5 and post-6.5 data to worry with, as the other
changes that require the dump/initdb/restore are system catalog issues,
right?  Since the new pg_upgrade would do an initdb as part of its
operation (in the new directory), the old system catalogs will only have
to be read for certain things, I would think.

Comments?

If we don't do it, someone else will. Yes, maybe I overstated the issue
-- unless you agree that RedHat's continued distribution of PostgreSQL
is a good thing.

If such a program were already written, wouldn't you use it, Brook?

--
Lamar Owen
WGCR Internet Radio
1 Peter 4:11