Thread: reindexing

reindexing

From
"LARC/J.L.Shipman/jshipman"
Date:
Hi,
    I am reindexing my 7.1.4 postgres database.  The postmaster seems to
create processes for each reindex request.  Is there any way to find
out more about the processes.

ps -aef | grep postgres

yields the following, but does not tell me which table is being
reindexed or anything meaningful
about the process.


postgres   605   604   0   Feb 18 ?           0:00 /usr/local/pgsql/
bin/postmaster
postgres  5599   579   0   Feb 21 ?          39:12 /usr/local/pgsql/
bin/postmaster
postgres 20101   579   0 10:56:52 ?           0:58 /usr/local/pgsql/
bin/postmaster
postgres   579     1   0   Feb 18 ?           0:02 /usr/local/pgsql/
bin/postmaster
postgres   604   579   0   Feb 18 ?           0:00 /usr/local/pgsql/
bin/postmaster
   dspace 21563 21391   0 13:27:00 pts/3       0:00 grep postgres
postgres  5645   579   0   Feb 21 ?          35:29 /usr/local/pgsql/
bin/postmaster
postgres 19695   579   0 10:13:22 ?           2:51 /usr/local/pgsql/
bin/postmaster
postgres 19713   579   0 10:15:02 ?           2:43 /usr/local/pgsql/
bin/postmaster
postgres  7441   579   0   Feb 21 ?          33:49 /usr/local/pgsql/
bin/postmaster
postgres 19963   579   0 10:42:25 ?           1:43 /usr/local/pgsql/
bin/postmaster
postgres 19658   579   0 10:09:56 ?           2:52 /usr/local/pgsql/
bin/postmaster
postgres 19981   579   0 10:44:43 ?           2:20 /usr/local/pgsql/
bin/postmaster
postgres  6276   579   0   Feb 21 ?          39:12 /usr/local/pgsql/
bin/postmaster
postgres 19667   579   0 10:10:56 ?           2:25 /usr/local/pgsql/
bin/postmaster
postgres  5654   579   0   Feb 21 ?          36:36 /usr/local/pgsql/
bin/postmaster
postgres  5657   579  20   Feb 21 ?          33:06 /usr/local/pgsql/
bin/postmaster
postgres  5656   579   0   Feb 21 ?          39:17 /usr/local/pgsql/
bin/postmaster
postgres  6216   579   0   Feb 21 ?          31:02 /usr/local/pgsql/
bin/postmaster
postgres  7508   579   0   Feb 21 ?          29:03 /usr/local/pgsql/
bin/postmaster
postgres 20159   579   0 11:03:25 ?           2:22 /usr/local/pgsql/
bin/postmaster
postgres  6275   579   0   Feb 21 ?          35:12 /usr/local/pgsql/
bin/postmaster
postgres  7474   579   0   Feb 21 ?          32:07 /usr/local/pgsql/
bin/postmaster
postgres 19884   579   0 10:33:52 ?           1:38 /usr/local/pgsql/
bin/postmaster
postgres  5655   579   0   Feb 21 ?          35:42 /usr/local/pgsql/
bin/postmaster
postgres 20100   579   0 10:56:43 ?           2:04 /usr/local/pgsql/
bin/postmaster
postgres  5598   579   0   Feb 21 ?          40:22 /usr/local/pgsql/
bin/postmaster
postgres 20259   579   0 11:15:33 ?           2:04 /usr/local/pgsql/
bin/postmaster
postgres 19696   579  19 10:13:57 ?           2:07 /usr/local/pgsql/
bin/postmaster
postgres  7509   579   0   Feb 21 ?          34:43 /usr/local/pgsql/
bin/postmaster
postgres 19946   579   0 10:40:11 ?           1:12 /usr/local/pgsql/
bin/postmaster
postgres 20006   579   0 10:47:06 ?           2:17 /usr/local/pgsql/
bin/postmaster
postgres  6258   579   0   Feb 21 ?          40:08 /usr/local/pgsql/
bin/postmaster


Re: reindexing

From
"Scott Marlowe"
Date:
On Fri, Feb 22, 2008 at 12:24 PM, LARC/J.L.Shipman/jshipman
<Jeffery.L.Shipman@nasa.gov> wrote:
> Hi,
>         I am reindexing my 7.1.4 postgres database.  The postmaster seems to
>  create processes for each reindex request.  Is there any way to find
>  out more about the processes.
>
>  ps -aef | grep postgres
>
>  yields the following, but does not tell me which table is being
>  reindexed or anything meaningful
>  about the process.

My pgsql-fu regarding obsolete versions is obsolete.  You do realize
that 7.1.x hasn't been supported for a very long time, and for very
good reasons, right?  It's not just obsolete in terms of being
outdated by more modern versions, but is known to have a few data
eating bugs, not to mention the txid wraparound issue.  You should be
planning on how to upgrade it first, then things like this might be
less necessary and / or less of a problem to work with.

In later versions of pgsql you've got a stats collector that can do
things like tell you what queries are running.  As well as
autovacuuming and non-full vacuums that make things like reindexing
mostly uneeded.

Re: reindexing

From
Tom Lane
Date:
"Scott Marlowe" <scott.marlowe@gmail.com> writes:
> On Fri, Feb 22, 2008 at 12:24 PM, LARC/J.L.Shipman/jshipman
> <Jeffery.L.Shipman@nasa.gov> wrote:
>> I am reindexing my 7.1.4 postgres database.

> My pgsql-fu regarding obsolete versions is obsolete.  You do realize
> that 7.1.x hasn't been supported for a very long time, and for very
> good reasons, right?

There never was a 7.1.4 release, so I suspect the OP meant 7.4.1
... not that that speaks very much better for his software maintenance
habits.  Even with the more charitable interpretation, it's a version
that was obsoleted four years ago next week.

            regards, tom lane

Re: reindexing

From
Lew
Date:
Tom Lane wrote:
> There never was a 7.1.4 release, so I suspect the OP meant 7.4.1
> .... not that that speaks very much better for his software maintenance
> habits.  Even with the more charitable interpretation, it's a version
> that was obsoleted four years ago next week.

In my experience at various "big-iron" shops (government agencies, large
health-care organizations and the like), four years is not a long time for
enterprise software - a version often has to be at least four years old before
the powers-that-be decide to try it.  One has only to look at how many
organizations still use Oracle 8, or Java 1.3, for example, to see how
conservative many shops are with respect to upgrades.

I'm not saying they should be that conservative, but many organizations are
and we must be ready to deal with that.

--
Lew

Re: reindexing

From
paul rivers
Date:
Lew wrote:
> Tom Lane wrote:
>> There never was a 7.1.4 release, so I suspect the OP meant 7.4.1
>> .... not that that speaks very much better for his software maintenance
>> habits.  Even with the more charitable interpretation, it's a version
>> that was obsoleted four years ago next week.
>
> In my experience at various "big-iron" shops (government agencies,
> large health-care organizations and the like), four years is not a
> long time for enterprise software - a version often has to be at least
> four years old before the powers-that-be decide to try it.  One has
> only to look at how many organizations still use Oracle 8, or Java
> 1.3, for example, to see how conservative many shops are with respect
> to upgrades.
>
> I'm not saying they should be that conservative, but many
> organizations are and we must be ready to deal with that.
>

This is completely the opposite of my experience at a very large global
financial company.  They are extremely concerned with staying current,
and in fact audit regulations require it for any software not written
in-house.  If they were still running Oracle 8, for example, they would
fail internal audit precisely because it is no longer a supported Oracle
version, and thus security and such patches are no longer available.
The same would go for operating system patches, firmware, whatever.  The
release cycle does tend to be slower (from quarterly to yearly) for,
say, things like AIX or z/OS or DB2, but updates are coming out
routinely [including security and bug fixes, as well as feature
additions], and in my experience these shops are definitely keeping up.

The only places I've had direct experience with that tend to run very
old versions of things are doing so for all the wrong reasons.  They
seem to be learning, albeit slowly and painfully, the demerits of not
keeping current.

Just my $0.02,
Paul


Re: reindexing

From
Tom Lane
Date:
Lew <lew@lwsc.ehost-services.com> writes:
> Tom Lane wrote:
>> There never was a 7.1.4 release, so I suspect the OP meant 7.4.1
>> .... not that that speaks very much better for his software maintenance
>> habits.  Even with the more charitable interpretation, it's a version
>> that was obsoleted four years ago next week.

> In my experience at various "big-iron" shops (government agencies, large
> health-care organizations and the like), four years is not a long time for
> enterprise software - a version often has to be at least four years old before
> the powers-that-be decide to try it.  One has only to look at how many
> organizations still use Oracle 8, or Java 1.3, for example, to see how
> conservative many shops are with respect to upgrades.

This is not equivalent to "still using Oracle 8".  This is "still using
Oracle 8 and we haven't applied any of Oracle's updates for it".  Is it
even possible for a shop to do that?  I can hardly believe that Oracle
would honor a support contract for a version that's missing four years
worth of bug fixes.

As for the "not wanting to adopt too quickly" argument, why'd they adopt
7.4.1 in the first place?  If you're of the view that no software is
acceptably stable till it's been out a couple years, you should be using
something with a minor number rather higher than 1.

            regards, tom lane

Re: reindexing

From
Greg Smith
Date:
On Wed, 27 Feb 2008, Lew wrote:

> One has only to look at how many organizations still use Oracle 8, or
> Java 1.3, for example, to see how conservative many shops are with
> respect to upgrades.  I'm not saying they should be that conservative,
> but many organizations are and we must be ready to deal with that.

Companies that act so conversatively are already getting nailed by lack of
support in the public versions of software.  For example, in 2007 DST was
moved around in the US for no good reason, requiring an update to the
Olson Timezone Database.  If you're a Java user, and you're on 1.3, you
couldn't get that update unless you have a support contract--the free
version won't do it.  (ref:
http://java.sun.com/javase/timezones/DST_faq.html )  Even there only a
small subset of platforms are supported.  Getting older Oracle versions to
work there obviously requires the appropriate support contract to see the
Metalink update, and as I can tell only 8.1 was updated, people running
8.0 were left out.

If some big-iron shop who is so blind to security issues that they want to
keep 7.4 on life support, they certainly can find someone to deliver such
a support agreement on a contract basis.  But they shouldn't expect the
public project to keep them afloat for free, and saying this project "must
be ready" to handle them is quite debatable.  Given the limited resources
of the public volunteers here, supporting ancient versions is a drain it's
hard to justify outside the context of such a support agreement.  Using
your own examples, Oracle and Sun sure don't, why should PostgreSQL?

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

Re: reindexing

From
"Joshua D. Drake"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 27 Feb 2008 11:28:32 -0500
Lew <lew@lwsc.ehost-services.com> wrote:

> In my experience at various "big-iron" shops (government agencies,
> large health-care organizations and the like), four years is not a
> long time for enterprise software - a version often has to be at
> least four years old before the powers-that-be decide to try it.  One
> has only to look at how many organizations still use Oracle 8, or
> Java 1.3, for example, to see how conservative many shops are with
> respect to upgrades.

Yes but Tom wasn't talking about upgrades. He was talking about
maintenance. You can bet that any respectable enterprise shop is at
least running the latest service packs for the respective releases. The
community does support 7.4 still. However the version that is supported
is service release (or service pack) 19. Thus 7.4.19.

> 
> I'm not saying they should be that conservative, but many
> organizations are and we must be ready to deal with that.
> 

And we already do, far more than we should IMO. This idea that the
volunteer community should somehow provide enterprise class support is
a non starter. That is what the companies surrounding the community
are for. If companies want the community and not the companies
surrounding the community to provide that kind of support, those
companies need to start paying for it.


Sincerely,

Joshua D. Drake
- -- 
The PostgreSQL Company since 1997: http://www.commandprompt.com/ 
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director |  PostgreSQL political pundit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHxbU/ATb/zqfZUUQRAt1hAJ9NCAK6xTQtF6hcI95rwolqlNpXoACdHIoJ
IVfVRiN5PTpwxAWH6ohY1us=
=L7ov
-----END PGP SIGNATURE-----

Re: reindexing

From
Tom Lane
Date:
Greg Smith <gsmith@gregsmith.com> writes:
> If some big-iron shop who is so blind to security issues that they want to
> keep 7.4 on life support, they certainly can find someone to deliver such
> a support agreement on a contract basis.  But they shouldn't expect the
> public project to keep them afloat for free, and saying this project "must
> be ready" to handle them is quite debatable.

Well, whether we *must* do it or not is arguable; but the point in this
thread is that we *do* do it.  The 7.4 branch is up to 7.4.19.  But the
OP was (apparently) still running 7.4.1, and his failure to take
advantage of that free support was what I was lecturing him about ;-)

            regards, tom lane

Re: reindexing

From
Lew
Date:
Greg Smith wrote:
> If some big-iron shop who is so blind to security issues that they want
> to keep 7.4 on life support, they certainly can find someone to deliver
> such a support agreement on a contract basis.  But they shouldn't expect
> the public project to keep them afloat for free, and saying this project
> "must be ready" to handle them is quite debatable.  Given the limited
> resources of the public volunteers here, supporting ancient versions is
> a drain it's hard to justify outside the context of such a support
> agreement.  Using your own examples, Oracle and Sun sure don't, why
> should PostgreSQL?

I am not arguing that Postgres, Oracle, Sun or anyone else should have to
support such obsolete products, or that they are the only source for that
support.  I only state the fact that many organizations are slow to move off
even obsolete products - this is something I have observed more than once in
more than one contract.  I only claimed in my post that "we must be ready to
deal with that", since it is a fact, not that vendors should have to support
those products for free.

For example, in my work I deal with that by strongly urging my clients not to
use obsolete software, after explaining that the software in question is
actually obsolete.  They don't always agree with my recommendation, then I
deal with that in turn.  It's not like they make me their decision maker.

I agree that no one should have to support obsolete products for free, and
that these organizations should upgrade.

--
Lew