Thread: Re: [BUGS] Rapid deteriation of performance (might be caused by

On Fri, 4 Apr 2003, Robert John Shepherd wrote:

> Up until a few days ago I have been running Postgresl 7.2.3 with Tsearch
> from the contrib dir, but at various times the performance of the
> database would suddenly and rapidly deteriate so that queries which
> previously took 500ms then took 8 or 9 seconds.

Hmm, what are the before and after explain analyze results?  Also, what
are your conf settings for shared buffers, sort memory and the fsm
parameters?

> The only cure is a backup and restore of the database, vacuuming and
> analysing does nothing. I even tried rebuilding all indexes once which
> didn't seem to help.

Did you do a regular vacuum or vacuum full?  If only the former, it's
possible that you need to either vacuum more frequently and/or raise the
free space map settings in your configuration file.

What does vacuum full verbose <table>; give you for the tables involved?

> Help! (And sorry if this is the wrong list)
pgsql-performance is a better list, so I've replied to there.  You'll
probably need to join in order to reply to list.

> > Up until a few days ago I have been running Postgresl 7.2.3 with
Tsearch
> > from the contrib dir, but at various times the performance of the
> > database would suddenly and rapidly deteriate so that queries which
> > previously took 500ms then took 8 or 9 seconds.

> Hmm, what are the before and after explain analyze results?  Also,
what
> are your conf settings for shared buffers, sort memory and the fsm
> parameters?

shared_buffers = 40960
sort_mem = 20480
#max_fsm_relations = 1000
#max_fsm_pages = 10000

As you can see I've not uncommented or touched the fsm parameters, I
have no idea what they do. Optimisation wise I have only played with
shared_buffers, sort_mem and max_connections.


> > The only cure is a backup and restore of the database

> Did you do a regular vacuum or vacuum full?  If only the former, it's
> possible that you need to either vacuum more frequently and/or raise
the
> free space map settings in your configuration file.

I've been running this daily:

    vacuumdb -h localhost -a -z

Should I be using the full switch then?

I'll get back to you on the other questions if you think they are still
needed.


> pgsql-performance is a better list, so I've replied to there.  You'll
> probably need to join in order to reply to list.

Thanks, especially for not shouting at me heh, this is stressful enough
as it is.


Yours Unwhettedly,
Robert John Shepherd.

Editor
DVD REVIEWER
The UK's BIGGEST Online DVD Magazine
http://www.dvd.reviewer.co.uk

For a copy of my Public PGP key, email: pgp@robertsworld.org.uk

On Fri, 4 Apr 2003, Robert John Shepherd wrote:

> > > Up until a few days ago I have been running Postgresl 7.2.3 with
> Tsearch
> > > from the contrib dir, but at various times the performance of the
> > > database would suddenly and rapidly deteriate so that queries which
> > > previously took 500ms then took 8 or 9 seconds.
>
> > Hmm, what are the before and after explain analyze results?  Also,
> what
> > are your conf settings for shared buffers, sort memory and the fsm
> > parameters?
>
> shared_buffers = 40960
> sort_mem = 20480
> #max_fsm_relations = 1000
> #max_fsm_pages = 10000
>
> As you can see I've not uncommented or touched the fsm parameters, I
> have no idea what they do. Optimisation wise I have only played with
> shared_buffers, sort_mem and max_connections.
>
>
> > > The only cure is a backup and restore of the database
>
> > Did you do a regular vacuum or vacuum full?  If only the former, it's
> > possible that you need to either vacuum more frequently and/or raise
> the
> > free space map settings in your configuration file.
>
> I've been running this daily:
>
>     vacuumdb -h localhost -a -z
>
> Should I be using the full switch then?

Well, you generally shouldn't need to if the fsm settings are high enough.
If you're doing really big updates like update each row of a 1 billion
row table, you may end up having to do one immediately following that.
Of course, if you're doing that, performance is probably not your biggest
concern. ;)

Explain analyze'll tell us if the system is changing plans (presumably to
a worse one) - for example, deciding to move to a sequence scan because it
thinks that the index scan is now to expensive, or conversely moving to an
index scan because it thinks that there'll be too many reads, while those
page reads actually are fairly localized. The vacuum full verbose should
get some idea of how much empty space is there.

> > pgsql-performance is a better list, so I've replied to there.  You'll
> > probably need to join in order to reply to list.
>
> Thanks, especially for not shouting at me heh, this is stressful enough
> as it is.

:)

> > I've been running this daily:
> >     vacuumdb -h localhost -a -z
> > Should I be using the full switch then?
>
> Well, you generally shouldn't need to if the fsm settings are high
enough.
> If you're doing really big updates like update each row of a 1 billion
> row table, you may end up having to do one immediately following that.
> Of course, if you're doing that, performance is probably not your
biggest
> concern. ;)

Not doing that, no. ;)


> Explain analyze'll tell us if the system is changing plans (presumably
to
> a worse one)

It wasn't, oddly enough.

I've added a new table that cuts down 85% of the work this query has to
do, and it seems to have helped an awful lot at the moment. Of course
only time will tell. :)

Thanks for the suggestions.


Yours Unwhettedly,
Robert John Shepherd.

Editor
DVD REVIEWER
The UK's BIGGEST Online DVD Magazine
http://www.dvd.reviewer.co.uk

For a copy of my Public PGP key, email: pgp@robertsworld.org.uk