Thread: Query response time

Query response time

From

"Jonathan Sinclair"

Date:

08 August 2006, 12:24:59

Hi all,

I am using PostgresSQL 7.4 and having some serious performance issues.
Trawling through the archives and previous posts the only visible advice
I could see was either by running vacuum or setting the fsynch flag to
false.

I am using tables that only contain approx 2GB of data. However
performing a number of simple conditional select statements takes a
great deal of time. Putting limits on the data obviously reduces the
time, but there is still a delay. (Note: on one particular query I set
the limit to 538 and the query returns in under 2mins if the limit
becomes 539 the query loops indefinitely!)
From previous experience I know these delays are longer than both
Informix and MySql. In some instances it takes so long I end up having
to kill the query.

The install was performed by yum onto a RAID server using Centos. I am
sure there is something fundamentally wrong for I can't believe that
postgres would have the reputation it has based on the statistics I'm
getting. Does anyone have any advice?

The data I am using was imported from an Informix system as part of a
migration strategy.
I know this is long shot but I hope someone can shed some light.

Regards,

Jonathan

This email may contain information which is privileged or confidential. This information is intended only for the named
recipient.If you are not the intended recipient, please be aware that disclosure, copying, distribution or use of this
informationis prohibited. If you have received this email in error, we would be grateful if you would inform us as soon
aspossible by telephoning +44 (0) 1769 573431, or by email to admin@molevalleyfarmers.com and then delete this email.
Viewsor opinions expressed in this email are those of the writer, and are not necessarily the views of Mole Valley
FarmersLimited or its subsidiary companies. Unless specifically stated, this email does not constitute any part of an
offeror contract.

Re: Query response time

From

Chris Mair

Date:

08 August 2006, 12:48:16

> I am using PostgresSQL 7.4 and having some serious performance issues.
> Trawling through the archives and previous posts the only visible advice
> I could see was either by running vacuum or setting the fsynch flag to
> false.
> 
> I am using tables that only contain approx 2GB of data. However
> performing a number of simple conditional select statements takes a
> great deal of time. Putting limits on the data obviously reduces the
> time, but there is still a delay. (Note: on one particular query I set
> the limit to 538 and the query returns in under 2mins if the limit
> becomes 539 the query loops indefinitely!)
> >From previous experience I know these delays are longer than both
> Informix and MySql. In some instances it takes so long I end up having
> to kill the query.

Please run the command "analyze" on your database (you can do that with
the psql shell), then post to this list:

- your query
- the output of the command "explain + your query"
- the layout of the tables concerned by the query ("\d tablename" from psql)


> The install was performed by yum onto a RAID server using Centos. I am
> sure there is something fundamentally wrong for I can't believe that
> postgres would have the reputation it has based on the statistics I'm
> getting. Does anyone have any advice?

> The data I am using was imported from an Informix system as part of a
> migration strategy.
> I know this is long shot but I hope someone can shed some light.

If you're on pgsql-performance@postgresql.org as well, we might continue
discussion there.

Bye :)
Chris.


-- 

Chris Mair
http://www.1006.org

Re: Query response time

From

Richard Broersma Jr

Date:

08 August 2006, 12:53:43

> I am using PostgresSQL 7.4 and having some serious performance issues.
> Trawling through the archives and previous posts the only visible advice
> I could see was either by running vacuum or setting the fsynch flag to
> false.
> 
> I am using tables that only contain approx 2GB of data. However
> performing a number of simple conditional select statements takes a
> great deal of time. Putting limits on the data obviously reduces the
> time, but there is still a delay. (Note: on one particular query I set
> the limit to 538 and the query returns in under 2mins if the limit
> becomes 539 the query loops indefinitely!)
> From previous experience I know these delays are longer than both
> Informix and MySql. In some instances it takes so long I end up having
> to kill the query.
> 
> The install was performed by yum onto a RAID server using Centos. I am
> sure there is something fundamentally wrong for I can't believe that
> postgres would have the reputation it has based on the statistics I'm
> getting. Does anyone have any advice?
> 
> The data I am using was imported from an Informix system as part of a
> migration strategy.
> I know this is long shot but I hope someone can shed some light.

Are the Update/Insert queries slow or is it the select queries that are taking awhile?

For select queries, an explain analyze of the offending query would be helpful. Also, in addition
to vacuuming you may want to reindex you db in order to clean all of the dead tuples from your
indexs.

For heavy insert/update queries check your postgres logs to see if any messages suggest increasing
your check-point-segments.

If this is the case, try increasing you check_point_segments and try moving your pg_xlog to a
different spindle.

Regards,

Richard Broersma Jr.

Re: Query response time

From

Roman Neuhauser

Date:

09 August 2006, 15:25:02

# jonathan.sinclair@molevalleyfarmers.com / 2006-08-08 13:25:47 +0100:
> I am using PostgresSQL 7.4 and having some serious performance issues.

> I am using tables that only contain approx 2GB of data.

> The install was performed by yum onto a RAID server using Centos. I am
> sure there is something fundamentally wrong for I can't believe that
> postgres would have the reputation it has based on the statistics I'm
> getting. Does anyone have any advice?
   You'll need to configure PostrgreSQL, the default settings seem   to be tuned so that PostgreSQL starts even in very
limited  environments.
 
   http://www.postgresql.org/docs/8.1/static/runtime-config.html

-- 
How many Vietnam vets does it take to screw in a light bulb?
You don't know, man.  You don't KNOW.
Cause you weren't THERE.             http://bash.org/?255991

Re: Query response time

From

"Jonathan Sinclair"

Date:

11 August 2006, 13:58:00

Hi all. Thanks for your help so far. However after configuring my system
I am still getting major lag times with a couple of queries. The first,
which suffers from the '538/539'(see old email included below) bug, is
caused by running the following statement:

SELECT t1.col1, SUM(test) test_col, SUM(col2)
FROM table1 tl, table2 t2
WHERE t1.date BETWEEN '01/10/2004' AND '01/10/2004'
AND t1.col3 = t2.col1
AND t1.col5 = t2.col2
AND t2.col3 BETWEEN 50.00 AND 200.00
GROUP BY t1.col1
HAVING SUM(test) BETWEEN 95.00 AND 100.00
ORDER BY 2 DESC, t1.col1;

I would like to know if anyone has any ideas why this problem arises.
(It's not due to the date format being ambiguous; I have set it to
European standard)

A second statement I have prepared has the same problem but this time
works fine with a limit of 2 but breaks as soon as I set the limit to 3.
The offending statement is:

SELECT t1.col1, t1.col2, t2.col2, t3.col2
FROM table1 t1, table2 t2, table3 t3,
WHERE t1.date BETWEEN '01/10/2004' AND '01/10/2004'
AND t3.date = t1.date + 365
AND t2.date = t1.date + 14
AND t1.col1 = t2.col1
AND t1.col1 = t3.col1
And t1.col1 <> ''
AND t2.col2 < t1.col2;

Both queries work for limited limit sets. But break at different points
i.e. enter infinite loops?
Does anyone have any ideas? Both queries process fine in Informix!

I have run vacuum analyze and full commands on both tables! This didn't
solve the problem.

Regards,

Jonathan

-----Original Message-----

Hi all,

From previous experience I know these delays are longer than both
Informix and MySql. In some instances it takes so long I end up having
to kill the query.

The data I am using was imported from an Informix system as part of a
migration strategy.

I know this is long shot but I hope someone can shed some light.

Regards,

Jonathan

Re: Query response time

From

Scott Marlowe

Date:

15 August 2006, 13:50:01

On Fri, 2006-08-11 at 08:58, Jonathan Sinclair wrote:
> Hi all. Thanks for your help so far. However after configuring my system
> I am still getting major lag times with a couple of queries. The first,
> which suffers from the '538/539'(see old email included below) bug, is
> caused by running the following statement:
> 
> SELECT t1.col1, SUM(test) test_col, SUM(col2)
> FROM table1 tl, table2 t2
> WHERE t1.date BETWEEN '01/10/2004' AND '01/10/2004'
> AND t1.col3 = t2.col1
> AND t1.col5 = t2.col2
> AND t2.col3 BETWEEN 50.00 AND 200.00
> GROUP BY t1.col1
> HAVING SUM(test) BETWEEN 95.00 AND 100.00
> ORDER BY 2 DESC, t1.col1;
> 
> I would like to know if anyone has any ideas why this problem arises.
> (It's not due to the date format being ambiguous; I have set it to
> European standard)

Have you looked at the plan for this query?

explain select...

If I remember correctly, the planner has a bug where if you do a between
with the same date, it does a seq scan instead of using an index.  But
my memory on that's a bit rough.

How does it work if you change the where clause to be "t1.date =
'01/10/2004'???

P.s. this is more of a performance than a SQL question.  Not that that's
a big deal or anything, the SQL list  is kinda slow and can likely use
the traffic. :)