Re: PG performance in high volume environment (many INSERTs and lots of aggregation reporting) - Mailing list pgsql-performance

From Merlin Moncure
Subject Re: PG performance in high volume environment (many INSERTs and lots of aggregation reporting)
Date
Msg-id b42b73150901280955r7f1ba554l323c84dc50696bfb@mail.gmail.com
Whole thread Raw
In response to PG performance in high volume environment (many INSERTs and lots of aggregation reporting)  (Phoenix Kiula <phoenix.kiula@gmail.com>)
List pgsql-performance
On 1/28/09, Phoenix Kiula <phoenix.kiula@gmail.com> wrote:
> [Ppsted similar note to PG General but I suppose it's more appropriate
>  in this list. Apologies for cross-posting.]
>
>  Hi. Further to my bafflement with the "count(*)" queries as described
>  in this thread:
>
>  http://archives.postgresql.org/pgsql-general/2009-01/msg00804.php
>
>  It seems that whenever this question has come up, Postgresql comes up
>  very short in terms of "count(*)" functions.
>
>  The performance is always slow, because of the planner's need to guess
>  and such. I don't fully understand how the statistics work (and the
>  explanation on the PG website is way too geeky) but he columns I work
>  with already have a stat level of 100. Not helping at all.

Your issue is not statistics/planner.  postgres just can't apply the
special case optimization that some other database do because of the
locking model.

all planner's 'guess'.  the main goal of statistics is to make the
guess better educated.

>  We are now considering a web based logging functionality for users of
>  our website. This means the table could be heavily INSERTed into. We
>  get about 10 million hits a day, and I'm guessing that we will have to
>  keep this data around for a while.

10m hits/day = 115 hits/sec.  This is no problem for even workstation
box assuming your disks can handle the syncs.  however, with extreme
insert heavy loads it helps alot to look at partitioning/rotation to
ease the pain of big deletes.

merlin

pgsql-performance by date:

Previous
From: Richard Huxton
Date:
Subject: Re: PG performance in high volume environment (many INSERTs and lots of aggregation reporting)
Next
From: Scott Marlowe
Date:
Subject: Re: [PERFORMANCE] Buying hardware