Re: postmaster growing to consume all memory - Mailing list pgsql-general

From Martijn van Oosterhout
Subject Re: postmaster growing to consume all memory
Date
Msg-id 20040127110704.GC7128@svana.org
Whole thread Raw
In response to Re: postmaster growing to consume all memory  (Lincoln Yeoh <lyeoh@pop.jaring.my>)
Responses Re: postmaster growing to consume all memory  (Tom Lane <tgl@sss.pgh.pa.us>)
List pgsql-general
On Tue, Jan 27, 2004 at 06:48:49PM +0800, Lincoln Yeoh wrote:
> There appear to be 1391110 distinct IPs (using 7.3.4). I did a vacuum full
> analyze when on 7.4 after loading the data tho.
>
> What is supposed to happen if the planner guesses right?

Aah, it seems the statistics were *way* off. It was using the new
HashAggregate code which is more efficient than the Aggregate/Group/Sort
combo when there are a small number of groups. When there are a large number
it takes a lot of memory and gets pretty slow.

I'm afraid I'll have to defer to someone else (Tom?) as why the estimate was
out by three orders of magnitude.

I'd suggest playing around with statistics and seeing if you can work out
why they were so bad.

Hope this helps,
--
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> (... have gone from d-i being barely usable even by its developers
> anywhere, to being about 20% done. Sweet. And the last 80% usually takes
> 20% of the time, too, right?) -- Anthony Towns, debian-devel-announce

Attachment

pgsql-general by date:

Previous
From: Lincoln Yeoh
Date:
Subject: Re: postmaster growing to consume all memory
Next
From:
Date:
Subject: Increase stored proc. parameters max count