On Thu, Feb 09, 2006 at 02:03:41PM -0500, Mark Woodward wrote:
> > "Mark Woodward" <pgsql@mohawksoft.com> writes:
> >> Again, regardless of OS used, hashagg will exceed "working memory" as
> >> defined in postgresql.conf.
> >
> > So? If you've got OOM kill enabled, it can zap a process whether it's
> > strictly adhered to work_mem or not. The OOM killer is entirely capable
> > of choosing a victim process whose memory footprint hasn't changed
> > materially since it started (eg, the postmaster).
>
> Sorry, I must strongly disagree here. The postgresql.conf "working mem" is
> a VERY IMPORTANT setting, it is intended to limit the consumption of
> memory by the postgresql process. Often times PostgreSQL will work along
Actually, no, it's not designed for that at all.
> side other application servers on the same system, infact, may be a
> sub-part of application servers on the same system. (This is, in fact, how
> it is used on one of my site servers.)
>
> Clearly, if the server will use 1000 times this number (Set for 1024K, but
> exceeds 1G) this is broken, and it may cause other systems to fail or
> perform very poorly.
>
> If it is not something that can be fixed, it should be clearly documented.
work_mem (integer)
Specifies the amount of memory to be used by internal sort operations and hash tables before switching to
temporarydisk files. The value is specified in kilobytes, and defaults to 1024 kilobytes (1 MB). Note that for a
complexquery, several sort or hash operations might be running in parallel; each one will be allowed to use as much
memoryas this value specifies before it starts to put data into temporary files. Also, several running sessions could
be doing such operations concurrently. So the total memory used could be many times the value of work_mem; it is
necessaryto keep this fact in mind when choosing the value. Sort operations are used for ORDER BY, DISTINCT, and
mergejoins. Hash tables are used in hash joins, hash-based aggregation, and hash-based processing of IN subqueries.
So it says right there that it's very easy to exceed work_mem by a very
large amount. Granted, this is a very painful problem to deal with and
will hopefully be changed at some point, but it's pretty clear as to how
this works.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
