Re: [GENERAL] problems with memory - Mailing list pgsql-general

From selkovjr@mcs.anl.gov
Subject Re: [GENERAL] problems with memory
Date
Msg-id 200002190008.SAA11095@mail.xnet.com
Whole thread Raw
In response to Re: [GENERAL] problems with memory  (Alfred Perlstein <bright@wintelcom.net>)
List pgsql-general
> * Dean Browett <dbrowett@bizonline.net> [000218 15:55] wrote:
> > Hi,
> >
> > We are running postgres-6.5.3 on a dual pentium 300 machine, 0.5Gb RAM under
> > Linux Redhat 6.0 (kernel 2.2.14). The machine we are using sits on a 100Mb
> > network and the nics are 3com3c590's. We are also using a DPT Raid
> > controller in a raid5 configuration set up as 1 logical drive.
> >
> > We are try to insert a large amount of data into the database. What happens
> > is that when we first start loading data everything is fine. Over a period
> > of time (1.5hrs) there is a marked decrease in performance in terms of both
> > memory and cpu usage. At this time cpu usage has crept up to 45-50% and
> > memory usage is 100Mb and rising slowly and there is only one connection to
> > the database.
> >
> > All the statistics are gleaned from using 'top'.
> >
> >
> > Consequently, the database gets slower and slower until it loads at the rate
> > of <3KBps at which point it becomes more exciting to watch paint dry 8-)).
> >
> > Postgres is the only program running (except for normal system programs)
> > that uses a significant amount of memory.
> >
> > Is Postgres known to leak memory? What causes the high cpu usage?
> >
> > Any ideas would be greatly appreciated.
>
> You really haven't given very much information on the rules and constraints
> in your tables, one problem that I had was that a constraint on a table
> of mine caused extreme slowdown because each row inserted needed to be
> validated through a constraint, as the table grew the amount of data that
> needed to be scanned for each insert grew exponentially.
>
> hope this helps,
> -Alfred

The same is true of any type of index. Make sure you don't have
indices defined unitl after the insert. Also, do a COPY instead of
INSERT if possible.

--Gene

pgsql-general by date:

Previous
From: Alfred Perlstein
Date:
Subject: Re: [GENERAL] problems with memory
Next
From: Ed Loehr
Date:
Subject: Re: [GENERAL] newbie Q's