Re: Postgre Eating Up Too Much RAM - Mailing list pgsql-admin
| From | Aaron Bono |
|---|---|
| Subject | Re: Postgre Eating Up Too Much RAM |
| Date | |
| Msg-id | CAHfMse2_9yV0-3NSLiAWdy7vu+_NWRt_B7Li7tirTNHViKB40g@mail.gmail.com Whole thread Raw |
| In response to | Re: Postgre Eating Up Too Much RAM (Tom Lane <tgl@sss.pgh.pa.us>) |
| Responses |
Re: Postgre Eating Up Too Much RAM
|
| List | pgsql-admin |
On our old server, our hosting company said the server was running out of RAM and then became unresponsive. I haven't checked about the new server yet.
==================================================================
Aaron Bono
Aranya Software Technologies, Inc.
http://www.aranya.com
http://codeelixir.com
==================================================================
I noticed our problems started about the time when we loaded a new client into the database that had nearly 1 GB of large files stored in BLOBs - PDFs, images, Word docs, etc. We have a daily process that pulls these large files out for search indexing. It is likely the new server crashed at about the time this indexing was taking place.
When we are reading large files out of the BLOBs (upwards of 100 MB a piece), could that cause Postgres to eat up the RAM that remains? With a server having 32 GB RAM I would think only two database connections (that should be all that the processes use for the indexing) would NOT have this effect.
I am glad to see I am not totally missing something obvious but am also a bit flummoxed over this issue. With this server upgrade I changed OS (CentOS to Ubuntu), upgraded Postgres (8.3 to 8.4), increased the RAM (6 GB to 32 GB), increased the hard drive space (1/2 TB to over 1.5 TB on a RAID 10), changed to completely new hardware, removed a ton of stuff on the server we didn't need (like CPanel and its baggage) and even had our daily backups turned off temporarily. In fact, the old server was lasting 2 days or more before having problems and with the new server it went belly up in just a day.
Is there any kind of diagnostics you can think of that would help get to the root of the problem - something I could put in a cron job or a monitor app I could run on the server that would at least tell us what is going on if / when it happens again?
Thanks for all the help!
Aaron
==================================================================
Aaron Bono
Aranya Software Technologies, Inc.
http://www.aranya.com
http://codeelixir.com
==================================================================
On Tue, Nov 13, 2012 at 9:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Craig Ringer <craig@2ndQuadrant.com> writes:
> On 11/14/2012 06:12 AM, Aaron Bono wrote:>> Am I reading this right? Are there individual connections using overYeah. Since Aaron's got shared_buffers set to 256MB, the shared memory
>> 300 MB or RAM by themselves?
> If I recall correctly, RSS is charged against a PostgreSQL back-end when
> it touches `shared_buffers`. So that doesn't necessarily mean that the
> back-end is using the full amount of memory listed as RSS.
segment is something more than that (maybe 270-280MB, hard to be sure
without checking). The RSS numbers probably count all or nearly all of
that for each process, but of course there's really only one copy of the
shared memory segment. RSS is likely double-counting the postgres
executable as well, which means that the actual additional memory used
per process is probably just a few meg, which is in line with most
folks' experience with PG.
The "free" stats didn't look like a machine under any sort of memory
pressure --- there's zero swap usage, and nearly half of real RAM is
being used for disk cache, which means the kernel can find no better
use for it than caching copies of disk files. Plus there's still 10G
that's totally free. Maybe things get worse when the machine's been up
longer, but this sure isn't evidence of trouble.
I'm inclined to think that the problem is not RAM consumption at all but
something else. What exactly happens when the server "hangs"?
regards, tom lane
--
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin
pgsql-admin by date: