Thread: Memory issues

On Mon, Sep 24, 2012 at 12:45 AM, Shiran Kleiderman <shirank1@gmail.com> wrote:
>
>
> Hi,
> I'm using and Amazon ec2 instance with the following spec and the
> application that I'm running uses a postgres DB 9.1.
> The app has 3 main cron jobs.
>
> Ubuntu 12, High-Memory Extra Large Instance
> 17.1 GB of memory
> 6.5 EC2 Compute Units (2 virtual cores with 3.25 EC2 Compute Units each)
> 420 GB of instance storage
> 64-bit platform
>
> I've changed the main default values under file postgresql.conf to:
> shared_buffers = 4GB
> work_mem = 16MB
> wal_buffers = 16MB
> checkpoint_segments = 32
> effective_cache_size = 8GB
>
> When I run the app, after an hour or two, free -m looks like below ans the
> crons can't run due to memory loss or similar (i'm new to postgres and db
> admin).
> Thanks!
>
> free -m, errors:
>
> total used free shared buffers cached
> Mem: 17079 13742 3337 0 64 11882
> -/+ buffers/cache: 1796 15283
> Swap: 511 0 511

You have 11.8G cached, that's basically free memory on demand.

> total used free shared buffers cached
> Mem: 17079 16833 245 0 42 14583
> -/+ buffers/cache: 2207 14871
> Swap: 511 0 511

Here you have 14.5G cached, again that's free memory so to speak.
I.e. when something needs it it gets allocated.

> **free above stays low even when nothing is running.
>
>
> **errors:
> DBI connect('database=---;host=localhost','postgres',...) failed: could not
> fork new process for connection: Cannot allocate memory
> could not fork new process for connection: Cannot allocate memory

This error is happening in your client process.  Maybe it's 32 bit or
something and running out of local memory in its process space? Maybe
memory is so fragmented that no large blocks can get allocated or
something?  Either way, your machine has plenty of memory according to
free.  BTW, it's pretty common for folks new to unix to mis-read free
and not realize that cached memory + free memory is what's really
available.

On Tue, Sep 25, 2012 at 7:00 PM, Shiran Kleiderman <shirank1@gmail.com> wrote:
>
> Hi
> Thanks for your answer.
> I understood that the server is ok memory wise.
> What can I check on the client side or the DB queries?

Well you're connecting to localhost so I'd expect you to show a memory
issue in free I'm not seeing.  Are you really connecting to localhost
or not?

On Monday, September 24, 2012 08:45:06 AM Shiran Kleiderman wrote:
> Hi,
> I'm using and Amazon ec2 instance with the following spec and the
> application that I'm running uses a postgres DB 9.1.
> The app has 3 main cron jobs.
>
> *Ubuntu 12, High-Memory Extra Large Instance
> 17.1 GB of memory
> 6.5 EC2 Compute Units (2 virtual cores with 3.25 EC2 Compute Units each)
> 420 GB of instance storage
> 64-bit platform*
>
> I've changed the main default values under file *postgresql.conf* to:
> shared_buffers = 4GB
> work_mem = 16MB
> wal_buffers = 16MB
> checkpoint_segments = 32
> effective_cache_size = 8GB
>
> When I run the app, after an hour or two, free -m looks like below ans the
> crons can't run due to memory loss or similar (i'm new to postgres and db
> admin).
> Thanks!
>
> free -m, errors:
>
> total used free shared buffers cached
> Mem: 17079 13742 3337 0 64 11882
> -/+ buffers/cache: 1796 15283
> Swap: 511 0 511
>
> total used *free* shared buffers cached
> Mem: 17079 16833 *245 *0 42 14583
> -/+ buffers/cache: 2207 14871
> Swap: 511 0 511
>
> **free above stays low even when nothing is running.
>
> **errors:
> *DBI connect('database=---;host=localhost','postgres',...) failed: could
> not fork new process for connection: Cannot allocate memory*
> could not fork new process for connection: Cannot allocate memory
>
> and
> execute failed: ERROR: out of memory
> DETAIL: Failed on request of size 968. [for Statement "
> SELECT DISTINCT....
could you show cat /proc/meminfo?

Greetings,

Andres
--
 Andres Freund                       http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

On 10/14/12 5:52 PM, Alexander Gataric wrote:
>
> I'm on a project which requires adding PostgreSQL tables to DB2
> Federated Server. I'm getting an error with PostgreSQL data types
> boolean, text, bytea, and XML. I believe this can be solved with the
> CREATE TYPE MAPPING in Fed Server. Does anyone know which values to
> use? I'm not that familiar with Fed Server.
>
> Also, the Postgres data is being extracted and inserted from the same
> table using Optim Archive. Does this pose an additional challenge with
> setting up the mapping?
>

I suggest you talk to your IBM support contacts for these issues, they
really have little to do with Postgres and are completely beyond
postgres' control.



--
john r pierce                            N 37, W 122
santa cruz ca                         mid-left coast