Re: out of memory errors - Mailing list pgsql-general

I was reading in to the parameter a little more and it appears that the
defuault for vm.overcommit_ratio is 50%, I am considering bumping this
up to 95% so the sums look like this:

max memory allocation for process = swap + ratio of physical memory

21 + (16 * 0.95) = 36.2GB

This in theory should always leave me with roughly 1GB of free physical
memory, swap may be blown though :) (if my understanding of this
parameter is correct).

What I dont understand is, even at its default, the overcommit ratio is
50% of physical, which would make it 21GB + 8GB, ending up at around
29GB (which looks about right in the meminfo output below), so, assuming
my understanding is correct:

     [1] How can an analyze process run out of memory on this setting if
it is asking for, at most, maintenance_work_mem (plus some overhead) 128MB
     [2] How can a new connection run out of memory, I presume work_mem
+ some overhead, I'm guessing around 2MB memory?

I'm beginning to wonder if my issue is somewhere else now.

Thanks for the tip though at looking at vm.overcommit_ratio, I obvisouly
overlooked this setting when setting vm.overcommit_memory = 2

Any other pointers would be greatly appreciated :)

Reference:

https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-captun.html

Thanks
Bruce

On 16/06/2014 14:21, Bruce McAlister wrote:
> Hi,
>
> On 16/06/2014 14:15, Andres Freund wrote:
>> Hi,
>>
>> On 2014-06-16 13:56:23 +0100, Bruce McAlister wrote:
>>>      [1] 3 x ESX VM's
>>>                  [a] 8 vCPU's each
>>>                  [b] 16GB memory each
>>> # Dont hand out more memory than neccesary
>>> vm.overcommit_memory = 2
>> So you haven't tune overcommit_ratio at all? Can you show
>> /proc/meminfo's contents?
>> My guess is that the CommitLimit is too low...
>>
>
> No I have not tune overcommit_ratio.
>
> Below is the /proc/meminfo contents. One note though, the database is
> currently not running on this node, just in case i need to make some
> changes that require a restart.
>
> [root@bfievdb01 heartbeat]# cat /proc/meminfo
> MemTotal:       16333652 kB
> MemFree:         2928544 kB
> Buffers:          197216 kB
> Cached:          1884032 kB
> SwapCached:            0 kB
> Active:          4638780 kB
> Inactive:        1403676 kB
> Active(anon):    4006088 kB
> Inactive(anon):     7120 kB
> Active(file):     632692 kB
> Inactive(file):  1396556 kB
> Unevictable:       65004 kB
> Mlocked:           56828 kB
> SwapTotal:      22015984 kB
> SwapFree:       22015984 kB
> Dirty:              3616 kB
> Writeback:             0 kB
> AnonPages:       4026228 kB
> Mapped:            82408 kB
> Shmem:             45352 kB
> Slab:             197052 kB
> SReclaimable:     106804 kB
> SUnreclaim:        90248 kB
> KernelStack:        4000 kB
> PageTables:        15172 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:    30182808 kB
> Committed_AS:    4342644 kB
> VmallocTotal:   34359738367 kB
> VmallocUsed:     7004496 kB
> VmallocChunk:   34352726816 kB
> HardwareCorrupted:     0 kB
> AnonHugePages:   3868672 kB
> HugePages_Total:       0
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
> DirectMap4k:       10240 kB
> DirectMap2M:    16766976 kB
>
> Thanks
> Bruce