Re: Pre-allocation of shared memory ... - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Pre-allocation of shared memory ...
Date
Msg-id 9431.1055626709@sss.pgh.pa.us
Whole thread Raw
In response to Re: Pre-allocation of shared memory ...  ("Andrew Dunstan" <andrew@dunslane.net>)
List pgsql-hackers
"Andrew Dunstan" <andrew@dunslane.net> writes:
> I *know* the latest RH kernel docs *say* they have paranoid mode that
> supposedly guarantees against OOM - it was me that pointed that out
> originally :-). I just checked on the latest sources (today it's RH8, kernel
> 2.4.20-18.8) to be doubly sure, and can't see the patches. (That would be
> really bad of RH, btw, if I'm correct - saying in your docs you support
> something that you don't)

I tried a direct test on my RHL 8.0 box, and was able to prove that
indeed the overcommit 2/3 modes do something, though whether they work
exactly as documented is another question.

I wrote this silly little test program to get an approximate answer
about the largest amount a program could malloc:

#include <stdio.h>
#include <stdlib.h>

int
main (int argc, char **argv)
{ size_t min = 1024;        /* assume this'd work */ size_t max = -1;        /* = max unsigned */ size_t sz; void
*ptr;
 while ((max - min) >= 1024ul) {   sz = (((unsigned long long) max) + ((unsigned long long) min)) / 2;   ptr =
malloc(sz);  if (ptr) {     free(ptr);
 
//      printf("malloc(%lu) succeeded\n", sz);     min = sz;   } else {
//      printf("malloc(%lu) failed\n", sz);     max = sz;   } }
 printf("Max malloc is %lu Kb\n", min / 1024);
 return 0;
}

and got these results:

[root@rh1 tmp]# echo 0 > /proc/sys/vm/overcommit_memory
[root@rh1 tmp]# ./alloc
Max malloc is 1489075 Kb
[root@rh1 tmp]# echo 1 > /proc/sys/vm/overcommit_memory
[root@rh1 tmp]# ./alloc
Max malloc is 2063159 Kb
[root@rh1 tmp]# echo 2 > /proc/sys/vm/overcommit_memory
[root@rh1 tmp]# ./alloc
Max malloc is 1101639 Kb
[root@rh1 tmp]# echo 3 > /proc/sys/vm/overcommit_memory
[root@rh1 tmp]# ./alloc
Max malloc is 974179 Kb

So it's definitely doing something.  /proc/meminfo shows
       total:    used:    free:  shared: buffers:  cached:
Mem:  261042176 160456704 100585472        0 72015872 63344640
Swap: 1077501952 44974080 1032527872
MemTotal:       254924 kB
MemFree:         98228 kB
MemShared:           0 kB
Buffers:         70328 kB
Cached:          59244 kB
SwapCached:       2616 kB
Active:         102532 kB
Inact_dirty:     11644 kB
Inact_clean:     21840 kB
Inact_target:    27200 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       254924 kB
LowFree:         98228 kB
SwapTotal:     1052248 kB
SwapFree:      1008328 kB
Committed_AS:    77164 kB

It does appear that the limit in mode 3 is not too far from where
you'd expect (SwapTotal - Committed_AS), and mode 2 allows about
128M more, which is correct since there's 256 M of RAM.
        regards, tom lane


pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Pre-allocation of shared memory ...
Next
From: "Andrew Dunstan"
Date:
Subject: Re: Pre-allocation of shared memory ...