Thread: Memory Utilization Issue

Memory Utilization Issue

From
Sachin Srivastava
Date:

Dear Concern,

 

Always my server memory utilization is remain >99%. I have 4 DB server and RAM of the server is (32 GB, 64 GB, 64 GB and 132 GB). In every server always we are getting the memory utilization > 99%. Kindly suggest why this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux server. Kindly find the "TOP" result, "ulimit -a"  result, ("kernel.shmax", "kernel.shmall", "kernel.sem" value) and pg_setting result of one server as below. If you require any other information then inform to me.

 

 

[root@CPPMOMA_DB01 ~]# top

top - 12:08:08 up 15 days, 15:45,  5 users,  load average: 1.05, 1.19, 1.35

Tasks: 605 total,   1 running, 604 sleeping,   0 stopped,   0 zombie

Cpu(s):  6.9%us,  2.1%sy,  0.0%ni, 78.3%id, 12.7%wa,  0.0%hi,  0.1%si,  0.0%st

Mem:  32832364k total, 32621168k used,   211196k free,    77572k buffers


[root@CPPMOMA_DB01 ~]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 256323
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 256323
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@CPPMOMA_DB01 ~]#


kernel.shmmax = 32212254720

kernel.shmall = 1073741824

kernel.sem = 250 32000 100 384

 

postgres=# SELECT name, source, setting FROM pg_settings WHERE source != 'default' AND source != 'override' ORDER by 2, 1;

            name            |        source        |             setting

----------------------------+----------------------+----------------------------------

 application_name           | client               | psql.bin

 client_encoding            | client               | UTF8

 archive_command            | configuration file   | cp %p /dbkup/momacpp_213_live/%f

 archive_mode               | configuration file   | on

 autovacuum                 | configuration file   | on

 autovacuum_max_workers     | configuration file   | 3

 checkpoint_segments        | configuration file   | 200

 checkpoint_timeout         | configuration file   | 300

 checkpoint_warning         | configuration file   | 30

 DateStyle                  | configuration file   | ISO, MDY

 default_text_search_config | configuration file   | pg_catalog.english

 effective_cache_size       | configuration file   | 524288

 lc_messages                | configuration file   | en_US.UTF-8

 lc_monetary                | configuration file   | en_US.UTF-8

 lc_numeric                 | configuration file   | en_US.UTF-8

 lc_time                    | configuration file   | en_US.UTF-8

 listen_addresses           | configuration file   | *

 log_destination            | configuration file   | stderr

 log_directory              | configuration file   | pg_log

 logging_collector          | configuration file   | on

 log_line_prefix            | configuration file   | %t

 log_rotation_age           | configuration file   | 1440

 maintenance_work_mem       | configuration file   | 1638400

 max_connections            | configuration file   | 2000

 max_files_per_process      | configuration file   | 2000

 max_wal_senders            | configuration file   | 5

 port                       | configuration file   | 5432

 shared_buffers             | configuration file   | 1572864

 temp_buffers               | configuration file   | 4096

 wal_level                  | configuration file   | archive

 work_mem                   | configuration file   | 32768

 log_timezone               | environment variable | Asia/Kolkata

 max_stack_depth            | environment variable | 2048

 TimeZone                   | environment variable | Asia/Kolkata

(34 rows)

 

postgres=#

 

 

 

Regards,

Sachin Srivastava
Assistant Technical Lead(Oracle/PostgreSQL)    | TSG
Cyient
| www.cyient.com

Re: Memory Utilization Issue

From
Albe Laurenz
Date:
Sachin Srivastava wrote:
> Always my server memory utilization is remain >99%. I have 4 DB server and RAM of the server is (32
> GB, 64 GB, 64 GB and 132 GB). In every server always we are getting the memory utilization > 99%.
> Kindly suggest why this problem is and which parameter will resolve this problem.
> 
> I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux server. Kindly find the
> "TOP" result, "ulimit -a"  result, ("kernel.shmax", "kernel.shmall", "kernel.sem" value) and
> pg_setting result of one server as below. If you require any other information then inform to me.

That sounds just fine.
Linux uses memory for the file system cache.  That memory is shown as "in use", but
it is available for processes if they need it.

It doesn't look like your machine is swapping.

Do you experience problems?

Yours,
Laurenz Albe

Re: Memory Utilization Issue

From
Mathew Moon
Date:
What is the output of 'free -m' ? Look at the third column second row (+/- cache). This is the most relevant number. Even if you are swapping that can happen with plenty of RAM available if 'swappiness' is set too low. 

Sent from my iPhone

On May 20, 2015, at 2:25 AM, Sachin Srivastava <ssr.teleatlas@gmail.com> wrote:

Dear Concern,

 

Always my server memory utilization is remain >99%. I have 4 DB server and RAM of the server is (32 GB, 64 GB, 64 GB and 132 GB). In every server always we are getting the memory utilization > 99%. Kindly suggest why this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux server. Kindly find the "TOP" result, "ulimit -a"  result, ("kernel.shmax", "kernel.shmall", "kernel.sem" value) and pg_setting result of one server as below. If you require any other information then inform to me.

 

 

[root@CPPMOMA_DB01 ~]# top

top - 12:08:08 up 15 days, 15:45,  5 users,  load average: 1.05, 1.19, 1.35

Tasks: 605 total,   1 running, 604 sleeping,   0 stopped,   0 zombie

Cpu(s):  6.9%us,  2.1%sy,  0.0%ni, 78.3%id, 12.7%wa,  0.0%hi,  0.1%si,  0.0%st

Mem:  32832364k total, 32621168k used,   211196k free,    77572k buffers


[root@CPPMOMA_DB01 ~]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 256323
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 256323
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@CPPMOMA_DB01 ~]#


kernel.shmmax = 32212254720

kernel.shmall = 1073741824

kernel.sem = 250 32000 100 384

 

postgres=# SELECT name, source, setting FROM pg_settings WHERE source != 'default' AND source != 'override' ORDER by 2, 1;

            name            |        source        |             setting

----------------------------+----------------------+----------------------------------

 application_name           | client               | psql.bin

 client_encoding            | client               | UTF8

 archive_command            | configuration file   | cp %p /dbkup/momacpp_213_live/%f

 archive_mode               | configuration file   | on

 autovacuum                 | configuration file   | on

 autovacuum_max_workers     | configuration file   | 3

 checkpoint_segments        | configuration file   | 200

 checkpoint_timeout         | configuration file   | 300

 checkpoint_warning         | configuration file   | 30

 DateStyle                  | configuration file   | ISO, MDY

 default_text_search_config | configuration file   | pg_catalog.english

 effective_cache_size       | configuration file   | 524288

 lc_messages                | configuration file   | en_US.UTF-8

 lc_monetary                | configuration file   | en_US.UTF-8

 lc_numeric                 | configuration file   | en_US.UTF-8

 lc_time                    | configuration file   | en_US.UTF-8

 listen_addresses           | configuration file   | *

 log_destination            | configuration file   | stderr

 log_directory              | configuration file   | pg_log

 logging_collector          | configuration file   | on

 log_line_prefix            | configuration file   | %t

 log_rotation_age           | configuration file   | 1440

 maintenance_work_mem       | configuration file   | 1638400

 max_connections            | configuration file   | 2000

 max_files_per_process      | configuration file   | 2000

 max_wal_senders            | configuration file   | 5

 port                       | configuration file   | 5432

 shared_buffers             | configuration file   | 1572864

 temp_buffers               | configuration file   | 4096

 wal_level                  | configuration file   | archive

 work_mem                   | configuration file   | 32768

 log_timezone               | environment variable | Asia/Kolkata

 max_stack_depth            | environment variable | 2048

 TimeZone                   | environment variable | Asia/Kolkata

(34 rows)

 

postgres=#

 

 

 

Regards,

Sachin Srivastava
Assistant Technical Lead(Oracle/PostgreSQL)    | TSG
Cyient
| www.cyient.com

Re: Memory Utilization Issue

From
Naveed Shaikh
Date:
Could you also please check the Transparent huge page(THP) are enabled on the server or not, they can also result in intermittent poor performance along with high system cpu time counted against the database processes.

This can be confirmed by below command:

cat /sys/kernel/mm/redhat_transparent_hugepage/enabled

Here "always" is selected, showing THP are enabled, this needs to be disable with following command:

echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled

Thanks & Regards,
Naveed Shaikh





On Wed, May 20, 2015 at 12:55 PM, Sachin Srivastava <ssr.teleatlas@gmail.com> wrote:

Dear Concern,

 

Always my server memory utilization is remain >99%. I have 4 DB server and RAM of the server is (32 GB, 64 GB, 64 GB and 132 GB). In every server always we are getting the memory utilization > 99%. Kindly suggest why this problem is and which parameter will resolve this problem.

I am using PostgreSQL 9.1.2 and PostGIS 1.5 version on every Red hat Linux server. Kindly find the "TOP" result, "ulimit -a"  result, ("kernel.shmax", "kernel.shmall", "kernel.sem" value) and pg_setting result of one server as below. If you require any other information then inform to me.

 

 

[root@CPPMOMA_DB01 ~]# top

top - 12:08:08 up 15 days, 15:45,  5 users,  load average: 1.05, 1.19, 1.35

Tasks: 605 total,   1 running, 604 sleeping,   0 stopped,   0 zombie

Cpu(s):  6.9%us,  2.1%sy,  0.0%ni, 78.3%id, 12.7%wa,  0.0%hi,  0.1%si,  0.0%st

Mem:  32832364k total, 32621168k used,   211196k free,    77572k buffers


[root@CPPMOMA_DB01 ~]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 256323
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 256323
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
[root@CPPMOMA_DB01 ~]#


kernel.shmmax = 32212254720

kernel.shmall = 1073741824

kernel.sem = 250 32000 100 384

 

postgres=# SELECT name, source, setting FROM pg_settings WHERE source != 'default' AND source != 'override' ORDER by 2, 1;

            name            |        source        |             setting

----------------------------+----------------------+----------------------------------

 application_name           | client               | psql.bin

 client_encoding            | client               | UTF8

 archive_command            | configuration file   | cp %p /dbkup/momacpp_213_live/%f

 archive_mode               | configuration file   | on

 autovacuum                 | configuration file   | on

 autovacuum_max_workers     | configuration file   | 3

 checkpoint_segments        | configuration file   | 200

 checkpoint_timeout         | configuration file   | 300

 checkpoint_warning         | configuration file   | 30

 DateStyle                  | configuration file   | ISO, MDY

 default_text_search_config | configuration file   | pg_catalog.english

 effective_cache_size       | configuration file   | 524288

 lc_messages                | configuration file   | en_US.UTF-8

 lc_monetary                | configuration file   | en_US.UTF-8

 lc_numeric                 | configuration file   | en_US.UTF-8

 lc_time                    | configuration file   | en_US.UTF-8

 listen_addresses           | configuration file   | *

 log_destination            | configuration file   | stderr

 log_directory              | configuration file   | pg_log

 logging_collector          | configuration file   | on

 log_line_prefix            | configuration file   | %t

 log_rotation_age           | configuration file   | 1440

 maintenance_work_mem       | configuration file   | 1638400

 max_connections            | configuration file   | 2000

 max_files_per_process      | configuration file   | 2000

 max_wal_senders            | configuration file   | 5

 port                       | configuration file   | 5432

 shared_buffers             | configuration file   | 1572864

 temp_buffers               | configuration file   | 4096

 wal_level                  | configuration file   | archive

 work_mem                   | configuration file   | 32768

 log_timezone               | environment variable | Asia/Kolkata

 max_stack_depth            | environment variable | 2048

 TimeZone                   | environment variable | Asia/Kolkata

(34 rows)

 

postgres=#

 

 

 

Regards,

Sachin Srivastava
Assistant Technical Lead(Oracle/PostgreSQL)    | TSG
Cyient
| www.cyient.com