Thread: Out of memory: postgres killed after changing "checkoint_completion_target" and reloading postgres configuration file

Hi,

 

Problem :

------------

Dec  4 10:42:41 “myserver” kernel: [7604991.147522] Out of memory: Kill process 2904 (postgres) score 217 or sacrifice child

How to cause the error:

-----------------------------

I changed the parameter “checkpoint_completion_target” from 0.5 to 0.9 in the postgresql.conf file unless the production is up and used by multiple users. After that I made a postgres reload the 4th dec 2014 at 10:20AM

 

(Note that same problem has occurred several weeks ago when I changed “checkpoint_segment” from 3 to 10 in the postgresql.conf file after reloading postgres)

 

Application that are running

-----------------------------------

AS JBoss  7.1.1.Final with a connection pool

Postgres version running in production

------------------------------------------------

PostgreSQL 9.1.8 on x86_64-unknown-linux-gnu, compiled by gcc-4.4.real (Debian 4.4.5-8) 4.4.5, 64-bit

 

OS Server production

--------------------------

Linux “myserver” 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux


The following memories value was done today:

postgres@”myserver”:/var/log/postgresql$ free -m

                     total        used         free        shared    buffers     cached

Mem:         32115      29498       2617          0            110              20128

-/+ buffers/cache:       9258      22857

Swap:            0          0          0

ð  No swap allowed in this server

 

Observation:

----------------

The database seems to run well from the 4th dec between 10:20AM to 10:42 AM before falling back suddently to out of memory.
It switched to recovery mode and restart automatically.

There were no problem encountered like that before making any change to the checkpoint parameters.

 

Attached files

-----------------

Syslog file and postgresql.log  file are attached. I made some clean up in the postgres log file.

 

Question:

------------

Is it not recommended to change any “checkpoint_xxx” parameters in the postgresql.conf file when the production is running ?

 

Kind Regards,

SLANIA Pierre

Attachment
On 12/08/2014 12:47 PM, Pierre Slania wrote:
> Problem :
> ------------
> Dec  4 10:42:41 "myserver" kernel: [7604991.147522] Out of memory: Kill process 2904 (postgres) score 217 or
sacrificechild 
>
> How to cause the error:
> -----------------------------
> I changed the parameter "checkpoint_completion_target" from 0.5 to 0.9 in the postgresql.conf file unless the
productionis up and used by multiple users. After that I made a postgres reload the 4th dec 2014 at 10:20AM 
>
> (Note that same problem has occurred several weeks ago when I changed "checkpoint_segment" from 3 to 10 in the
postgresql.conffile after reloading postgres) 
>
> Application that are running
> -----------------------------------
> AS JBoss  7.1.1.Final with a connection pool
>
> Postgres version running in production
> ------------------------------------------------
> PostgreSQL 9.1.8 on x86_64-unknown-linux-gnu, compiled by gcc-4.4.real (Debian 4.4.5-8) 4.4.5, 64-bit
>
> OS Server production
> --------------------------
> Linux "myserver" 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux
>
> The following memories value was done today:
>
> postgres@"myserver":/var/log/postgresql$ free -m
>                       total        used         free        shared    buffers     cached
> Mem:         32115      29498       2617          0            110              20128
> -/+ buffers/cache:       9258      22857
> Swap:            0          0          0
>
> ð  No swap allowed in this server
>
> Observation:
> ----------------
> The database seems to run well from the 4th dec between 10:20AM to 10:42 AM before falling back suddently to out of
memory.
> It switched to recovery mode and restart automatically.
> There were no problem encountered like that before making any change to the checkpoint parameters.
>
> Attached files
> -----------------
> Syslog file and postgresql.log  file are attached. I made some clean up in the postgres log file.
>
> Question:
> ------------
> Is it not recommended to change any "checkpoint_xxx" parameters in the postgresql.conf file when the production is
running? 

There's no such recommendation. It should be totally fine to change
those parameters. I don't know why you got the OOM error, but I don't
see changing checkpoint settings could cause that.

- Heikki