Re: FATAL: semctl(15073290, 4, SETVAL, 0) failed: Invalid argument - Mailing list pgsql-general

On 03/20/2018 11:57 AM, JotaComm wrote:
> ​Hello,
> 
> Today I found this message in my Postgres log:
> 
> FATAL:  semctl(15073290, 4, SETVAL, 0) failed: Invalid argument
> user=,db=,app=,host= LOG:  server process (PID 30741) exited with exit 
> code 1
> user=,db=,app=,host= LOG:  terminating any other active server processes
> WARNING:  terminating connection because of crash of another server process
> DETAIL:  The postmaster has commanded this server process to roll back 
> the current transaction and exit, because another server process exited 
> abnormally and possibly corrupted shared memory.
> HINT:  In a moment you should be able to reconnect to the database and 
> repeat your command.
> 
> Postgres: PostgreSQL 9.6.3 on x86_64-pc-linux-gnu, compiled by gcc 
> (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609, 64-bit
> 
> Linux: Linux INVST-APP-01A 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 
> 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
> 
> FATAL:  semctl(15073290, 4, SETVAL, 0) failed: Invalid argument
> user=,db=,app=,host= LOG:  server process (PID 30741) exited with exit 
> code 1
> user=,db=,app=,host= LOG:  terminating any other active server processes
> WARNING:  terminating connection because of crash of another server process
> DETAIL:  The postmaster has commanded this server process to roll back 
> the current transaction and exit, because another server process exited 
> abnormally and possibly corrupted shared memory.
> HINT:  In a moment you should be able to reconnect to the database and 
> repeat your command.
> 
> Any suggestions to discovery about this behavior?

?:
https://www.postgresql.org/docs/10/static/kernel-resources.html
"
18.4.2. systemd RemoveIPC

If systemd is in use, some care must be taken that IPC resources (shared 
memory and semaphores) are not prematurely removed by the operating 
system. This is especially of concern when installing PostgreSQL from 
source. Users of distribution packages of PostgreSQL are less likely to 
be affected, as the postgres user is then normally created as a system user.

The setting RemoveIPC in logind.conf controls whether IPC objects are 
removed when a user fully logs out. System users are exempt. This 
setting defaults to on in stock systemd, but some operating system 
distributions default it to off.

A typical observed effect when this setting is on is that the semaphore 
objects used by a PostgreSQL server are removed at apparently random 
times, leading to the server crashing with log messages like

     LOG: semctl(1234567890, 0, IPC_RMID, ...) failed: Invalid argument

Different types of IPC objects (shared memory vs. semaphores, System V 
vs. POSIX) are treated slightly differently by systemd, so one might 
observe that some IPC resources are not removed in the same way as 
others. But it is not advisable to rely on these subtle differences.

A “user logging out” might happen as part of a maintenance job or 
manually when an administrator logs in as the postgres user or something 
similar, so it is hard to prevent in general.

What is a “system user” is determined at systemd compile time from the 
SYS_UID_MAX setting in /etc/login.defs.

Packaging and deployment scripts should be careful to create the 
postgres user as a system user by using useradd -r, adduser --system, or 
equivalent.

Alternatively, if the user account was created incorrectly or cannot be 
changed, it is recommended to set

RemoveIPC=no

in /etc/systemd/logind.conf or another appropriate configuration file.
Caution

At least one of these two things has to be ensured, or the PostgreSQL 
server will be very unreliable.
"

> 
> Thanks a lot.
> 
> Best regards
> 
> -- 
> JotaComm
> http://jotacomm.wordpress.com


-- 
Adrian Klaver
adrian.klaver@aklaver.com