Hi team,
We are using PG16.9 in Patroni Postgres setup in production. Last night we have face an issue where postgres server got crash with “Segmentation fault” on table with auto vacuum task. Interestingly in standby node while we perform vacuum on same table standby node was also got crash. The table was not able to get query on the table as while executing select statement also crash the database.
we have observed few error prior to crash for same table. (Table name and function details change for purpose)
"ERROR : Error occurred at function get_details page 117 of relation ""impacted_table"" should be empty but is not"
During the same time the other table also getting below error. However once the database restart, we were not getting any issue for table oid= 1108029
“ERROR : Error occurred at function get_details unexpected data beyond EOF in block 16276 of relation base/33195/1108029"
At last it got failed as follows
2025-10-15 02:50:52.428 [432443]LOG: terminating any other active server processes"
2025-10-15 02:50:52.428 [432443]DETAIL: Failed process was running: autovacuum: VACUUM ANALYZE schema.impacted_table"
2025-10-15 02:50:52.428 [432443]LOG: server process (PID 390906) was terminated by signal 11: Segmentation fault"
2025-10-15 02:50:55.475 [432443]LOG: all server processes terminated; reinitializing"
2025-10-15 02:51:32.575 [432443]LOG: received immediate shutdown request"
2025-10-15 02:51:32.629 [432443]LOG: database system is shut down"
The function which was executing having truncate table and insert/update statement executing on this table. As this is normally functionality but we are suspecting it during the execution runtime it corrupt the shared memory and data inflight which got replicated to Replica node/DR site and corrupt the same table.
We had to drop the table “impacted_table” from database once the database is up.
however we are not able to identify exact root cause behind “segmentation fault” error for this table and need expert advice to find the root case and also need suggestions to prevention steps.
Thanks & Regards,
-------------------------
Ishan Joshi
The information transmitted herein is intended only for the person or entity to which it is addressed and may contain confidential, proprietary and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
