I have a 8.3.6 postgres database running on Arch Linux (2.6.28 kernel)
with the following entry in pg_locks:
locktype | database | relation | page | tuple | virtualxid |
transactionid | classid | objid | objsubid | virtualtransaction |
pid | mode | granted
relation | 16385 | 16427 | | |
| | | | | 54/0 |
10453 | ShareUpdateExclusiveLock | t
That pid is 10453. The weird part is that there is no process running
on this machine with that pid.
I had to bounce an OpenMQ broker this morning (this database is the DB
for an OpenMQ HA setup) and couldn't get it to reconnect to postgres.
On inspecting the database I found dozens of vacuum processes waiting
(I have a cron job that vacuums each night) and chewing up connection
slots. Killing those left a few autovacuum worker process waiting.
Killing those left just this one orphaned pid apparently holding a
lock. Assumably they were all waiting on the lock "held" by 10453.
The database continues to function normally, but when I (or the
autovacuum process) attempts to vacuum or analyze I get this message:
"WARNING: could not send signal to process 10453: No such process"
It can't kill it because that process/pid does not exist.
Additionally if I "vacuum verbose" the vacuum does seem to run to
completion before waiting forever after issuing that warning. I have
tried killing the autovacuum launcher process and letting it restart
but still as soon as the next vacuum is issued it gets blocked and
waits.
Is there any way for me to clear that orphaned entry out of pg_locks?
What could I have done to cause this?
I have used postgres for everything including our main product
database (hundreds of transactions/sec, 100's of GB of data) for years
and have never seen this scenario.
Any help is appreciated, I can easily provide any additional
information that may be helpful.
Joe Uhl
