Re: issue with gininsert under very high load - Mailing list pgsql-hackers

From Tom Lane
Subject Re: issue with gininsert under very high load
Date
Msg-id 19237.1392237218@sss.pgh.pa.us
Whole thread Raw
In response to Re: issue with gininsert under very high load  (Andres Freund <andres@2ndquadrant.com>)
Responses Re: issue with gininsert under very high load  (Andres Freund <andres@2ndquadrant.com>)
List pgsql-hackers
Andres Freund <andres@2ndquadrant.com> writes:
> On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote:
>> On investigation I found that a number of processes were locked waiting for
>> one wedged process to end its transaction, which never happened (this
>> transaction should normally take milliseconds). oprofile revealed that
>> postgres was spending 87% of its time in s_lock(), and strace on the wedged
>> process revealed that it was in a tight loop constantly calling select(). It
>> did not respond to a SIGTERM.

> That's a deficiency of the gin fastupdate cache: a) it bases it's size
> on work_mem which usually makes it *far* too big b) it doesn't perform the
> cleanup in one go if it can get a suitable lock, but does independent
> locking for each entry. That usually leads to absolutely horrific
> performance under concurreny.

I'm not sure that what Andrew is describing can fairly be called a
concurrent-performance problem.  It sounds closer to a stuck lock.
Are you sure you've diagnosed it correctly?
        regards, tom lane



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: Recovery inconsistencies, standby much larger than primary
Next
From: Andrew Dunstan
Date:
Subject: Re: issue with gininsert under very high load