Thread: what does pg_activity mean when the database is stuck?

Dnt have any unfrmtn

From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Si Chen
Sent: Wednesday, June 11, 2014 4:34 PM
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] what does pg_activity mean when the database is stuck?

I didn't see any from the log.  It was just a whole bunch of pretty standard looking SELECT queries.  There were no
INSERT/COMMITstatements which were still active before the SELECT's, just a few which are waiting after a lot of SELECT
statements.

Also, if the process just shows COMMIT, is there any way to see what it's trying to commit?  

On Wed, Jun 11, 2014 at 9:29 AM, Jeff Janes <jeff.janes@gmail.com> wrote:
On Wed, Jun 11, 2014 at 8:59 AM, Si Chen <sichen@opensourcestrategies.com> wrote:
I have a problem where postgresql 9.3 got stuck, and the number of postgresql processes increased from about 15 to 225
in10 minutes.
 

I ran the query:
select pid, query_start, waiting, state, query from pg_stat_activity order by query_start;

But it showed mostly select statements -- all of them the same one, with a couple of joins.  They are not in a waiting
statebut have been running for over 2 hours.  
 

I also checked for locks with the query on
http://wiki.postgresql.org/wiki/Lock_Monitoring

But it returned no locked tables.

So what does this mean?  Is the select query getting stuck?  

Do you have a huge chunk of newly insert, not yet committed, rows?  This sounds like the issue where all of the
processesfight with each other over the right to check uncommitted rows in order to verify that they are actually
uncommitted.

Cheers,

Jeff


-- 
Si Chen
Open Source Strategies, Inc.
sichen@opensourcestrategies.com
http://www.OpenSourceStrategies.com
LinkedIn: http://www.linkedin.com/in/opentaps
Twitter: http://twitter.com/opentaps


When you query pg_stat_activity, what do you see in state column, and how state_change compares to query_start?

Regards,
Igor Neyman

On Wed, Jun 11, 2014 at 5:37 PM, Si Chen
<sichen@opensourcestrategies.com> wrote:
> The state is "idle".  I don't have the state_change, but I will try to
> collect it if it happens again.

If they are idle, then the problem is probably with your application
-- you're grabbing new connections and not closing them or reusing
them.  It's a very common problem.  The 'query' when idle represents
the last query run -- the database finished it and is sitting around.

merlin

On Thu, Jun 12, 2014 at 11:44 AM, Si Chen
<sichen@opensourcestrategies.com> wrote:
> Is there a way to configure postgresql to automatically release connections
> that have been idle for a set amount of time?
Not directly. However with 9.3 you could use a background worker like this one:
https://github.com/michaelpq/pg_plugins/tree/master/kill_idle
It tracks connections that have been idle for a given amount of time
and closes them with pg_terminate_backend when theyr are idle for a
amount of time longer than the one set.
--
Michael

On Thu, Jun 12, 2014 at 6:34 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> On Thu, Jun 12, 2014 at 11:44 AM, Si Chen
> <sichen@opensourcestrategies.com> wrote:
>> Is there a way to configure postgresql to automatically release connections
>> that have been idle for a set amount of time?
> Not directly. However with 9.3 you could use a background worker like this one:
> https://github.com/michaelpq/pg_plugins/tree/master/kill_idle
> It tracks connections that have been idle for a given amount of time
> and closes them with pg_terminate_backend when theyr are idle for a
> amount of time longer than the one set.
Just adding that this bgworker is rather experimental, and that the
common recoommendation is to have your application close correctly the
connection it uses...
--
Michael

From: pgsql-general-owner@postgresql.org [mailto:pgsql-general-owner@postgresql.org] On Behalf Of Si Chen
Sent: Wednesday, June 11, 2014 10:44 PM
To: pgsql-general@postgresql.org
Subject: Re: [GENERAL] what does pg_activity mean when the database is stuck?

Is there a way to configure postgresql to automatically release connections that have been idle for a set amount of
time?

On Wed, Jun 11, 2014 at 3:41 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
On Wed, Jun 11, 2014 at 5:37 PM, Si Chen
<sichen@opensourcestrategies.com> wrote:
> The state is "idle".  I don't have the state_change, but I will try to
> collect it if it happens again.
If they are idle, then the problem is probably with your application
-- you're grabbing new connections and not closing them or reusing
them.  It's a very common problem.  The 'query' when idle represents
the last query run -- the database finished it and is sitting around.

merlin

-- 
Si Chen
Open Source Strategies, Inc.
sichen@opensourcestrategies.com
http://www.OpenSourceStrategies.com
LinkedIn: http://www.linkedin.com/in/opentaps
Twitter: http://twitter.com/opentaps


The best solution for this is to use some connection pooler, such as PgBouncer.
B.t.w., PgBouncer can also disconnect idle client connections (if you really wish) based on configuration setting.

Regards,
Igor Neyman

Si Chen <sichen@opensourcestrategies.com> writes:

> Is there a way to configure postgresql to automatically release connections that have been idle for a set amount of
time?

I've seen recent discussion in hackers that this may be added to an
upcoming release.


> On Wed, Jun 11, 2014 at 3:41 PM, Merlin Moncure <mmoncure@gmail.com> wrote:
>
>     On Wed, Jun 11, 2014 at 5:37 PM, Si Chen
>     <sichen@opensourcestrategies.com> wrote:
>     > The state is "idle".  I don't have the state_change, but I will try to
>     > collect it if it happens again.
>
>     If they are idle, then the problem is probably with your application
>     -- you're grabbing new connections and not closing them or reusing
>     them.  It's a very common problem.  The 'query' when idle represents
>     the last query run -- the database finished it and is sitting around.
>
>     merlin
>
> --
> Si Chen
> Open Source Strategies, Inc.
> sichen@opensourcestrategies.com
> http://www.OpenSourceStrategies.com
> LinkedIn: http://www.linkedin.com/in/opentaps
> Twitter: http://twitter.com/opentaps
>

--
Jerry Sievers
Postgres DBA/Development Consulting
e: postgres.consulting@comcast.net
p: 312.241.7800

On 6/12/2014 4:24 PM, Si Chen wrote:
> PgBouncer looks pretty cool.  Do you recommend using it with jdbc with
> about 50 - 100 normal connections?

Java has quite a few built in connection pooling options.

they all work best if your software is configured to grab a connection,
use it for a transaction, then release it back to the pool.

--
john r pierce                                      37N 122W
somewhere on the middle of the left coast