Thread: [Fwd: postgres 8.4.1 number of connections]

-------- Original Message --------

Subject:	postgres 8.4.1 number of connections
Date:	Thu, 26 Aug 2010 14:25:47 -0500
From:	Maria L. Wilson <Maria.L.Wilson-1@nasa.gov>
Reply-To:	Wilson, Maria Louise (LARC-E301)[SCIENCE SYSTEMS APPLICATIONS] <m.l.wilson@nasa.gov>
To:	pgsql-performance-owner@postgresql.org <pgsql-performance-owner@postgresql.org>

we have this application (using jboss/java/hibernate) on linux accessing 
data on 3 postgres database servers using 8.4.1. 

One of our many concerns has been the way we handle connections to the 
database.  java/hibernate handle their own pooling so I understand that 
using anything else is out of the question.  Our jboss configuration 
currently defaults to 5 connections per database.  On our main database 
server, we handle 7 of the databases that this application uses.  On 
this one server, we usually average around 300 - 500 connections to the 
server.  In our postgres conf file on this particular machine we set the 
max_connection parameter to 1000.  If we set it to much lower we end up 
with connection errors.  Any comments or better way to handle this?

thanks,  Maria Wilson
NASA, Langley Research Center
Hampton, Virginia 23666

On Thu, Aug 26, 2010 at 2:29 PM, Maria L. Wilson
<Maria.L.Wilson-1@nasa.gov> wrote:
>
>
> we have this application (using jboss/java/hibernate) on linux accessing
> data on 3 postgres database servers using 8.4.1.
>
> One of our many concerns has been the way we handle connections to the
> database.  java/hibernate handle their own pooling so I understand that
> using anything else is out of the question.

It's not impossible to put another connection pooler between hibernate
and your database server.  May not be useful or better, but it's
possible.

>  Our jboss configuration
> currently defaults to 5 connections per database.

I assume this is for each app server.  How many app servers do you have?

>  On our main database
> server, we handle 7 of the databases that this application uses.  On
> this one server, we usually average around 300 - 500 connections to the
> server.  In our postgres conf file on this particular machine we set the
> max_connection parameter to 1000.  If we set it to much lower we end up
> with connection errors.  Any comments or better way to handle this?

Other than breaking out the databases onto their own server, or trying
to pool to fewer connections, not really.  OTOH, a db server with
enough memory can handle having that many connections as long as you
don't run into any thundering herd issues.

Oh, and you should really update your pgsql to 8.4.4.  8.4.1 had some
issues with heavy load I ran into last year about this time.  Which is
why I'm still on 8.3.11

--
To understand recursion, one must first understand recursion.

> One of our many concerns has been the way we handle connections to the
> database.  java/hibernate handle their own pooling so I understand that
> using anything else is out of the question.

It's not out of the question, but it's probably not necessary.

> Our jboss configuration
> currently defaults to 5 connections per database.  On our main database
> server, we handle 7 of the databases that this application uses.  On
> this one server, we usually average around 300 - 500 connections to the
> server.

Do you have 10 to 20 application servers?  Otherwise I don't understand
how you're getting 300 to 500 connections.  Do you have any stats about
how many connections for each database each JBOSS server actually needs?
 It seems unusual that it would need more than one or two.

Also, why does the application use 7 different databases?

--
                                  -- Josh Berkus
                                     PostgreSQL Experts Inc.
                                     http://www.pgexperts.com

On Thu, Aug 26, 2010 at 23:13, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> On Thu, Aug 26, 2010 at 2:29 PM, Maria L. Wilson
> <Maria.L.Wilson-1@nasa.gov> wrote:
>>
>>
>> we have this application (using jboss/java/hibernate) on linux accessing
>> data on 3 postgres database servers using 8.4.1.
>>
>> One of our many concerns has been the way we handle connections to the
>> database.  java/hibernate handle their own pooling so I understand that
>> using anything else is out of the question.
>
> It's not impossible to put another connection pooler between hibernate
> and your database server.  May not be useful or better, but it's
> possible.

I've been using pgbouncer in combination with the jboss connection
pooler several times, mainly for the reason that reconfiguring the
jboss connection poolers (particularly if you have lots of them) can
cause quite a bit of downtime. Just sticking a 1-1 mapping pgbouncer
in between with support for SUSPEND makes a lot of difference if you
switch master/slave on your replication /ha. It'll still break the
connections for jboss, but it'll recover from that a *lot* faster than
a reconfig.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

thanks for your response....
we have 3 app servers that attach to this one particular database server.
What kind of load issues did you find on 8.4.1?  I'd be interested in anything documented on it - this might make an upgrade a higher priority!

thanks,  Maria

Scott Marlowe wrote:

On Thu, Aug 26, 2010 at 2:29 PM, Maria L. Wilson
<Maria.L.Wilson-1@nasa.gov> wrote: 

we have this application (using jboss/java/hibernate) on linux accessing
data on 3 postgres database servers using 8.4.1.

One of our many concerns has been the way we handle connections to the
database.  java/hibernate handle their own pooling so I understand that
using anything else is out of the question.   

It's not impossible to put another connection pooler between hibernate
and your database server.  May not be useful or better, but it's
possible.
 

 Our jboss configuration
currently defaults to 5 connections per database.   

I assume this is for each app server.  How many app servers do you have?
 

 On our main database
server, we handle 7 of the databases that this application uses.  On
this one server, we usually average around 300 - 500 connections to the
server.  In our postgres conf file on this particular machine we set the
max_connection parameter to 1000.  If we set it to much lower we end up
with connection errors.  Any comments or better way to handle this?   

Other than breaking out the databases onto their own server, or trying
to pool to fewer connections, not really.  OTOH, a db server with
enough memory can handle having that many connections as long as you
don't run into any thundering herd issues.

Oh, and you should really update your pgsql to 8.4.4.  8.4.1 had some
issues with heavy load I ran into last year about this time.  Which is
why I'm still on 8.3.11

--
To understand recursion, one must first understand recursion.