Thread: Scaling
My SMB database is currently running on a HP-UX box running in a master - slave slonyI cluster to create a hot spare database. Recently the server has be getting hammered and we are consistently hitting our max db connections. We have our web CMS and our web apps, our desktop client apps, and a couple application servers running off of this database server and it is crumbling under the stress. Currently we have been doing table partitioning and table archiving to trim the fat off of the indexing processes etc.
My project at hand is to upgrade this configuration to a linux cluster of servers, 2 - 3. The ideal solution is to just do load balancing through DNS requests and have 2 servers in a multi master configuration. So far we have not had any hardware issues on our current set up ( knock on wood). So in doing this I am not immediately looking for a hot spare set up. But in the future am looking at setting up a VPN tunnel to an offsite server and sending WAL updates to that.
Any and all comments are truly appreciated as well as any nudges in various directions.
David.
--
David Hornsby
Beechglen Development Inc.
E-Mail: david@beechglen.com
Phone: 513 922 0509 x432
Fax: 513 347 2834
Web: beechglen.com
On 04/24/2011 04:21 PM, David Hornsby wrote:
> My SMB database is currently running on a HP-UX box running in a master
> - slave slonyI cluster to create a hot spare database. Recently the
> server has be getting hammered and we are consistently hitting our max
> db connections. We have our web CMS and our web apps, our desktop client
> apps, and a couple application servers running off of this database
> server and it is crumbling under the stress. Currently we have been
> doing table partitioning and table archiving to trim the fat off of the
> indexing processes etc.
>
> My project at hand is to upgrade this configuration to a linux cluster
> of servers, 2 - 3. The ideal solution is to just do load balancing
> through DNS requests and have 2 servers in a multi master configuration.
> So far we have not had any hardware issues on our current set up ( knock
> on wood). So in doing this I am not immediately looking for a hot spare
> set up. But in the future am looking at setting up a VPN tunnel to an
> offsite server and sending WAL updates to that.
>
> Any and all comments are truly appreciated as well as any nudges in
> various directions.
I'm not aware of any production ready multi-master solution for
PostgreSQL. So I would not suggest perusing that at the moment.
If your primary problem is hitting your max db connections (and the
performance degradation that goes along with that), I would suggest
looking in to connection pooling. This will allow you to cap the number
of connections without just rejecting connections and will enable you to
dedicate more resources to each of the active connections. Ultimately,
your overall throughput should increase.
pgbouncer is very easy to set up and is great if you want simple
pooling. pgpool is is also a great pooler and a lot more (e.g. load
balancing, horizontal scaling, etc), but is a bit more complex to set up.
- Chris
Zitat von Chris Ernst <cernst@zvelo.com>: > On 04/24/2011 04:21 PM, David Hornsby wrote: >> My SMB database is currently running on a HP-UX box running in a master >> - slave slonyI cluster to create a hot spare database. Recently the >> server has be getting hammered and we are consistently hitting our max >> db connections. We have our web CMS and our web apps, our desktop client >> apps, and a couple application servers running off of this database >> server and it is crumbling under the stress. Currently we have been >> doing table partitioning and table archiving to trim the fat off of the >> indexing processes etc. >> >> My project at hand is to upgrade this configuration to a linux cluster >> of servers, 2 - 3. The ideal solution is to just do load balancing >> through DNS requests and have 2 servers in a multi master configuration. >> So far we have not had any hardware issues on our current set up ( knock >> on wood). So in doing this I am not immediately looking for a hot spare >> set up. But in the future am looking at setting up a VPN tunnel to an >> offsite server and sending WAL updates to that. >> >> Any and all comments are truly appreciated as well as any nudges in >> various directions. > > I'm not aware of any production ready multi-master solution for > PostgreSQL. So I would not suggest perusing that at the moment. http://wiki.postgresql.org/wiki/Bucardo But i have never used it so no idea how stable/mature it is. Regards Andreas
Attachment
Hi All, I am trying to accurately graph the output of the Bucardo check_postgres --dbstats script for trending over time. We arealso setting up Nagios alerts based on the script. Does anyone have existing graph templates for this usage? The reason I am asking here and not a cacti list is that the datagets collected fine, but I am not familiar enough with RRDtool/Cacti to create informative graphs. The data source anddata templates work fine, and cacti picks up all the fields in the output. If no one has any I will just keep diggingthrough RRDtool docs. Alternatively, does anyone have recommendations for other visual historical trending tools for Postgres? Thank you in advance. See you guys next week in Ottawa! Neal Brown Spiceworks Inc.
On 04/24/2011 03:21 PM, David Hornsby wrote: > My SMB database is currently running on a HP-UX box running in a master > - slave slonyI cluster to create a hot spare database. Recently the > server has be getting hammered and we are consistently hitting our max > db connections. We have our web CMS and our web apps, our desktop client > apps, and a couple application servers running off of this database > server and it is crumbling under the stress. Currently we have been > doing table partitioning and table archiving to trim the fat off of the > indexing processes etc. > > My project at hand is to upgrade this configuration to a linux cluster > of servers, 2 - 3. The ideal solution is to just do load balancing > through DNS requests and have 2 servers in a multi master configuration. > So far we have not had any hardware issues on our current set up ( knock > on wood). So in doing this I am not immediately looking for a hot spare > set up. But in the future am looking at setting up a VPN tunnel to an > offsite server and sending WAL updates to that. You are more likely to need one master, two hot standbys and push your reads to the slaves. JD -- Command Prompt, Inc. - http://www.commandprompt.com/ PostgreSQL Support, Training, Professional Services and Development The PostgreSQL Conference - http://www.postgresqlconference.org/ @cmdpromptinc - @postgresconf - 509-416-6579