Thread: Scalability
I am new to PostgreSQL and just beginning to learn the product. I will probrobably be using it exclusively on Windows. I was surprised to learn that PostgreSQL creates a new process for each connection. Doesn't this severely limit its scalability by consuming resources rapidly on the server as the number of user increases? -- .Bill.
Bill wrote:
> I am new to PostgreSQL and just beginning to learn the product. I will
> probrobably be using it exclusively on Windows.
>
> I was surprised to learn that PostgreSQL creates a new process for each
> connection. Doesn't this severely limit its scalability by consuming
> resources rapidly on the server as the number of user increases?
The Windows version is not anywhere near as scalable as the unix
versions. Depending on your hardware you will top out a Windows
installation about about 350-400 connections. You can get more out of
Windows by modifying the registry but I am unsure of how far it will go.
I have Linux installations that happily hum along with 2000-5000
connections.
So in answer to your question, in general -- no the process methodology
we use does not limit scalability and it makes our code base much
simpler that the equivalent threading model.
Sincerely,
Joshua D. Drake
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
On Oct 29, 2006, at 02:31 , Bill wrote: > I was surprised to learn that PostgreSQL creates a new process for > each > connection. Doesn't this severely limit its scalability by consuming > resources rapidly on the server as the number of user increases? On Windows, yes; Win32 processes carry a not insignificant amount of baggage with them, and you would probably want a connection pool between your app and PostgreSQL in order to throttle the load. PostgreSQL was designed and optimized for Unix, where processes are lightweight, very fast to set up and come with added security benefits such as protected address spaces. You will notice most daemons follow this approach. Some, like Apache 2.x, offer a hybrid approach with n threads distributed over m processes. I suspect it would take a large engineering effort to retrofit PostgreSQL to do this on Windows, since a lot of PostgreSQL's internals probably aren't thread-safe, but in theory it could work. Alexander.
"Joshua D. Drake" wrote: > Bill wrote: > > I am new to PostgreSQL and just beginning to learn the product. I > > will probrobably be using it exclusively on Windows. > > > > I was surprised to learn that PostgreSQL creates a new process for > > each connection. Doesn't this severely limit its scalability by > > consuming resources rapidly on the server as the number of user > > increases? > > The Windows version is not anywhere near as scalable as the unix > versions. Depending on your hardware you will top out a Windows > installation about about 350-400 connections. You can get more out of > Windows by modifying the registry but I am unsure of how far it will > go. > > I have Linux installations that happily hum along with 2000-5000 > connections. > > So in answer to your question, in general -- no the process > methodology we use does not limit scalability and it makes our code > base much simpler that the equivalent threading model. > > Sincerely, > > Joshua D. Drake Thanks. How much memory does PostgreSQL consume with 2000 connections? Which Linux distribution do you prefer? -- .Bill.
>> So in answer to your question, in general -- no the process
>> methodology we use does not limit scalability and it makes our code
>> base much simpler that the equivalent threading model.
>>
>> Sincerely,
>>
>> Joshua D. Drake
>
> Thanks. How much memory does PostgreSQL consume with 2000 connections?
Entirely depends on what you are doing with it :). There is 16GB-32GB in
these particular machines.
>
> Which Linux distribution do you prefer?
>
I prefere Ubuntu, but FreeBSD or pretty much any other 2.6 based Linux
distribution will do.
Joshua D. Drake
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
Top at 350-400 concurrent connections! Although more than my project's peak estimates, it is a bit discomforting. Are there any *promising* load testing numbers with Windows 2003? I'd be happy to share results from my simulations. Matt > -------- Original Message -------- > Subject: Re: [GENERAL] Scalability > From: "Joshua D. Drake" <jd@commandprompt.com> > Date: Sat, October 28, 2006 8:38 pm > To: Bill <postgresql@dbginc.com> > Cc: pgsql-general@postgresql.org > > Bill wrote: > > I am new to PostgreSQL and just beginning to learn the product. I will > > probrobably be using it exclusively on Windows. > > > > I was surprised to learn that PostgreSQL creates a new process for each > > connection. Doesn't this severely limit its scalability by consuming > > resources rapidly on the server as the number of user increases? > > The Windows version is not anywhere near as scalable as the unix > versions. Depending on your hardware you will top out a Windows > installation about about 350-400 connections. You can get more out of > Windows by modifying the registry but I am unsure of how far it will go. > > I have Linux installations that happily hum along with 2000-5000 > connections. > > So in answer to your question, in general -- no the process methodology > we use does not limit scalability and it makes our code base much > simpler that the equivalent threading model. > > Sincerely, > > Joshua D. Drake
volunteer@spatiallink.org wrote:
> Top at 350-400 concurrent connections! Although more than my project's
> peak estimates, it is a bit discomforting. Are there any *promising*
> load testing numbers with Windows 2003? I'd be happy to share results
> from my simulations.
Well keep in mind this is a Windows limitation not a PostgreSQL one. I
am sure those who are better than I at tuning Windows will be able to
hack the registry to get it to go farther.
However.... Windows has never been nor will it likely ever be as
scalable as the unix variants.
Joshua D. Drake
>
> Matt
>
>> -------- Original Message --------
>> Subject: Re: [GENERAL] Scalability
>> From: "Joshua D. Drake" <jd@commandprompt.com>
>> Date: Sat, October 28, 2006 8:38 pm
>> To: Bill <postgresql@dbginc.com>
>> Cc: pgsql-general@postgresql.org
>>
>> Bill wrote:
>>> I am new to PostgreSQL and just beginning to learn the product. I will
>>> probrobably be using it exclusively on Windows.
>>>
>>> I was surprised to learn that PostgreSQL creates a new process for each
>>> connection. Doesn't this severely limit its scalability by consuming
>>> resources rapidly on the server as the number of user increases?
>> The Windows version is not anywhere near as scalable as the unix
>> versions. Depending on your hardware you will top out a Windows
>> installation about about 350-400 connections. You can get more out of
>> Windows by modifying the registry but I am unsure of how far it will go.
>>
>> I have Linux installations that happily hum along with 2000-5000
>> connections.
>>
>> So in answer to your question, in general -- no the process methodology
>> we use does not limit scalability and it makes our code base much
>> simpler that the equivalent threading model.
>>
>> Sincerely,
>>
>> Joshua D. Drake
>
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate