Re: Restricting Postgres - Mailing list pgsql-performance

From Martin Foster
Subject Re: Restricting Postgres
Date
Msg-id 418A2A4E.6030409@ethereal-realms.org
Whole thread Raw
In response to Re: Restricting Postgres  ("Matt Clark" <matt@ymogen.net>)
Responses Re: Restricting Postgres  ("Matthew Nuzum" <newz@bearfruit.org>)
List pgsql-performance
Matt Clark wrote:

>>I have a dual processor system that can support over 150 concurrent
>>connections handling normal traffic and load.   Now suppose I setup
>>Apache to spawn all of it's children instantly, what will
>
> ...
>
>>This will spawn 150 children in a short order of time and as
>>this takes
>
>
> "Doctor, it hurts when I do this!"
> "Well, don't do that then..."
>
> Sorry, couldn't resist ;-)
>
> Our Apache/PG driven website also needs to be able to deal with occasional
> large peaks, so what we do is:
>
> StartServers 15        # Don't create too many children initially
> MinSpareServers 10    # Always have at least 10 spares lying around
> MaxSpareServers 20    # But no more than 20
> MaxClients 150        # Up to 150 - the default 256 is too much for our
> RAM
>
>
> So on server restart 15 Apache children are created, then one new child
> every second up to a maximum of 150.
>
> Apache's 'ListenBackLog' is around 500 by default, so there's plenty of
> scope for queuing inbound requests while we wait for sufficient children to
> be spawned.
>
> In addition we (as _every_ high load site should) run Squid as an
> accelerator, which dramatically increases the number of client connections
> that can be handled.  Across 2 webservers at peak times we've had 50,000
> concurrently open http & https client connections to Squid, with 150 Apache
> children doing the work that squid can't (i.e. all the dynamic stuff), and
> PG (on a separate box of course) whipping through nearly 800 mixed selects,
> inserts and updates per second - and then had to restart Apache on one of
> the servers for a config change...  Not a problem :-)
>
> One little tip - if you run squid on the same machine as apache, and use a
> dual-proc box, then because squid is single-threaded it will _never_ take
> more than half the CPU - nicely self balancing in a way.
>
> M
>

I've heard of the merits of Squid in the use as a reverse proxy.
However, well over 99% of my traffic is dynamic, hence why I may be
experiencing behavior that people normally do not expect.

As I have said before in previous threads, the scripts are completely
database driven and at the time the database averaged 65 queries per
second under MySQL before a migration, while the webserver was averaging
2 to 4.

    Martin Foster
    Creator/Designer Ethereal Realms
    martin@ethereal-realms.org



pgsql-performance by date:

Previous
From: Pierre-Frédéric Caillaud
Date:
Subject: Re: Anything to be gained from a 'Postgres Filesystem'?
Next
From: Martin Foster
Date:
Subject: Re: Restricting Postgres