Re: RFC: Very large scale postgres support - Mailing list pgsql-hackers

From Alex J. Avriette
Subject Re: RFC: Very large scale postgres support
Date
Msg-id 20040208224137.GA12909@posixnap.net
Whole thread Raw
In response to Re: RFC: Very large scale postgres support  (Andreas Pflug <pgadmin@pse-consulting.de>)
Responses Re: RFC: Very large scale postgres support  (Rod Taylor <pg@rbt.ca>)
Re: RFC: Very large scale postgres support  (Chris <list@1006.org>)
List pgsql-hackers
On Sun, Feb 08, 2004 at 08:07:14PM +0100, Andreas Pflug wrote:

> >I feel that it would be a very good thing if some thinking on this
> >subject was done. In the future, people will hopefully begin using
> >postgres for more intense applications. We are looking at perhaps many
> >tens of billions of transactions per day within the next year or two.
> > 
> >
> 
> tens of billions =10e10 per day? This is probably a typo, because this 
> would mean > 100,000 requests per second? Do you want to feed a monitor 

That's what I said, and what I meant. Ten billion transactions equates
to 115,740 transactions per second.

> with pixel data right from the database, using individual queries for 
> each pixel? Or record each irc user's keyclick in the world concurrently 
> online in a single database?

Just because you don't think there is a valid use for that sort of
traffic doesn't mean there isn't one. Imagine, if you will, a hundred
thousand agents making four to five requests a second.  Now, imagine
these requests are all going to the same database.

I'll leave the rest of this exercise up to you.

The fact is, there are situations in which such extreme traffic is
warranted. My concern is that I am not able to use postgres in such
situations because it cannot scale to that level. I feel that it would
be possible to reach that level with support in the postmaster for
replication. 

With software load balancing (eg rotors or similar) and updates between
postmasters, it would be (it seems to me) possible to drastically
increase the available capacity of a database installation through the
addition of more nodes. This has the added benefit of allowing us to
distribute network resources.

Alex

--
alex@posixnap.net
Alex J. Avriette, Unix Systems Gladiator
The Emperor Wears No Clothes.
http://www.drugsense.org/wodclock.htm


pgsql-hackers by date:

Previous
From: Kevin Brown
Date:
Subject: Re: [pgsql-hackers-win32] Sync vs. fsync during checkpoint
Next
From: Neil Conway
Date:
Subject: psql tab completion & USERSET vars