Re: Caching by Postgres

From: mark@mark.mielke.cc
Subject: Re: Caching by Postgres
Date: ,
Msg-id: 20050823200342.GA18891@mark.mielke.cc
(view: Whole thread, Raw)
In response to: Re: Caching by Postgres  (Donald Courtney)
List: pgsql-performance

Tree view

Caching by Postgres  (gokulnathbabu manoharan, )
 Re: Caching by Postgres  (John A Meinel, )
  Re: Caching by Postgres  (Josh Berkus, )
 Re: Caching by Postgres  (Bruno Wolff III, )
 Re: Caching by Postgres  (Frank Wiles, )
  Re: Caching by Postgres  (Donald Courtney, )
   Re: Caching by Postgres  (Tom Lane, )
   Re: Caching by Postgres  (Josh Berkus, )
    Re: Caching by Postgres  (Michael Stone, )
   Re: Caching by Postgres  (, )
   Re: Caching by Postgres  (William Yu, )
    Re: Caching by Postgres  (PFC, )
     Re: Caching by Postgres  (Josh Berkus, )
      Re: Caching by Postgres  (PFC, )
     Re: Caching by Postgres  (Gavin Sherry, )
      Re: Caching by Postgres  (Tom Lane, )
       Re: Caching by Postgres  (Gavin Sherry, )
    Re: Caching by Postgres  (Donald Courtney, )
     Re: Caching by Postgres  (Stephen Frost, )
     Re: Caching by Postgres  (, )
      Re: Caching by Postgres  (Alan Stange, )
       Re: Caching by Postgres  (, )
        Re: Caching by Postgres  (Alan Stange, )
         Re: Caching by Postgres  (, )
        Re: Caching by Postgres  (Michael Stone, )
         Re: Caching by Postgres  (, )
       Re: Caching by Postgres  (PFC, )
      Re: Caching by Postgres  (Thomas Ganss, )
     Re: Caching by Postgres  (William Yu, )
 Re: Caching by Postgres  (Chris Browne, )
 Re: Caching by Postgres  (Chris Browne, )

On Tue, Aug 23, 2005 at 02:41:39PM -0400, Donald Courtney wrote:
> I mean well with this comment -
> This whole issue of data caching is  a troubling issue with postreSQL
> in that even if you ran postgreSQL on a 64 bit address space
> with larger number of CPUs you won't see much of a scale up
> and possibly even a drop.   I am not alone in having the *expectation*
> that a database should have some cache size parameter and
> the option to skip the file system.   If I use oracle, sybase, mysql
> and maxdb they all have the ability to size a data cache and move
> to 64 bits.
> Is this a crazy idea - that a project be started to get this adopted?
> Is it
> too big and structural to contemplate?
> From one who likes postgreSQL

Hey Donald. :-)

This is an operating system issue, not a PostgreSQL issue. If you have
more physical memory than fits in 32-bit addresses, and your operating
system isn't using this extra memory to cache files (or anything
else), than your OS is what I would consider to be broken (or at the
very least, not designed for a 64-bit host).

The only questions that can be asked here is - 1) can PostgreSQL do a
better job than the OS at best utilizing system RAM, and 2) if so, is
the net gain worth the added complexity to PostgreSQL?

I happen to think that yes, PostgreSQL can do a better job than most
OS's, as it has better information to make decisions as to which pages
are worth keeping, and which are not, but no, it isn't worth the
effort until PostgreSQL developers start running out of things to do.

Buy your 64-bit platforms - but if page caching is your concern, 1)
ensure that you really have more physical memory than can fit in 32
bits, and 2) ensure that your operating system is comfortable caching
data pages from files above the 32-bit mark.

Cheers,
mark

--
 /  /      __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   |
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/



pgsql-performance by date:

From: Josh Berkus
Date:
Subject: Re: Caching by Postgres
From: "Jignesh K. Shah"
Date:
Subject: Re: Read/Write block sizes