Re: PostgreSQL as a local in-memory cache

From: Pierre C
Subject: Re: PostgreSQL as a local in-memory cache
Date: ,
Msg-id: op.vegjnqfreorkce@apollo13
(view: Whole thread, Raw)
In response to: Re: PostgreSQL as a local in-memory cache  (Josh Berkus)
List: pgsql-performance

Tree view

PostgreSQL as a local in-memory cache  ("", )
 Re: PostgreSQL as a local in-memory cache  ("", )
  Re: PostgreSQL as a local in-memory cache  (Josh Berkus, )
   Re: PostgreSQL as a local in-memory cache  ("Pierre C", )
   Re: PostgreSQL as a local in-memory cache  (Dimitri Fontaine, )
    Re: PostgreSQL as a local in-memory cache  (Tom Lane, )
     Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
      Re: PostgreSQL as a local in-memory cache  (Pavel Stehule, )
       Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
      Re: PostgreSQL as a local in-memory cache  (Robert Haas, )
       Re: PostgreSQL as a local in-memory cache  (Dave Page, )
        Re: PostgreSQL as a local in-memory cache  (Tom Lane, )
         Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
         Re: PostgreSQL as a local in-memory cache  (Dimitri Fontaine, )
       Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
     Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
      Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
       Re: PostgreSQL as a local in-memory cache  (Robert Haas, )
        Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
         Re: PostgreSQL as a local in-memory cache  ("Kevin Grittner", )
         Re: PostgreSQL as a local in-memory cache  (Robert Haas, )
          Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
           Re: PostgreSQL as a local in-memory cache  (Tom Lane, )
            Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
             Re: PostgreSQL as a local in-memory cache  (Jignesh Shah, )
              Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
               Re: PostgreSQL as a local in-memory cache  (Brad Nicholson, )
                Re: PostgreSQL as a local in-memory cache  (Bruce Momjian, )
               Re: PostgreSQL as a local in-memory cache  (Jignesh Shah, )
                Re: PostgreSQL as a local in-memory cache  (Greg Smith, )
    Re: PostgreSQL as a local in-memory cache  ("Pierre C", )
     Re: PostgreSQL as a local in-memory cache  (Matthew Wakeling, )
      Re: PostgreSQL as a local in-memory cache  ("Pierre C", )
       Re: PostgreSQL as a local in-memory cache  (Josh Berkus, )
        Re: PostgreSQL as a local in-memory cache  (Rob Wultsch, )
         Re: PostgreSQL as a local in-memory cache  (Robert Haas, )
          Re: PostgreSQL as a local in-memory cache  (Josh Berkus, )
           Re: PostgreSQL as a local in-memory cache  (Pavel Stehule, )
            Re: PostgreSQL as a local in-memory cache  ("Joshua D. Drake", )
             Re: PostgreSQL as a local in-memory cache  (Pavel Stehule, )
              Re: PostgreSQL as a local in-memory cache  ("A.M.", )
               Re: PostgreSQL as a local in-memory cache  (Pavel Stehule, )
            Re: PostgreSQL as a local in-memory cache  (Josh Berkus, )
             Re: PostgreSQL as a local in-memory cache  (Pavel Stehule, )
            Re: PostgreSQL as a local in-memory cache  ("Joshua D. Drake", )
      Re: PostgreSQL as a local in-memory cache  (Josh Berkus, )
    Re: PostgreSQL as a local in-memory cache  (Josh Berkus, )
     Re: PostgreSQL as a local in-memory cache  (Tom Lane, )
   Re: PostgreSQL as a local in-memory cache  (Greg Smith, )
   Re: PostgreSQL as a local in-memory cache  (Robert Haas, )
 Re: PostgreSQL as a local in-memory cache  (Dave Crooke, )
  Re: PostgreSQL as a local in-memory cache  (Craig James, )

> Especially as, in repeated tests, PostgreSQL with persistence turned off
> is just as fast as the fastest nondurable NoSQL database.  And it has a
> LOT more features.

An option to completely disable WAL for such use cases would make it a lot
faster, especially in the case of heavy concurrent writes.

> Now, while fsync=off and tmpfs for WAL more-or-less eliminate the IO for
> durability, they don't eliminate the CPU time.

Actually the WAL overhead is some CPU and lots of locking.

> Which means that a caching version of PostgreSQL could be even faster.
> To do that, we'd need to:
>
> a) Eliminate WAL logging entirely
> b) Eliminate checkpointing
> c) Turn off the background writer
> d) Have PostgreSQL refuse to restart after a crash and instead call an
> exteral script (for reprovisioning)
>
> Of the three above, (a) is the most difficult codewise.

Actually, it's pretty easy, look in xlog.c



pgsql-performance by date:

From: Tom Wilcox
Date:
Subject: Re: requested shared memory size overflows size_t
From: Mark Kirkwood
Date:
Subject: wal_synch_method = open_sync safe on RHEL 5.5?