Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins - Mailing list pgsql-hackers

From Robert Haas
Subject Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins
Date
Msg-id CA+TgmobBuOgaP438yNo5PKHGXRhOhS3MWdEF5kxJm8mC5q1O7A@mail.gmail.com
Whole thread Raw
In response to Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins  (Simon Riggs <simon@2ndQuadrant.com>)
Responses Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins  (Andres Freund <andres@2ndquadrant.com>)
List pgsql-hackers
On Wed, Apr 9, 2014 at 5:34 AM, Simon Riggs <simon@2ndquadrant.com> wrote:
> We've got to the stage now that saving this much memory is essential,
> so this patch is a must-have.
>
> The patch does all I would expect and no more, so approach and details
> look good to me.
>
> Performance? Discussed many years ago, but I suspect the micro-tuning
> of those earlier patches wasn't as good as it is here.

I think this approach is practically a slam-dunk when the number of
pins is small (as it typically is).  I'm less clear what happens when
we overflow from the small array into the hashtable.  That certainly
seems like it could be a loss, but how do we construct such a case to
test it?  A session with lots of suspended queries?  Can we generate a
regression by starting a few suspended queries to use up the array
elements, and then running a scan that pins and unpins many buffers?

One idea is: if we fill up all the array elements and still need
another one, evict all the elements to the hash table and then start
refilling the array.  The advantage of that over what's done here is
that the active scan will always being using an array slot rather than
repeated hash table manipulations.  I guess you'd still have to probe
the hash table repeatedly, but you'd avoid entering and removing items
frequently.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Amit Kapila
Date:
Subject: Re: Dynamic Shared Memory stuff
Next
From: Andres Freund
Date:
Subject: Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins