On 2014-04-16 10:29:29 -0400, Robert Haas wrote:
> On Wed, Apr 16, 2014 at 9:35 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > I think this is the wrong level to optimize things. Imo there's two
> > possible solutions (that don't exclude each other):
> >
> > * perform the clock sweep in one process so there's a very fast way to
> > get to a free buffer. Possibly in a partitioned way.
> >
> > * Don't take a global exclusive lock while performing the clock
> > sweep. Instead increase StrategyControl->nextVictimBuffer in chunks
> > under an exclusive lock, and then scan the potential victim buffers in
> > those chunks without a global lock held.
>
> I definitely agree with both of these ideas. But isn't it sort of
> off-topic for this thread?
Yes, I agree it's somewhat offtopic - I only started on it (I think)
because Merlin commented on it. But I also agree with Merlin's that
comment at the moment that the scalability issues (concurrency and size
of shared buffers). If you can't use a large enough s_b to contain a
significant portion of your workload, you're relying on the OS cache
anyway.
> 1. Improving the rate at which we can evict buffers, which is what
> you're talking about here.
>
> 2. Improving the choice of which buffers we evict, which is what
> Peter's talking about, or at least what I think he's talking about.
>
> Those things are both important, but they're different, and I'm not
> sure that working on one precludes working on the other. There's
> certainly the potential for overlap, but not necessarily.
I don't think that that they neccessarily preclude each other
either. But my gut feeling tells me that it'll be hard to have
interesting algorithmic improvements on the buffer eviction choice
because any additional complexity around that will have prohibitively
high scalability impacts due to the coarse locking.
Greetings,
Andres Freund
-- Andres Freund http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services
