Request a Demo
Home   >   mailing lists

Re: Restructured Shared Buffer Hash Table - Mailing list pgsql-hackers

From Dhruv Aron
Subject Re: Restructured Shared Buffer Hash Table
Date
Msg-id CAAStW3LLnPYypEh3fQu+z-t1cAw7fUJh=0aQDZcMh=XOuJQF_Q@mail.gmail.com
Whole thread
In response to Re: Restructured Shared Buffer Hash Table  (Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>)
Responses Re: Restructured Shared Buffer Hash Table
List pgsql-hackers
Tree view
On Tue, 7 Jul 2026 at 20:14, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> Or maybe it's not a problem, in which case some kind of a
> worst case scenario benchmark to show that would be nice. Maybe test how
> it behaves when you have a lot of hash collisions, I think that'd make
> BufTableDelete() more expensive.

My additional benchmarking would suggest that, on average (one entry per bucket), the spinlock would be held for an extra ~50ns compared to releasing the header lock earlier for the current dynahash implementation:

Bucket Chain Length

Average Time Under Spinlock for Dynahash Implementation  (ns)

Average Time Under Spinlock for New Table, Releasing Lock After Delete (ns)

1

64.4

112.9

2

63.4

118.6

4

63.1

122.3

8

68.8

139.8

16

76.0

159.1

32

77.7

195.0

64

102.0

248.6



On Tue, Jul 7, 2026 at 8:28 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote:
> If we are using a different data structure for buffer lookup, 
> it will be good to consider ease of resizing as well.

Yes, we have internal functionality to resize (both shrink and expand) this restructured table without rehashing the entire table, acquiring a global lock, or introducing new synchronization primitives. Will create and share a follow-up patch going into more detail soon.

Thanks,
Dhruv Aron

On Tue, Jul 7, 2026 at 8:28 PM Ashutosh Bapat <ashutosh.bapat.oss@gmail.com> wrote:
On Wed, Jul 8, 2026 at 12:44 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
>
> On 07/07/2026 21:41, Dhruv Aron wrote:
> > At Databricks, we’ve found that the existing dynahash table structure is
> > leaving performance gains on the table when it comes to shared buffer
> > lookups: the multi-level structure (directory, segment, bucket chain,
> > freelist) appears excessive for the shared buffers and could be
> > simplified to boost performance and lower memory overhead. As such, we
> > are proposing a specialized hash table just for this purpose and would
> > appreciate feedback on this approach.
>
> Yeah, the dynahash has features that we just don't need in the buffer
> lookup table...
>
> The indirection with the directory is actually unnecessary for all the
> shared memory hash tables, since none of them can be resized. I've
> wondered if we should try to eliminate that from dynahash for all shmem
> hash tables. But the buffer lookup table is very performance-critical,
> so if there are any performance gains to be had, it's indeed probably
> worthwhile to have a separate implementation just for it.

For very large buffer pools, the buffer lookup table spans multiple
memory pages. With buffer resizing capability it will be good to be
able to resize the buffer lookup table as well. Right now the patch
does not resize buffer look up table because of its memory layout and
complexity involved in that operation. If we are using a different
data structure for buffer lookup, it will be good to consider ease of
resizing as well.

--
Best Wishes,
Ashutosh Bapat

pgsql-hackers by date:

Previous
From: "Greg Burd"
Date:
Subject: Re: [PATCH] Batched clock sweep to reduce cross-socket atomic contention
Next
From: Melanie Plageman
Date:
Subject: Re: Why clearing the VM doesn't require registering vm buffer in wal record