Thread: simplehash: preserve consistency in case of OOM
Right now, if allocation fails while growing a hashtable, it's left in an inconsistent state and can't be used again. Patch attached. -- Jeff Davis PostgreSQL Contributor Team - AWS
Attachment
Hi, On 2023-11-17 10:42:54 -0800, Jeff Davis wrote: > Right now, if allocation fails while growing a hashtable, it's left in > an inconsistent state and can't be used again. I'm not against allowing this - but I am curious, in which use cases is this useful? > @@ -446,10 +459,11 @@ SH_CREATE(MemoryContext ctx, uint32 nelements, void *private_data) > /* increase nelements by fillfactor, want to store nelements elements */ > size = Min((double) SH_MAX_SIZE, ((double) nelements) / SH_FILLFACTOR); > > - SH_COMPUTE_PARAMETERS(tb, size); > + size = SH_COMPUTE_SIZE(size); > > - tb->data = (SH_ELEMENT_TYPE *) SH_ALLOCATE(tb, sizeof(SH_ELEMENT_TYPE) * tb->size); > + tb->data = (SH_ELEMENT_TYPE *) SH_ALLOCATE(tb, sizeof(SH_ELEMENT_TYPE) * size); > > + SH_UPDATE_PARAMETERS(tb, size); > return tb; > } Maybe add a comment explaining why it's important to update parameters after allocating? Greetings, Andres Freund
On Fri, 2023-11-17 at 12:13 -0800, Andres Freund wrote:
> On 2023-11-17 10:42:54 -0800, Jeff Davis wrote:
> > Right now, if allocation fails while growing a hashtable, it's left
> > in
> > an inconsistent state and can't be used again.
>
> I'm not against allowing this - but I am curious, in which use cases
> is this
> useful?
I committed a cache for search_path (f26c2368dc), and afterwards got
concerned that I missed some potential OOM hazards. I separately posted
a patch to fix those (mostly by simplifying things, which in hindsight
was how it should have been done to begin with). Along the way, I also
noticed that simplehash itself is not safe in that case.
I don't think there are other bugs in the system due to simplehash and
OOM, because it's mainly used in the executor.
Please tell me if you think the use of simplehash for a search_path
cache is the wrong tool for the job.
> Maybe add a comment explaining why it's important to update
> parameters after
> allocating?
Will do.
Regards,
Jeff Davis
On Fri, Nov 17, 2023 at 12:13 PM Andres Freund <andres@anarazel.de> wrote: > > On 2023-11-17 10:42:54 -0800, Jeff Davis wrote: > > Right now, if allocation fails while growing a hashtable, it's left in > > an inconsistent state and can't be used again. +1 to the patch. > I'm not against allowing this - but I am curious, in which use cases is this > useful? I don't have an answer to that, but a guess would be when the server is dealing with memory pressure. In my view the patch has merit purely on the grounds of increasing robustness. > > @@ -446,10 +459,11 @@ SH_CREATE(MemoryContext ctx, uint32 nelements, void *private_data) > > /* increase nelements by fillfactor, want to store nelements elements */ > > size = Min((double) SH_MAX_SIZE, ((double) nelements) / SH_FILLFACTOR); > > > > - SH_COMPUTE_PARAMETERS(tb, size); > > + size = SH_COMPUTE_SIZE(size); > > > > - tb->data = (SH_ELEMENT_TYPE *) SH_ALLOCATE(tb, sizeof(SH_ELEMENT_TYPE) * tb->size); > > + tb->data = (SH_ELEMENT_TYPE *) SH_ALLOCATE(tb, sizeof(SH_ELEMENT_TYPE) * size); > > > > + SH_UPDATE_PARAMETERS(tb, size); > > return tb; > > } > > Maybe add a comment explaining why it's important to update parameters after > allocating? Perhaps something like this: + /* + * Update parameters _after_ allocation succeeds; prevent + * bogus/corrupted state. + */ + SH_UPDATE_PARAMETERS(tb, size); Best regards, Gurjeet http://Gurje.et
On 2023-11-17 13:00:19 -0800, Jeff Davis wrote: > Please tell me if you think the use of simplehash for a search_path > cache is the wrong tool for the job. No, seems fine. I just was curious - as you said, the older existing users won't ever care about this case.