Re: Direct I/O - Mailing list pgsql-hackers

From Andres Freund
Subject Re: Direct I/O
Date
Msg-id 20230408180816.5qzx7hhulhsdpv2u@awork3.anarazel.de
Whole thread Raw
In response to Re: Direct I/O  (Andres Freund <andres@anarazel.de>)
Responses Re: Direct I/O
Re: Direct I/O
List pgsql-hackers
Hi,

On 2023-04-07 23:04:08 -0700, Andres Freund wrote:
> There were some failures in CI (e.g. [1] (and perhaps also bf, didn't yet
> check), about "no unpinned buffers available".  I was worried for a moment
> that this could actually be relation to the bulk extension patch.
> 
> But it looks like it's older - and not caused by direct_io support (except by
> way of the test existing). I reproduced the issue locally by setting s_b even
> lower, to 16 and made the ERROR a PANIC.
>
> [backtrace]
> 
> If you look at log_newpage_range(), it's not surprising that we get this error
> - it pins up to 32 buffers at once.
> 
> Afaics log_newpage_range() originates in 9155580fd5fc, but this caller is from
> c6b92041d385.
> 
> 
> It doesn't really seem OK to me to unconditionally pin 32 buffers. For the
> relation extension patch I introduced LimitAdditionalPins() to deal with this
> concern. Perhaps it needs to be exposed and log_newpage_buffers() should use
> it?
> 
> 
> Do we care about fixing this in the backbranches? Probably not, given there
> haven't been user complaints?

Here's a quick prototype of this approach. If we expose LimitAdditionalPins(),
we'd probably want to add "Buffer" to the name, and pass it a relation, so
that it can hand off LimitAdditionalLocalPins() when appropriate? The callsite
in question doesn't need it, but ...

Without the limiting of pins the modified 004_io_direct.pl fails 100% of the
time for me.

Presumably the reason it fails occasionally with 256kB of shared buffers
(i.e. NBuffers=32) is that autovacuum or checkpointer briefly pins a single
buffer. As log_newpage_range() thinks it can just pin 32 buffers
unconditionally, it fails in that case.

Greetings,

Andres Freund

Attachment

pgsql-hackers by date:

Previous
From: Stephen Frost
Date:
Subject: Re: longfin missing gssapi_ext.h
Next
From: Melanie Plageman
Date:
Subject: Re: Parallel Full Hash Join