Postgres Pro
Downloads
Home   >   mailing lists

Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes? - Mailing list pgsql-hackers

From Bharath Rupireddy
Subject Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?
Date
Msg-id CALj2ACXEBwY_bM3kmZEkYpcXsM+yGitpYHi4FdT6MSk6YRtKTQ@mail.gmail.com
Whole thread Raw
In response to Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?  (Andres Freund <andres@anarazel.de>)
Responses Re: Use pg_pwritev_with_retry() instead of write() in dir_open_for_write() to avoid partial writes?
List pgsql-hackers
Tree view 
On Wed, Feb 15, 2023 at 6:25 AM Andres Freund <andres@anarazel.de> wrote:
>
> > Done, PSA v2 patch.
>
> This feels way too complicated to me.  How about something more like the
> attached?

Thanks. I kind of did cost analysis of v2 and v3:

Input: zero-fill a file of size 256*8K.
v2 patch:
iovec initialization with zerobuf for loop - 1 time
zero-fill 32 blocks at once - 8 times
stack memory - sizeof(PGAlignedBlock) + sizeof(struct iovec) * PG_IOV_MAX

v3 patch:
iovec initialization with zerobuf for loop - 8 times (7 times more
than v2 patch)
zero-fill 32 blocks at once - 8 times (no change from v2 patch)
stack memory - sizeof(PGAlignedBlock) + sizeof(struct iovec) *
PG_IOV_MAX (no change from v2 patch)

The v3 patch reduces initialization of iovec array elements which is a
clear win when pg_pwrite_zeros is called for sizes less than BLCKSZ
many times (I assume this is what is needed for the relation extension
lock improvements feature). However, it increases the number of iovec
initialization with zerobuf for the cases when pg_pwrite_zeros is
called for sizes far greater than BLCKSZ (for instance, WAL file
initialization).

FWIW, I attached v4 patch, a simplified version of the v2  - it
initializes all the iovec array elements if the total blocks to be
written crosses lengthof(iovec array), otherwise it initializes only
the needed blocks.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
Attachment

pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: [EXTERNAL] Re: [PATCH] Support using "all" for the db user in pg_ident.conf
Next
From: Kyotaro Horiguchi
Date:
Subject: Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)