Re: pg_preadv() and pg_pwritev() - Mailing list pgsql-hackers

From Andres Freund
Subject Re: pg_preadv() and pg_pwritev()
Date
Msg-id 20201220224040.evr5pduibvm6dze5@alap3.anarazel.de
Whole thread Raw
In response to Re: pg_preadv() and pg_pwritev()  (Thomas Munro <thomas.munro@gmail.com>)
Responses RE: pg_preadv() and pg_pwritev()
Re: pg_preadv() and pg_pwritev()
List pgsql-hackers
Hi,

On 2020-12-20 16:26:42 +1300, Thomas Munro wrote:
> > 1. port.h cannot assume that <limits.h> has already been included;
> > nor do I want to fix that by including <limits.h> there.  Do we
> > really need to define a fallback value of IOV_MAX?  If so,
> > maybe the answer is to put the replacement struct iovec and
> > IOV_MAX in some new header.
> 
> Ok, I moved all this stuff into port/pg_uio.h.

Can we come up with a better name than 'uio'? I find that a not exactly
meaningful name.

Or perhaps we could just leave the functions in port/port.h, but extract
the value of the define at configure time? That way only pread.c /
pwrite.c would need it.


> > 3. The patch as given won't prove anything except that the code
> > compiles.  Is it worth fixing at least one code path to make
> > use of pg_preadv and pg_pwritev, so we can make sure this code
> > is tested before there's layers of other new code on top?
> 
> OK, here's a patch to zero-fill fresh WAL segments with pwritev().

I think that's a good idea. However, I see two small issues: 1) If we
write larger amounts at once, we need to handle partial writes. That's a
large enough amount of IO that it's much more likely to hit a memory
shortage or such in the kernel - we had to do that a while a go for WAL
writes (which can also be larger), if memory serves.

Perhaps we should have pg_pwritev/readv unconditionally go through
pwrite.c/pread.c and add support for "continuing" partial writes/reads
in one central place?


> I'm drawing a blank on trivial candidate uses for preadv(), without
> infrastructure from later patches.

Can't immediately think of something either.


Greetings,

Andres Freund



pgsql-hackers by date:

Previous
From: Masahiko Sawada
Date:
Subject: Re: Commit fest manager for 2021-01
Next
From: Alastair Turner
Date:
Subject: Re: Proposed patch for key managment