Re: enhancing pg_basebackup speeds up to ~23Gbps (small fixes + io_uring/Direct I/O) - Mailing list pgsql-hackers

From Heikki Linnakangas
Subject Re: enhancing pg_basebackup speeds up to ~23Gbps (small fixes + io_uring/Direct I/O)
Date
Msg-id 2c976a26-c11c-49b6-9af6-8431914e8690@iki.fi
Whole thread
In response to Re: enhancing pg_basebackup speeds up to ~23Gbps (small fixes + io_uring/Direct I/O)  (Jakub Wartak <jakub.wartak@enterprisedb.com>)
Responses Re: enhancing pg_basebackup speeds up to ~23Gbps (small fixes + io_uring/Direct I/O)
List pgsql-hackers
On 06/10/2026 16:10, Jakub Wartak wrote:
> further more sendFile() has this too above the "if (cnt < BLCKSZ)"
>      /*
>      * If we get a partial read, that must mean that the relation is
>      * being truncated. Ultimately, it should be truncated to a
>      * multiple of BLCKSZ, since this path should only be reached for
>      * relation files, but we might transiently observe an
>      * intermediate value.
>      *
>      * It should be fine to treat this just as if the entire block had
>      * been truncated away - i.e. fill this and all later blocks with
>      * zeroes. WAL replay will fix things up.
>      */
> 
> and below that, there is ereport() that says just checksums couldn't be
> verified. I mean in 0004 we are just growing and so it assumes we shouldn't
> get any problems with this code. It looks like it is written while expecting
> proper pread()/pwrite() atomicity when extending for up to 32kb, or am I
> missing something?

I think that comment is wrong. A short read is possible for other 
reasons than a truncated file, e.g. if the read is interrupted by a 
signal. In the past, we've made that same assumption that you never get 
a short read on a BLCKSZ-sized read in other places too, but it was 
always questionable. Thomas Munro fixed it for smgrread() in commit 
4908c58720.

I think we should now fix that in read_file_data_into_buffer(), too. 
Make it retry, so that it never returns fewer bytes than requested 
except for EOF. With a larger buffer, a short read becomes more likely.

- Heikki




pgsql-hackers by date:

Previous
From: Vadim Ponomarev
Date:
Subject: Re: Reduce SyncRepLock contention on the commit path
Next
From: Pavel Borisov
Date:
Subject: Re: [PATCH] intXshr, intXshl: return error on shift count out of range