RE: effective_io_concurrency and NVMe devices - Mailing list pgsql-hackers

From Jakub Wartak
Subject RE: effective_io_concurrency and NVMe devices
Date
Msg-id AM8PR07MB82488639DCCBE3BF1306DD3EF6DE9@AM8PR07MB8248.eurprd07.prod.outlook.com
Whole thread Raw
In response to Re: effective_io_concurrency and NVMe devices  (Nathan Bossart <nathandbossart@gmail.com>)
List pgsql-hackers
Hi Nathan,

> > NVMe devices have a maximum queue length of 64k:
[..]
> > but our effective_io_concurrency maximum is 1,000:
[..]
> > Should we increase its maximum to 64k?  Backpatched?  (SATA has a
> > maximum queue length of 256.)
>
> If there are demonstrable improvements with higher values, this seems
> reasonable to me.  I would even suggest removing the limit completely so
> this doesn't need to be revisited in the future.

Well, are there any? I remember playing with this (although for ANALYZE Stephen's case [1]) and got quite contrary
results[2] -- see going to 16 from 8 actually degraded performance. 
I somehow struggle to understand how 1000+ fadvise() syscalls would be a net benefit on storage with latency of ~0.1..
0.3msas each syscall on it's own is overhead (quite contrary, it should help on very slow one?)  
Pardon if I'm wrong (I don't have time to lookup code now), but maybe Bitmap Scans/fadvise() logic would first need to
performsome fadvise() offset/length aggregations to bigger fadvise() syscalls and in the end real hardware observable
I/Oconcurrency would be bigger (assuming that fs/LVM/dm/mq layer would split that into more parallel IOs). 

-J.

[1] - https://commitfest.postgresql.org/30/2799/
[2] -
https://www.postgresql.org/message-id/flat/VI1PR0701MB69603A433348EDCF783C6ECBF6EF0@VI1PR0701MB6960.eurprd07.prod.outlook.com







pgsql-hackers by date:

Previous
From: Peter Smith
Date:
Subject: Re: Multi-Master Logical Replication
Next
From: John Naylor
Date:
Subject: Re: PG15 beta1 sort performance regression due to Generation context change