Re: Show WAL write and fsync stats in pg_stat_io - Mailing list pgsql-hackers
From | Nazir Bilal Yavuz |
---|---|
Subject | Re: Show WAL write and fsync stats in pg_stat_io |
Date | |
Msg-id | CAN55FZ1ny+3kpdm5X3nGZ2Jp3wxZO-744eFgxktS6YQ3=OKR-A@mail.gmail.com Whole thread Raw |
In response to | Re: Show WAL write and fsync stats in pg_stat_io (Nazir Bilal Yavuz <byavuz81@gmail.com>) |
Responses |
Re: Show WAL write and fsync stats in pg_stat_io
Re: Show WAL write and fsync stats in pg_stat_io |
List | pgsql-hackers |
Hi, On Mon, 19 Feb 2024 at 10:28, Nazir Bilal Yavuz <byavuz81@gmail.com> wrote: > > Hi, > > On Thu, 18 Jan 2024 at 04:22, Michael Paquier <michael@paquier.xyz> wrote: > > > > On Wed, Jan 17, 2024 at 03:20:39PM +0300, Nazir Bilal Yavuz wrote: > > > I agree with your points. While the other I/O related work is > > > happening we can discuss what we should do in the variable op_byte > > > cases. Also, this is happening only for WAL right now but if we try to > > > extend pg_stat_io in the future, that problem possibly will rise > > > again. So, it could be good to come up with a general solution, not > > > only for WAL. > > > > Okay, I've marked the patch as RwF for this CF. > > I wanted to inform you that the 73f0a13266 commit changed all WALRead > calls to read variable bytes, only the WAL receiver was reading > variable bytes before. I want to start working on this again if possible. I will try to summarize the current status: * With the 73f0a13266 commit, the WALRead() function started to read variable bytes in every case. Before, only the WAL receiver was reading variable bytes. * With the 91f2cae7a4 commit, WALReadFromBuffers() is merged. We were discussing what we have to do when this is merged. It is decided that WALReadFromBuffers() does not call pgstat_report_wait_start() because this function does not perform any IO [1]. We may follow the same logic by not including these to pg_stat_io? * With the b5a9b18cd0 commit, streaming I/O is merged but AFAIK this does not block anything related to putting WAL stats in pg_stat_io. If I am not missing any new changes, the only problem is reading variable bytes now. We have discussed a couple of solutions: 1- Change op_bytes to something like -1, 0, 1, NULL etc. and document that this means some variable byte I/O is happening. I kind of dislike this solution because if the *only* read I/O is happening in variable bytes, it will look like write and extend I/Os are happening in variable bytes as well. As a solution, it could be documented that only read I/Os could happen in variable bytes for now. 2- Use op_bytes_[read | write | extend] columns instead of one op_bytes column, also use the first solution. This can solve the first solution's weakness but it introduces two more columns. This is more future proof compared to the first solution if there is a chance that some variable I/O could happen in write and extend calls as well in the future. 3- Create a new pg_stat_io_wal view to put WAL I/Os here instead of pg_stat_io. pg_stat_io could remain untouchable and we will have flexibility to edit this new view as much as we want. But the original aim of the pg_stat_io is evaluating all I/O from a single view and adding a new view breaks this aim. I hope that I did not miss anything and my explanations are clear. Any kind of feedback would be appreciated. [1] https://www.postgresql.org/message-id/CAFiTN-sE7CJn-ZFj%2B-0Wv6TNytv_fp4n%2BeCszspxJ3mt77t5ig%40mail.gmail.com -- Regards, Nazir Bilal Yavuz Microsoft
pgsql-hackers by date: