Thread: pg_stat_wal_receiver and flushedUpto/writtenUpto
Hi, As discussed in the thread that introduced d140f2f3 to rename receivedUpto to flushedUpto and add writtenUpto to the WAL receiver's shared memory information, the equivalent columns in pg_stat_wal_receiver have not been renamed: https://www.postgresql.org/message-id/CA+hUKGJ06d3h5JeOtAv4h52n0vG1jOPZxqMCn5FySJQUVZA32w@mail.gmail.com When I have implemented this system view, the idea was to keep a one-one mapping between the SQL interface and the shmem info even if we are not compatible with past versions, hence I think that before beta1 we had better fix that and: - rename received_lsn to flushed_lsn. - add one column for writtenUpto. Attached is a patch to do that. This needs also a catalog version bump, and I am adding an open item. Thanks, -- Michael
Attachment
On 2020-May-15, Michael Paquier wrote: > As discussed in the thread that introduced d140f2f3 to rename > receivedUpto to flushedUpto and add writtenUpto to the WAL receiver's > shared memory information, the equivalent columns in > pg_stat_wal_receiver have not been renamed: > When I have implemented this system view, the idea was to keep a > one-one mapping between the SQL interface and the shmem info even if > we are not compatible with past versions, hence I think that before > beta1 we had better fix that and: > - rename received_lsn to flushed_lsn. > - add one column for writtenUpto. Why do you put the column at the end? I would put written_lsn before flushed_lsn. Since this requires a catversion bump, I think it'd be best to do it before beta1 next week. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, May 15, 2020 at 01:43:11PM -0400, Alvaro Herrera wrote: > Why do you put the column at the end? I would put written_lsn before > flushed_lsn. Fine by me. I was thinking yesterday about putting the written position after the flushed one, and finished with something that maps with the structure. > Since this requires a catversion bump, I think it'd be best to do it > before beta1 next week. Yes. What do you think of the attached? -- Michael
Attachment
On 2020-May-16, Michael Paquier wrote: > On Fri, May 15, 2020 at 01:43:11PM -0400, Alvaro Herrera wrote: > > Why do you put the column at the end? I would put written_lsn before > > flushed_lsn. > > Fine by me. I was thinking yesterday about putting the written > position after the flushed one, and finished with something that maps > with the structure. IIRC the only reason to put the written LSN where it is is so that it's below the mutex, to indicate it's not protected by it. Conceptually, the written LSN is "before" the flushed LSN, which is why I propose to put it ahead of it. > > Since this requires a catversion bump, I think it'd be best to do it > > before beta1 next week. > > Yes. What do you think of the attached? Yeah, that seems good (I didn't verify the boilerplate in pg_stat_get_wal_receiver or pg_proc.dat). I propose + Last write-ahead log location already received and written to + disk, but not flushed. This should not be used for data + integrity checks. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, May 15, 2020 at 07:34:46PM -0400, Alvaro Herrera wrote: > IIRC the only reason to put the written LSN where it is is so that it's > below the mutex, to indicate it's not protected by it. Conceptually, > the written LSN is "before" the flushed LSN, which is why I propose to > put it ahead of it. Sure. My point was mainly that it is easier to compare if we are missing any fields in the view and the order is respected. But it makes also sense to do things your way, so let's do that. > Yeah, that seems good (I didn't verify the boilerplate in > pg_stat_get_wal_receiver or pg_proc.dat). I propose > > + Last write-ahead log location already received and written to > + disk, but not flushed. This should not be used for data > + integrity checks. Thanks. If there are no objections, I'll revisit that tomorrow and apply it with those changes, just in time for beta1. -- Michael
Attachment
On Sat, May 16, 2020 at 10:15:47AM +0900, Michael Paquier wrote: > Thanks. If there are no objections, I'll revisit that tomorrow and > apply it with those changes, just in time for beta1. Okay, done this part then. -- Michael
Attachment
On 2020/05/17 10:08, Michael Paquier wrote: > On Sat, May 16, 2020 at 10:15:47AM +0900, Michael Paquier wrote: >> Thanks. If there are no objections, I'll revisit that tomorrow and >> apply it with those changes, just in time for beta1. > > Okay, done this part then. I found that "received_lsn" is still used in high-availability.sgml. We should apply the following change in high-availability? - view's <literal>received_lsn</literal> indicates that WAL is being + view's <literal>flushed_lsn</literal> indicates that WAL is being BTW, we have pg_last_wal_receive_lsn() that returns the same lsn as pg_stat_wal_receiver.flushed_lsn. Previously both used the term "receive" in their names, but currently not. IMO it's better to use the same term in those names for the consistency, but it's not good idea to rename pg_last_wal_receive_lsn() to something like pg_last_wal_receive_lsn(). I have no better idea for now. So I'm ok with the current names. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION
On Tue, May 19, 2020 at 11:38:52PM +0900, Fujii Masao wrote: > I found that "received_lsn" is still used in high-availability.sgml. > We should apply the following change in high-availability? > > - view's <literal>received_lsn</literal> indicates that WAL is being > + view's <literal>flushed_lsn</literal> indicates that WAL is being Oops, thanks. Will fix. > BTW, we have pg_last_wal_receive_lsn() that returns the same lsn as > pg_stat_wal_receiver.flushed_lsn. Previously both used the term "receive" > in their names, but currently not. IMO it's better to use the same term in > those names for the consistency, but it's not good idea to rename > pg_last_wal_receive_lsn() to something like pg_last_wal_receive_lsn(). > I have no better idea for now. So I'm ok with the current names. I think you mean renaming pg_last_wal_receive_lsn() to something like pg_last_wal_flushed_lsn(), no? This name may become confusing because we lose the "receive" idea in the function, that we have with the "receiver" part of pg_stat_wal_receiver. Maybe something like that, though that's long: - pg_last_wal_receive_flushed_lsn() - pg_last_wal_receive_written_lsn() Anyway, a rename of this function does not strike me as strongly necessary, as that's less tied with the shared memory structure, and we document that pg_last_wal_receive_lsn() tracks the current LSN received and flushed. I am actually wondering if in the future it may not be better to remove this function, but it has no maintenance cost either so I would just let it as-is. -- Michael
Attachment
On 2020/05/20 8:31, Michael Paquier wrote: > On Tue, May 19, 2020 at 11:38:52PM +0900, Fujii Masao wrote: >> I found that "received_lsn" is still used in high-availability.sgml. >> We should apply the following change in high-availability? >> >> - view's <literal>received_lsn</literal> indicates that WAL is being >> + view's <literal>flushed_lsn</literal> indicates that WAL is being > > Oops, thanks. Will fix. Thanks for the fix! > >> BTW, we have pg_last_wal_receive_lsn() that returns the same lsn as >> pg_stat_wal_receiver.flushed_lsn. Previously both used the term "receive" >> in their names, but currently not. IMO it's better to use the same term in >> those names for the consistency, but it's not good idea to rename >> pg_last_wal_receive_lsn() to something like pg_last_wal_receive_lsn(). >> I have no better idea for now. So I'm ok with the current names. > > I think you mean renaming pg_last_wal_receive_lsn() to something like > pg_last_wal_flushed_lsn(), no? No, that's not good idea, as I told upthread. > This name may become confusing because > we lose the "receive" idea in the function, that we have with the > "receiver" part of pg_stat_wal_receiver. Maybe something like that, > though that's long: > - pg_last_wal_receive_flushed_lsn() > - pg_last_wal_receive_written_lsn() Yes, that's long. > Anyway, a rename of this function does not strike me as strongly > necessary, as that's less tied with the shared memory structure, and > we document that pg_last_wal_receive_lsn() tracks the current LSN > received and flushed. I am actually wondering if in the future it may > not be better to remove this function, but it has no maintenance > cost either so I would just let it as-is. Yeah, agreed. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION