Thread: Dumb mistakes in WalSndWriteData()
Hi, I^Wsomebody appears to have made a number of dumb mistakes in WalSndWriteData(), namely: 1) The timestamp is set way too late, after pq_putmessage_noblock(). That'll sometimes work, sometimes not. I have noidea how that ended up happening. It's eye-wateringly dumb. 2) We only do WalSndKeepaliveIfNecessary() if we're blocked on socket IO. But on a long-lived connection that might be alot of data, we should really do that once *before* trying to send the payload in the first place. 3) Similarly to 2) it might be worthwhile checking for interrupts everytime, not just when blocked on network IO. See also: http://archives.postgresql.org/message-id/CAMsr%2BYE2dSfHVr7iEv1GSPZihitWX-PMkD9QALEGcTYa%2Bsdsgg%40mail.gmail.com Greetings, Andres Freund
On Mon, Oct 31, 2016 at 4:59 AM, Andres Freund <andres@anarazel.de> wrote: > I^Wsomebody appears to have made a number of dumb mistakes in > WalSndWriteData(), namely: > 1) The timestamp is set way too late, after > pq_putmessage_noblock(). That'll sometimes work, sometimes not. I > have no idea how that ended up happening. It's eye-wateringly dumb. > > 2) We only do WalSndKeepaliveIfNecessary() if we're blocked on socket > IO. But on a long-lived connection that might be a lot of data, we > should really do that once *before* trying to send the payload in the > first place. > > 3) Similarly to 2) it might be worthwhile checking for interrupts > everytime, not just when blocked on network IO. > > See also: > http://archives.postgresql.org/message-id/CAMsr%2BYE2dSfHVr7iEv1GSPZihitWX-PMkD9QALEGcTYa%2Bsdsgg%40mail.gmail.com Do you intend to do something about these problems? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 2016-10-31 09:44:16 -0400, Robert Haas wrote: > On Mon, Oct 31, 2016 at 4:59 AM, Andres Freund <andres@anarazel.de> wrote: > > I^Wsomebody appears to have made a number of dumb mistakes in > > WalSndWriteData(), namely: > > 1) The timestamp is set way too late, after > > pq_putmessage_noblock(). That'll sometimes work, sometimes not. I > > have no idea how that ended up happening. It's eye-wateringly dumb. > > > > 2) We only do WalSndKeepaliveIfNecessary() if we're blocked on socket > > IO. But on a long-lived connection that might be a lot of data, we > > should really do that once *before* trying to send the payload in the > > first place. > > > > 3) Similarly to 2) it might be worthwhile checking for interrupts > > everytime, not just when blocked on network IO. > > > > See also: > > http://archives.postgresql.org/message-id/CAMsr%2BYE2dSfHVr7iEv1GSPZihitWX-PMkD9QALEGcTYa%2Bsdsgg%40mail.gmail.com > > Do you intend to do something about these problems? At least 1) and 2), yes. I basically wrote this email to have something to reference in my todo list... Greetings, Andres Freund
On Mon, Oct 31, 2016 at 10:54 PM, Andres Freund <andres@anarazel.de> wrote: > On 2016-10-31 09:44:16 -0400, Robert Haas wrote: >> On Mon, Oct 31, 2016 at 4:59 AM, Andres Freund <andres@anarazel.de> wrote: >> > I^Wsomebody appears to have made a number of dumb mistakes in >> > WalSndWriteData(), namely: >> > 1) The timestamp is set way too late, after >> > pq_putmessage_noblock(). That'll sometimes work, sometimes not. I >> > have no idea how that ended up happening. It's eye-wateringly dumb. >> > >> > 2) We only do WalSndKeepaliveIfNecessary() if we're blocked on socket >> > IO. But on a long-lived connection that might be a lot of data, we >> > should really do that once *before* trying to send the payload in the >> > first place. >> > >> > 3) Similarly to 2) it might be worthwhile checking for interrupts >> > everytime, not just when blocked on network IO. >> > >> > See also: >> > http://archives.postgresql.org/message-id/CAMsr%2BYE2dSfHVr7iEv1GSPZihitWX-PMkD9QALEGcTYa%2Bsdsgg%40mail.gmail.com >> >> Do you intend to do something about these problems? > > At least 1) and 2), yes. I basically wrote this email to have something > to reference in my todo list... Just looking at this thread, 1) and 2) is actually something like the attached? -- Michael