Hi Tom,
Thank you for the information.
And I'm sorry for not replying sooner. It took time to recall this issue.
(2014/05/08 8:58), Tom Lane wrote:
> I started thinking about this issue again after seeing a similar report at
> http://www.postgresql.org/message-id/flat/850BF81CE7324F81AF2A44C8660CF2EE@dell2
> and this time I think I see the problem. Look at the loop in pqReadData()
> created by this test:
>
> I've been able to reproduce buffer bloat by inserting some delay into
> this loop (I put "usleep(10000);" in front of the goto above) and then
> sending a steady stream of circa-50KB data messages.
Oh, I'm glad to hear that. Thank you for the explanation.
> exactly the opposite of what generally happens on Unix machines. Perhaps
> Windows schedules differently? Or maybe the reason we're seeing this now,
It might be so. Although I'm not sure why it occurred, I think our
conditions were rare:
Hypervisor: VMware vSphere Hypervisor 5.1
Allocated vCPUs: 1
OS: Windows Server 2012 or 2008R2 (64bit)
Run pg_dump under high load or run pb_dump with the Low priority using
Windows Task Manager
> Anyway, I still don't like your original suggestion of simply skipping
> the pqCheckInBufferSpace call; that's just losing the knowledge we have of
> how big the next message will be. Rather I think what we are missing here
> is that we should deduct the excess space to the left of inStart, and/or
> forcibly left-justify the buffer, before deciding that we need to enlarge
> the buffer. I think it'd be safe for pqCheckInBufferSpace to do that
> (ie move the data within the buffer), though I need to look at the callers
> more closely to be sure.
I agree with you.
The new pqCheckInBufferSpace seems to update inStart, inEnd, and inCursor.
I think it would be OK unless the callers assume they won't be updated.
Thank you.
Shin-ichi
