On Wed, Jul 8, 2020 at 7:31 PM Ajin Cherian <itsajin@gmail.com> wrote:
Thanks for showing the interest in patch. How have you ensured that streaming is happening? I don't think the proposed patch can ensure it for every case because we also rely on logical_decoding_work_mem to decide whether to stream/spill, see ReorderBufferCheckMemoryLimit. I think with your patch it will allow streaming for cases where we have large amount of WAL to decode.
Maybe I missed something but I looked at ReorderBufferCheckMemoryLimit, even there it checks the same function ReorderBufferCanStream () and decides whether to stream or spill. Did I miss something?
while (rb->size >= logical_decoding_work_mem * 1024L)
{
/*
* Pick the largest transaction (or subtransaction) and evict it from
* memory by streaming, if supported. Otherwise, spill to disk.
*/
if (ReorderBufferCanStream(rb) &&
(txn = ReorderBufferLargestTopTXN(rb)) != NULL)
{
/* we know there has to be one, because the size is not zero */
