Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding - Mailing list pgsql-hackers

From Ajin Cherian
Subject Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date
Msg-id CAFPTHDZZxQ_kCXi3mF8gdDQPJ7DzMe1Mgif1LODeMYyqD9QkDg@mail.gmail.com
Whole thread Raw
In response to Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding  (Ajin Cherian <itsajin@gmail.com>)
Responses Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
List pgsql-hackers


On Tue, Mar 19, 2024 at 1:49 PM Ajin Cherian <itsajin@gmail.com> wrote:


Of course you can, but this will only convert disk space into memory space.
 For details, please see the case in Email [1].

[1] https://www.postgresql.org/message-id/CAGfChW51P944nM5h0HTV9HistvVfwBxNaMt_s-OZ9t%3DuXz%2BZbg%40mail.gmail.com

Regards, lijie



In some testing, I see a crash:
(gdb) bt
#0  0x00007fa5bcbfd277 in raise () from /lib64/libc.so.6
#1  0x00007fa5bcbfe968 in abort () from /lib64/libc.so.6
#2  0x00000000009e0940 in ExceptionalCondition (
    conditionName=conditionName@entry=0x7fa5ab8b9842 "RelationSyncCache != NULL",
    fileName=fileName@entry=0x7fa5ab8b9820 "pgoutput.c", lineNumber=lineNumber@entry=1991)
    at assert.c:66
#3  0x00007fa5ab8b7804 in get_rel_sync_entry (data=data@entry=0x2492288,
    relation=relation@entry=0x7fa5be30a768) at pgoutput.c:1991
#4  0x00007fa5ab8b7cda in pgoutput_table_filter (ctx=<optimized out>, relation=0x7fa5be30a768,
    change=0x24c5c20) at pgoutput.c:1671
#5  0x0000000000813761 in filter_by_table_cb_wrapper (ctx=ctx@entry=0x2491fd0,
    relation=relation@entry=0x7fa5be30a768, change=change@entry=0x24c5c20) at logical.c:1268
#6  0x000000000080e20f in FilterByTable (ctx=ctx@entry=0x2491fd0, change=change@entry=0x24c5c20)
    at decode.c:690
#7  0x000000000080e8e3 in DecodeInsert (ctx=ctx@entry=0x2491fd0, buf=buf@entry=0x7fff0db92550)
    at decode.c:1070
#8  0x000000000080f43d in heap_decode (ctx=ctx@entry=0x2491fd0, buf=buf@entry=0x7fff0db92550)
    at decode.c:485
#9  0x000000000080eca6 in LogicalDecodingProcessRecord (ctx=ctx@entry=0x2491fd0, record=0x2492368)
    at decode.c:118
#10 0x000000000081338f in DecodingContextFindStartpoint (ctx=ctx@entry=0x2491fd0) at logical.c:672
#11 0x000000000083c650 in CreateReplicationSlot (cmd=cmd@entry=0x2490970) at walsender.c:1323
#12 0x000000000083fd48 in exec_replication_command (
    cmd_string=cmd_string@entry=0x239c880 "CREATE_REPLICATION_SLOT \"pg_16387_sync_16384_7371301304766135621\" LOGICAL pgoutput (SNAPSHOT 'use')") at walsender.c:2116

The reason for the crash is that the RelationSyncCache was NULL prior to reaching a consistent point.
Hi li jie, I see that you created a new thread with an updated version of this patch [1]. I used that patch and addressed the crash seen above, rebased the patch and addressed a few other comments.
I'm happy to help you with this patch and address comments if you are not available.

regards,
Ajin Cherian
Fujitsu Australia
Attachment

pgsql-hackers by date:

Previous
From: David Rowley
Date:
Subject: Re: Path to unreverting "Allow planner to use Merge Append to efficiently implement UNION"
Next
From: Ashutosh Bapat
Date:
Subject: Re: apply_scanjoin_target_to_paths and partitionwise join