Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding - Mailing list pgsql-hackers

From Ajin Cherian
Subject Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
Date
Msg-id CAFPTHDaLCL-CkrHeP5NEik4GFwVPEm8rewcvPNruwjHSaF1gNw@mail.gmail.com
Whole thread Raw
In response to Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding  (li jie <ggysxcq@gmail.com>)
Responses Re: Proposal: Filter irrelevant change before reassemble transactions during logical decoding
List pgsql-hackers


Of course you can, but this will only convert disk space into memory space.
 For details, please see the case in Email [1].

[1] https://www.postgresql.org/message-id/CAGfChW51P944nM5h0HTV9HistvVfwBxNaMt_s-OZ9t%3DuXz%2BZbg%40mail.gmail.com

Regards, lijie


Hi lijie,

Overall, I think the patch is a good improvement. Some comments from first run through of patch:
1. The patch no longer applies cleanly, please rebase.

2. While testing the patch, I saw something strange. If I try to truncate a table that is published. I still see the message:
2024-03-18 22:25:51.243 EDT [29385] LOG:  logical filter change by table pg_class

This gives the impression that the truncate operation on the published table has been filtered but it hasn't. Also the log message needs to be reworded. Maybe, "Logical filtering change by non-published table <relation_name>"

3. Below code:
@@ -1201,11 +1343,14 @@ DecodeMultiInsert(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+
+ if (FilterByTable(ctx, change))
+ continue;;

extra semi-colon after continue.

4. I am not sure if this is possible, but is there a way to avoid the overhead in the patch if the publication publishes "ALL TABLES"?

5. In function: pgoutput_table_filter() - this code appears to be filtering out not just unpublished tables but also applying row based filters on published tables as well. Is this really within the scope of the feature?

regards,
Ajin Cherian
Fujitsu Australia

pgsql-hackers by date:

Previous
From: Richard Guo
Date:
Subject: Re: A problem about partitionwise join
Next
From: John Naylor
Date:
Subject: Re: add AVX2 support to simd.h