On Fri, 12 Sep 2025 14:09:47 +0800
Zhang Mingli <zmlpostgres@gmail.com> wrote:
> Hi, all
>
> On Aug 30, 2025 at 04:04 +0800, Yugo Nagata <nagata@sraoss.co.jp>, wrote:
> >
> > Anyway, I will start rebasing the patches, reorganizing the patch set,
> > and applying fixes made in pg_ivm [2].
>
> First of all, we would like to express our sincere gratitude for your continuous efforts and contributions to the IVM
feature.
> Our AQUMV (Answer Query Using Materialized Views) functionality in Apache Cloudberry is built directly upon your
foundationalwork.
>
>
> We recently encountered an issue in one of our customer's production environments.
> The problem occurs in the function CreateIvmTriggersOnBaseTablesRecurse(), where a Bitmapset relidsis used to
record rte->relid.
>
> ```
> case T_RangeTblRef:
> {
> int rti = ((RangeTblRef *) node)->rtindex;
> RangeTblEntry *rte = rt_fetch(rti, qry->rtable);
>
> if (rte->rtekind == RTE_RELATION && !bms_is_member(rte->relid, *relids))
> {
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_INSERT, TRIGGER_TYPE_BEFORE, ex_lock);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_DELETE, TRIGGER_TYPE_BEFORE, ex_lock);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_UPDATE, TRIGGER_TYPE_BEFORE, ex_lock);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_TRUNCATE, TRIGGER_TYPE_BEFORE, true);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_INSERT, TRIGGER_TYPE_AFTER, ex_lock);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_DELETE, TRIGGER_TYPE_AFTER, ex_lock);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_UPDATE, TRIGGER_TYPE_AFTER, ex_lock);
> CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_TRUNCATE, TRIGGER_TYPE_AFTER, true);
>
> *relids = bms_add_member(*relids, rte->relid);
> }
> }
> ```
> However, the Bitmapset structure is typically designed to store small integers (such as relation index),
whereas rte->relidrepresentsthe Oid of a relation.
> For instance, when a table is created immediately after initializing a new cluster, its Oid might be 17019. Storing
sucha value in a Bitmapset consumes approximately 0.2MB of memory when creating an IVM.
>
> (gdb) p bmsToString(((Bitmapset *) relids))
> $13 = 0x5643d7bbb070 "(b 17019)"
> (gdb) p *((Bitmapset *) relids)
> $14 = {nwords = 266, words = 0x5643d79d1bd8}
>
> This memory usage becomes even more significant when dealing with larger Oids.
> Moreover, a more critical issue arises when the Oid exceeds 0x7FFFFFFF. Since Oid is an unsigned integer, but the
Bitmapsetexpects signed integers, an assertion failure will occur in such cases.
>
>
> We have encountered similar requirements in our own scenarios and eventually switched to using a HTAB implementation.
Hopethis proves useful for your work.
Sorry for the late reply. While reviewing the discussion around reworking this,
I noticed that I never replied to this comment.
That makes sense. I'll update the patch accordingly.
Regards,
Yugo Nagata
--
Yugo Nagata <nagata@sraoss.co.jp>
