Re: PATCH: logical_work_mem and logical streaming of largein-progress transactions - Mailing list pgsql-hackers

On Fri, Nov 15, 2019 at 3:50 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Thu, Nov 14, 2019 at 3:40 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> >
> > Apart from this, I have another question in
> > 0003-Issue-individual-invalidations-with-wal_level-logical.patch
> >
> > @@ -543,6 +588,18 @@ RegisterSnapshotInvalidation(Oid dbId, Oid relId)
> >  {
> >   AddSnapshotInvalidationMessage(&transInvalInfo->CurrentCmdInvalidMsgs,
> >      dbId, relId);
> > +
> > + /* Issue an invalidation WAL record (when wal_level=logical) */
> > + if (XLogLogicalInfoActive())
> > + {
> > + SharedInvalidationMessage msg;
> > +
> > + msg.sn.id = SHAREDINVALSNAPSHOT_ID;
> > + msg.sn.dbId = dbId;
> > + msg.sn.relId = relId;
> > +
> > + LogLogicalInvalidations(1, &msg, false);
> > + }
> >  }
> >
> > I am not sure why do we need to explicitly WAL log the snapshot
> > invalidation? because this is logged for invalidating the catalog
> > snapshot and for logical decoding we use HistoricSnapshot, not the
> > catalog snapshot.
> >
>
> I think it has been logged because without this patch as well we log
> all the invalidation messages at commit time and process them during
> decoding.  However, I agree that this particular invalidation message
> is not required for logical decoding for the reason you mentioned.  I
> think as we are explicitly logging invalidations, so it is better to
> avoid this if we can.

Ok
>
> Few other comments on this patch:
> 1.
> + case REORDER_BUFFER_CHANGE_INVALIDATION:
> +
> + /*
> + * Execute the invalidation message locally.
> + *
> + * XXX Do we need to care about relcacheInitFileInval and
> + * the other fields added to ReorderBufferChange, or just
> + * about the message itself?
> + */
> + LocalExecuteInvalidationMessage(&change->data.inval.msg);
> + break;
>
> Here, why are we executing messages individually?  Can't we just
> follow what we do in DecodeCommit which is to record the invalidations
> in ReorderBufferTXN as we encounter them and then allow them to
> execute on each REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID.  Is there a
> reason why we don't do ReorderBufferXidSetCatalogChanges when we
> receive any invalidation message?
IMHO, the reason is that in DecodeCommit, we get all the invalidation
at one time so, at REORDER_BUFFER_CHANGE_INTERNAL_COMMAND_ID, we don't
know which invalidation message to execute so for being safe we have
to execute all.  But, since we are logging all invalidation
individually, we exactly know at this stage which cache to invalidate.
So it is better to only invalidate required cache not all.

>
> 2.
> @@ -3025,8 +3073,8 @@ ReorderBufferRestoreChange(ReorderBuffer *rb,
> ReorderBufferTXN *txn,
>   * although we don't check the memory limit when restoring the changes in
>   * this branch (we only do that when initially queueing the changes after
>   * decoding), because we will release the changes later, and that will
> - * update the accounting too (subtracting the size from the counters).
> - * And we don't want to underflow there.
> + * update the accounting too (subtracting the size from the counters). And
> + * we don't want to underflow there.
>   */
>
> This seems like an unrelated change.
Indeed.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com