Re: Optimizing ResouceOwner to speed up COPY - Mailing list pgsql-hackers

From Tom Lane
Subject Re: Optimizing ResouceOwner to speed up COPY
Date
Msg-id 1569660.1760653040@sss.pgh.pa.us
Whole thread Raw
In response to Re: Optimizing ResouceOwner to speed up COPY  (Tomas Vondra <tomas@vondra.me>)
Responses Re: Optimizing ResouceOwner to speed up COPY
List pgsql-hackers
Tomas Vondra <tomas@vondra.me> writes:
> On 10/16/25 21:28, Tom Lane wrote:
>> I was thinking of adding some temporary instrumentation, like
>> just elog'ing whenever the count goes above 1, and seeing where
>> you get hits during the regression tests.  I'm prepared to believe
>> this is worth doing, but it'd be nice to have more examples
>> in mind.

> I tried that, and that gives me ~30k log messages with (count > 1). But
> that's a bit misleading, because a lot of that are the same "thing"
> going from 1 to N, which produces N messages.

> If I subtract all the COPY statements, loading data for regressison
> tests, that leaves ~7500 cases. There's a lot of cases with count 2 or
> 3, mostly simple queries. Even a simple "\d t" produces a bunch of such
> messages.

Okay, so we definitely do have other cases where the count will be
more than 1.

> There are some more extreme ones too. For example
>    select infinite_recurse();
> produces
> WARNING:  RESOURCEOWNER: plancache reference 0x34555828 resource owner
> Portal count 1340

Makes me wonder if we shouldn't make the count int64, just to remove
all worries about overflow.  That'd be free on 64-bit machines ...

> But I had another idea - see how large the ResourceOwners get, which
> would tell us how much "overhead" it really is. So I added logging into
> ResourceOwnerDelete (without the patch), and with that regression tests
> produce 113916 messages. And 113289 have the initial capacity 32, so
> array only. From the remaining ~600, only 72 have capacity over 64.

Ah, that's fairly convincing.  Seems like we can move ahead with this.

            regards, tom lane



pgsql-hackers by date:

Previous
From: Masahiko Sawada
Date:
Subject: Re: speedup COPY TO for partitioned table.
Next
From: Tomas Vondra
Date:
Subject: Re: Optimizing ResouceOwner to speed up COPY