Re: Fdw batch insert error out when set batch_size > 65535 - Mailing list pgsql-hackers

From Tomas Vondra
Subject Re: Fdw batch insert error out when set batch_size > 65535
Date
Msg-id ec642523-bf1a-a4cd-013c-b789ff091849@enterprisedb.com
Whole thread Raw
In response to Re: Fdw batch insert error out when set batch_size > 65535  (Tomas Vondra <tomas.vondra@enterprisedb.com>)
Responses Re: Fdw batch insert error out when set batch_size > 65535
Re: Fdw batch insert error out when set batch_size > 65535
List pgsql-hackers

On 6/9/21 4:05 PM, Tomas Vondra wrote:
> On 6/9/21 3:28 PM, Tom Lane wrote:
>> Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
>>> Note that the problem here is [1] - we're creating a lot of slots
>>> referencing the same tuple descriptor, which inflates the duration.
>>> There's a fix in the other thread, which eliminates ~99% of the
>>> overhead. I plan to push that fix soon (a day or two).
>>
>> Oh, okay, as long as there's a plan to bring the time back down.
>>
> 
> Yeah. Sorry for not mentioning this in the original message about the
> new regression test.
> 

I've pushed a fix addressing the performance issue.

There's one caveat, though - for regular builds the slowdown is pretty
much eliminated. But with valgrind it's still considerably slower. For
postgres_fdw the "make check" used to take ~5 minutes for me, now it
takes >1h. And yes, this is entirely due to the new test case which is
generating / inserting 70k rows. So maybe the test case is not worth it
after all, and we should get rid of it.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: recovery test failures on hoverfly
Next
From: Tom Lane
Date:
Subject: Re: Fdw batch insert error out when set batch_size > 65535