Resource cleanup callbacks for foreign data wrappers - Mailing list pgsql-hackers

From Mikhail Bautin
Subject Resource cleanup callbacks for foreign data wrappers
Date
Msg-id CA+7awWNPBd0vFqip0xeBtG+jXFjOGgdD2vcmb0udP-SP--34GQ@mail.gmail.com
Whole thread Raw
Responses Re: Resource cleanup callbacks for foreign data wrappers  (Haribabu Kommi <kommi.haribabu@gmail.com>)
List pgsql-hackers
Hello All,

We are using the foreign data wrapper API to integrate YugaByte DB (https://github.com/yugabyte/yugabyte-db) with PostgreSQL, so scanning tables stored in YugaByte DB is being done through Foreign Scan callbacks. These callbacks in turn read data from the underlying YugaByte DB cluster and maintain reference counts on RPC-related resources needed for that. Currently we are using PostgreSQL 10.4 codebase for this integration. I am trying to figure out what is the right way for foreign data wrappers to ensure resources used in a query are freed in all cases. It looks like the EndForeignScan callback is not being called in case an error happens during expression evaluation, such as during this query from int2.sql:

SELECT '' AS five, i.f1, i.f1 * int2 '2' AS x FROM INT2_TBL i;
ERROR:  smallint out of range

For the error-free case EndForeignScan appears on this codepath:

ExecEndForeignScan (postgres/src/backend/executor/nodeForeignscan.c:293)
ExecEndNode (postgres/src/backend/executor/execProcnode.c:649)
ExecEndPlan (postgres/src/backend/executor/execMain.c:1607)
standard_ExecutorEnd (postgres/src/backend/executor/execMain.c:495)
ExecutorEnd (postgres/src/backend/executor/execMain.c:466)
PortalCleanup (postgres/src/backend/commands/portalcmds.c:302)
PortalDrop (postgres/src/backend/utils/mmgr/portalmem.c:489)
exec_simple_query (postgres/src/backend/tcop/postgres.c:1118)
PostgresMain (postgres/src/backend/tcop/postgres.c:4126)
BackendRun (postgres/src/backend/postmaster/postmaster.c:4411)
BackendStartup (postgres/src/backend/postmaster/postmaster.c:4083)
ServerLoop (postgres/src/backend/postmaster/postmaster.c:1758)
PostmasterMain (postgres/src/backend/postmaster/postmaster.c:1366)
PostgresServerProcessMain (postgres/src/backend/main/main.c:228)

However, I see that ExecutorEnd is not being executed at all in PortalCleanup in portalcmds.c in case portal->status is PORTAL_FAILED. If this is by design, what would be the right callback mechanism for foreign data wrappers to ensure resources are freed?Would RegisterResourceReleaseCallback from resowner.h be more appropriate for this use case? I only see that being used for OpenSSL-related resources at the moment.

Thanks,
Mikhail

pgsql-hackers by date:

Previous
From: Krzysztof Nienartowicz
Date:
Subject: Re: Speeding up INSERTs and UPDATEs to partitioned tables
Next
From: Tomas Vondra
Date:
Subject: Re: shared-memory based stats collector