Re: Foreign join pushdown vs EvalPlanQual - Mailing list pgsql-hackers
From | Kouhei Kaigai |
---|---|
Subject | Re: Foreign join pushdown vs EvalPlanQual |
Date | |
Msg-id | 9A28C8860F777E439AA12E8AEA7694F801162000@BPXM15GP.gisp.nec.co.jp Whole thread Raw |
In response to | Re: Foreign join pushdown vs EvalPlanQual (Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>) |
Responses |
Re: Foreign join pushdown vs EvalPlanQual
(Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>)
|
List | pgsql-hackers |
> On 2015/10/28 6:04, Robert Haas wrote: > > On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita > > <fujita.etsuro@lab.ntt.co.jp> wrote: > >> Sorry, my explanation was not correct. (Needed to take in caffeine.) What > >> I'm concerned about is the following: > >> > >> SELECT * FROM localtab JOIN (ft1 LEFT JOIN ft2 ON ft1.x = ft2.x) ON > >> localtab.id = ft1.id FOR UPDATE OF ft1 > >> > >> LockRows > >> -> Nested Loop > >> Join Filter: (localtab.id = ft1.id) > >> -> Seq Scan on localtab > >> -> Foreign Scan on <ft1, ft2> > >> Remote SQL: SELECT * FROM ft1 LEFT JOIN ft2 WHERE ft1.x = ft2.x > >> FOR UPDATE OF ft1 > >> > >> Assume that ft1 performs late row locking. > > > If the SQL includes "FOR UPDATE of ft1", then it clearly performs > > early row locking. I assume you meant to omit that. > > Right. Sorry for my mistake. > > >> If an EPQ recheck was invoked > >> due to a concurrent transaction on the remote server that changed only the > >> value x of the ft1 tuple previously retrieved, then we would have to > >> generate a fake ft1/ft2-join tuple with nulls for ft2. (Assume that the ft2 > >> tuple previously retrieved was not a null tuple.) However, I'm not sure how > >> we can do that in ForeignRecheck; we can't know for example, which one is > >> outer and which one is inner, without an alternative local join execution > >> plan. Maybe I'm missing something, though. > > > I would expect it to issue a new query like: SELECT * FROM ft1 LEFT > > JOIN ft2 WHERE ft1.x = ft2.x AND ft1.tid = $0 AND ft2.tid = $1. > > We assume here that ft1 uses late row locking, so I thought the above > SQL should include "FOR UPDATE of ft1". But I still don't think that > that is right; the SQL with "FOR UPDATE of ft1" wouldn't generate the > fake ft1/ft2-join tuple with nulls for ft2, as expected. The reason for > that is that the updated version of the ft1 tuple wouldn't satisfy the > ft1.tid = $0 condition in an EPQ recheck, because the ctid for the > updated version of the ft1 tuple has changed. (IIUC, I think that if we > use a TID scan for ft1, the SQL would generate the expected result, > because I think that the TID condition would be ignored in the EPQ > recheck, but I don't think it's guaranteed to use a TID scan for ft1.) > Maybe I'm missing something, though. > It looks to me, we should not use ctid system column to identify remote row when postgres_fdw tries to support late row locking. The documentation says: http://www.postgresql.org/docs/devel/static/fdw-callbacks.html#FDW-CALLBACKS-UPDATE UPDATE and DELETE operations are performed against rows previously fetched by the table-scanning functions. The FDW mayneed extra information, such as a row ID or the values of primary-key columns, to ensure that it can identify the exactrow to update or delete The "rowid" should not be changed once it is fetched from the remote side until it is actually updated, deleted or locked, for correct identification. If ctid is used for this purpose, it is safe only when remote row is locked when it is fetched - it is exactly early row locking behavior, isn't it? > > This should be significantly more efficient than fetching the base > > rows from each of two tables with two separate queries. > > Maybe I think we could fix the SQL, so I have to admit that, but I'm > just wondering (1) what would happen for the case when ft1 uses late row > rocking and ft2 uses early row rocking and (2) that would be still more > efficient than re-fetching only the base row from ft1. > It should be decision by FDW driver. It is not easy to estimate a certain FDW driver mixes up early and late locking policy within a same remote join query. Do you really want to support such a mysterious implementation? Or, do you expect all the FDW driver is enforced to return a joined tuple if remote join case? It is different from my idea; it shall be an extra optimization option if FDW can fetch a joined tuple at once, but not always. So, if FDW driver does not support this optimal behavior, your driver can fetch two base tables then run local alternative join (or something other). > What I thought to improve the efficiency in the secondary-plan approach > that I proposed was that if we could parallelize re-fetching foreign > rows in ExecLockRows and EvalPlanQualFetchRowMarks, we would be able to > improve the efficiency not only for the case when performing a join of > foreign tables remotely but for the case when performing the join locally. > Parallelism is not a magic bullet... Thanks, -- NEC Business Creation Division / PG-Strom Project KaiGai Kohei <kaigai@ak.jp.nec.com>
pgsql-hackers by date: