Re: asynchronous execution - Mailing list pgsql-hackers

Hello,

I cannot respond until next Monday, so I move this to the next CF
by myself.

At Tue, 15 Nov 2016 20:25:13 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in
<20161115.202513.268072050.horiguchi.kyotaro@lab.ntt.co.jp>
> Hello, this is a maintenance post of reased patches.
> I added a change of ResourceOwnerData missed in 0005.
> 
> At Mon, 31 Oct 2016 10:39:12 +0900 (JST), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in
<20161031.103912.217430542.horiguchi.kyotaro@lab.ntt.co.jp>
> > This a PoC patch of asynchronous execution feature, based on a
> > executor infrastructure Robert proposed. These patches are
> > rebased on the current master.
> > 
> > 0001-robert-s-2nd-framework.patch
> > 
> >  Roberts executor async infrastructure. Async-driver nodes
> >  register its async-capable children and sync and data transfer
> >  are done out of band of ordinary ExecProcNode channel. So async
> >  execution no longer disturbs async-unaware node and slows them
> >  down.
> > 
> > 0002-Fix-some-bugs.patch
> > 
> >  Some fixes for 0001 to work. This is just to preserve the shape
> >  of 0001 patch.
> > 
> > 0003-Modify-async-execution-infrastructure.patch
> > 
> >  The original infrastructure doesn't work when multiple foreign
> >  tables is on the same connection. This makes it work.
> > 
> > 0004-Make-postgres_fdw-async-capable.patch
> > 
> >  Makes postgres_fdw to work asynchronously.
> > 
> > 0005-Use-resource-owner-to-prevent-wait-event-set-from-le.patch
> > 
> >  This addresses a problem pointed by Robers about 0001 patch,
> >  that WaitEventSet used for async execution can leak by errors.
> > 
> > 0006-Apply-unlikely-to-suggest-synchronous-route-of-ExecA.patch
> > 
> >  ExecAppend gets a bit slower by penalties of misprediction of
> >  branches. This fixes it by using unlikely() macro.
> > 
> > 0007-Add-instrumentation-to-async-execution.patch
> > 
> >  As the description above for 0001, async infrastructure conveys
> >  tuples outside ExecProcNode channel so EXPLAIN ANALYZE requires
> >  special treat to show sane results. This patch tries that.
> > 
> > 
> > A result of a performance measurement is in this message.
> > 
> > https://www.postgresql.org/message-id/20161025.182150.230901487.horiguchi.kyotaro@lab.ntt.co.jp
> > 
> > 
> > | t0  - SELECT sum(a) FROM <local single table>;
> > | pl  - SELECT sum(a) FROM <4 local children>;
> > | pf0 - SELECT sum(a) FROM <4 foreign children on single connection>;
> > | pf1 - SELECT sum(a) FROM <4 foreign children on dedicate connections>;
> > ...
> > | async
> > |   t0: 3885.84 ( 40.20)  0.86% faster (should be error but stable on my env..)
> > |   pl: 1617.20 (  3.51)  1.26% faster (ditto)
> > |  pf0: 6680.95 (478.72)  19.5% faster
> > |  pf1: 1886.87 ( 36.25)  77.1% faster

-- 
Kyotaro Horiguchi
NTT Open Source Software Center