Re: [HACKERS] [POC] Faster processing at Gather node - Mailing list pgsql-hackers

From Andres Freund
Subject Re: [HACKERS] [POC] Faster processing at Gather node
Date
Msg-id 20171018053341.53f5kfxeq7sqytjc@alap3.anarazel.de
Whole thread Raw
In response to Re: [HACKERS] [POC] Faster processing at Gather node  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
Hi,

On 2017-10-17 14:39:57 -0700, Andres Freund wrote:
> I've spent some time looking into this, and I'm not quite convinced this
> is the right approach.  Let me start by describing where I see the
> current performance problems around gather stemming from.

One further approach to several of these issues could also be to change
things a bit more radically:

Instead of the current shm_mq + tqueue.c, have a drastically simpler
queue, that just stores fixed width dsa_pointers. Dealing with that
queue will be quite a bit faster. In that queue one would store dsa.c
managed pointers to tuples.

One thing that makes that attractive is that that'd move a bunch of
copying in the leader process solely to the worker processes, because
the leader could just convert the dsa_pointer into a local pointer and
hand that upwards the execution tree.

We'd possibly need some halfway clever way to reuse dsa allocations, but
that doesn't seem impossible.

Greetings,

Andres Freund


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

pgsql-hackers by date:

Previous
From: Dilip Kumar
Date:
Subject: Re: [HACKERS] Partition-wise aggregation/grouping
Next
From: Masahiko Sawada
Date:
Subject: Re: [HACKERS] pgbench: Skipping the creating primary keys after initialization