Re: [WIP] Patches to enable extraction state of query execution from external session - Mailing list pgsql-hackers

From Maksim Milyutin
Subject Re: [WIP] Patches to enable extraction state of query execution from external session
Date
Msg-id 0c4706fe-34dc-49fd-ba0d-bc97bc14828a@postgrespro.ru
Whole thread Raw
In response to Re: [WIP] Patches to enable extraction state of query execution from external session  (Andres Freund <andres@anarazel.de>)
Responses Re: [WIP] Patches to enable extraction state of query execution from external session  (Andres Freund <andres@anarazel.de>)
List pgsql-hackers
> Hi,
>
> On 2016-08-29 18:22:56 +0300, maksim wrote:
>> Now I complete extension that provides facility to see the current state of
>> query execution working on external session in form of EXPLAIN ANALYZE
>> output. This extension works on 9.5 version, for 9.6 and later it doesn't
>> support detailed statistics for parallel nodes yet.
>
> Could you expand a bit on what you want this for exactly?

Max goal - to push my extension to postgres core. But now it's ready 
only for 9.5. Prerequisites of this extension are patches presented here.

>> 2. Patch that enables to interrupt the query executor
>> (executor_hooks.patch).
>> This patch enables to hang up hooks on executor function of each node
>> (ExecProcNode). I define hooks before any node execution and after
>> execution.
>> I use this patch to add possibility of query tracing by emitted rows from
>> any node. I interrupt query executor after any node delivers one or zero
>> rows to upper node. And after execution of specific number trace steps I can
>> get the query state of traceable backend which will be somewhat
>> deterministic. I use this possibility for regression tests of my extension.
>
> This will increase executor overhead.

In simple case we have checks on existence of hooks. We may suppose to 
use only not heavy processing on hooks under regular execution of query. 
In my case (query trace), I set up hooks under trace mode and throw off 
otherwise.
> I think we'll need to find a way> to hide this behind the existing if (instrument) branches.

And so can be. It doesn't matter for trace mode. But I think instrument 
branch is intended only for collecting statistics by nodes.

>> 3. Patch that enables to output runtime explain statistics
>> (runtime_explain.patch).
>> This patch extends the regular explain functionality. The problem is in the
>> point that regular explain call makes result output after query execution
>> performing InstrEndLoop on nodes where necessary. My patch introduces
>> specific flag *runtime* that indicates whether we explain running query and
>> does some insertions in source code dedicated to output the statistics of
>> running query.
>
> Unless I'm missing something this doesn't really expose a user of this
> functionality?

This patch is only for extension. As described in 
https://www.postgresql.org/message-id/CACACo5Sz7G0MFauC082iM=XX_hQ7qQ5ndR4JPo+H-O5vp6iCcQ@mail.gmail.com 
regular ExplainNode() prints only statistics after query execution and 
asserts under InstEndLoop(). My patch releases this problem and rewrite 
formulas for statistic parameters appropriate to running queries without 
affecting regular EXPLAIN outputs.


-- 
Maksim Milyutin
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company



pgsql-hackers by date:

Previous
From: Michael Paquier
Date:
Subject: Re: Missing checks when malloc returns NULL...
Next
From: Aleksander Alekseev
Date:
Subject: Re: Missing checks when malloc returns NULL...