Re: [HACKERS] BUG: pg_stat_statements query normalization issueswith combined queries - Mailing list pgsql-hackers

From Fabien COELHO
Subject Re: [HACKERS] BUG: pg_stat_statements query normalization issueswith combined queries
Date
Msg-id alpine.DEB.2.20.1612231037030.3892@lancre
Whole thread Raw
In response to Re: [HACKERS] BUG: pg_stat_statements query normalization issueswith combined queries  (Fabien COELHO <coelho@cri.ensmp.fr>)
Responses Re: [HACKERS] BUG: pg_stat_statements query normalization issueswith combined queries
Re: [HACKERS] BUG: pg_stat_statements query normalization issues withcombined queries
List pgsql-hackers
> Yes. I'll try to put together a patch and submit it to the next CF.

Here it is. I'll add this to the next CF.



This patch fixes the handling of compound/combined queries by 
pg_stat_statements (i.e. several queries sent together, eg with psql: 
"SELECT 1 \; SELECT 2;").

This bug was found in passing while investigating a strange performance 
issue I had with compound statements.


Collect query location information in parser:

  * create a ParseNode for statements with location information fields
    (qlocation & qlength).

  * add these fields to all parsed statements (*Stmt), Query and PlannedStmt.

  * statements location is filled in:

    - the lexer keep track of the last ';' encountered.

    - the information is used by the parser to compute the query length,
      which depends on whether the query ended on ';' or on the input end.

    - the query location is propagated to Query and PlannedStmt when built.


Fix pg_stat_statement:

  * the information is used by pg_stat_statement so as to extract the relevant part
    of the query string, including some trimming.

  * pg_stat_statement validation is greatly extended so as to test options
    and exercise various cases, including compound statements.


note 1: two non-statements tags (use for alter table commands) have been 
moved so that all statements tags are contiguous and easy to check.

note 2: the impact on the lexer & parser is quite minimal with this 
approach, about 30 LOC, most of which in one function. The largest changes 
are in the node header to add location fields.

note 3: the query length excludes the final separator, so that 
;-terminated and end of input terminated queries show the same.

note 4: the added test suggests that when tracking "all", queries in SQL 
user functions are not tracked, this might be a bug.

-- 
Fabien.
-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Attachment

pgsql-hackers by date:

Previous
From: Pavel Stehule
Date:
Subject: Re: [HACKERS] [COMMITTERS] pgsql: Simplify LWLock tranche machinery by removing array_base/array_s
Next
From: Craig Ringer
Date:
Subject: Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)