Re: four minor proposals for 9.5 - Mailing list pgsql-hackers

From Gregory Smith
Subject Re: four minor proposals for 9.5
Date
Msg-id 5344250A.6090103@gmail.com
Whole thread Raw
In response to Re: four minor proposals for 9.5  (Pavel Stehule <pavel.stehule@gmail.com>)
Responses Re: four minor proposals for 9.5  (Pavel Stehule <pavel.stehule@gmail.com>)
Re: four minor proposals for 9.5  (Robert Haas <robertmhaas@gmail.com>)
List pgsql-hackers
On 4/6/14 2:46 PM, Pavel Stehule wrote:
>
> Proposed options are interesting for "enterprise" using, when you have 
> a some more smart tools for log entry processing, and when you need a 
> complex view about performance of billions queries - when cancel time 
> and lock time is important piece in mosaic of server' fitness.

I once sent a design proposal over for something I called "Performance 
Events" that included this.  It will be difficult to get everything 
people want to track into log_line_prefix macro form.  You're right that 
this particular one can probably be pushed into there, but you're adding 
four macros just for this feature. And that's only a fraction of what 
people expect from database per-query performance metrics.

The problem I got stuck on with the performance event project was where 
to store the data collected.  If you want to keep up with read rates, 
you can't use the existing log infrastructure.  It has to be something 
faster, lighter.  I wanted to push the data into shared memory somewhere 
instead.  Then some sort of logging consumer could drain that queue and 
persist it to disk.

Since then, we've had a number of advances, particularly these two:

-Dynamic shared memory allocation.
-Query log data from pg_stat_statements can persist.

With those important fundamentals available, I'm wandering around right 
now trying to get development resources to pick the whole event logging 
idea up again.  The hardest parts of the infrastructure I was stuck on 
in the past are in the code today.

-- 
Greg Smith greg.smith@crunchydatasolutions.com
Chief PostgreSQL Evangelist - http://crunchydatasolutions.com/



pgsql-hackers by date:

Previous
From: Greg Stark
Date:
Subject: Re: Problem with displaying "wide" tables in psql
Next
From: Pavel Stehule
Date:
Subject: Re: four minor proposals for 9.5