Thread: Moving from pgFouine to pgBadger Issue with Total query duration metric

Moving from pgFouine to pgBadger Issue with Total query duration metric

From
Sreekanth Palluru
Date:
Hi ,
I am moving from pgFouine to pgBadger. When I compare the reports I see that there is major difference in time reported by metric Total query duration for given same postgres server log file.( excuse me for not sharing the postgres server logfile )
pgfouine reports around 2hrs where as pgBadger reports around 4hrs.

I am using below command options  to execute the report. I am mainly comparing pgBadger report with  queries.html report


Appreciate your help on this ? wanted to know why there so much difference in time ?




pgFouine

/home/postgres/fouine/pgfouine-1.2/pgfouine.php -logtype stderr -file $filename -top 30 \
-report /tmp/queries.html=overall,bytype,slowest,n-mosttime,n-mostfrequent,n-slowestaverage \
-report /tmp/hourly.html=overall,hourly \
-report /tmp/errors.html=overall,n-mostfrequenterrors \
-database $1




pgBadger

perl pgbadger  -t 30 --disable-session --disable-connection --disable-lock --disable-temporary --disable-checkpoint  --disable-autovacuum -o pgbadger_out_3.html -d <DBNAME>  /home/postgres/postgresql-2018-01-11_000000.log





pgFouine => Overall statistics

 

·  Overall statistics ^

  • Number of unique normalized queries: 208
  • Number of queries: 2,111
  • Total query duration: 2h36m45s
  • First query: 2018-01-11 00:01:06
  • Last query: 2018-01-11 23:20:07
  • Query peak: 6 queries/s at 2018-01-11 14:26:18
  • Number of errors: 70,043
  • Number of unique normalized errors: 19



pgBadger =>  Global Stats

 

Global Stats

  • 207 Number of unique normalized queries
  • 2,113 Number of queries
  • 4h41m48s Total query duration
  • 2018-01-11 00:01:06 First query
  • 2018-01-11 23:20:07 Last query
  • 6 queries/s at 2018-01-11 17:41:30 Query peak


--
Regards
Sreekanth

RE: Moving from pgFouine to pgBadger Issue with Total query duration metric

From
"Ahmed, Nawaz"
Date:

Hi,

 

It could be a case of pgFouine not picking up/reporting all the queries when it calculates those stats. I see all the metrics matching closely in either reports except for the total query duration and query peak times. pgFouine thinks that there has been a peak of 6 queries/second at 2018-01-11 14:26:18 while pgBadger thinks there has been a peak of 6 queries/second at 2018-01-11 17:41:30, which could be right but it may be a different set of queries.

 

Best Regards,

 

Nawaz Ahmed
Software Development Engineer

Fujitsu Australia Software Technology Pty Ltd

14 Rodborough Road, Frenchs Forest NSW 2086, Australia
T +61 2 9452 9027
Nawaz@fast.au.fujitsu.com
fastware.com.au


 

From: Sreekanth Palluru [mailto:sree4pg@gmail.com]
Sent: Friday, 12 January 2018 4:02 PM
To: PostgreSQL mailing lists <pgsql-general@postgresql.org>
Subject: Moving from pgFouine to pgBadger Issue with Total query duration metric

 

Hi ,

I am moving from pgFouine to pgBadger. When I compare the reports I see that there is major difference in time reported by metric Total query duration for given same postgres server log file.( excuse me for not sharing the postgres server logfile )

pgfouine reports around 2hrs where as pgBadger reports around 4hrs.

 

I am using below command options  to execute the report. I am mainly comparing pgBadger report with  queries.html report

 

Appreciate your help on this ? wanted to know why there so much difference in time ?

 

 

 

pgFouine

 

/home/postgres/fouine/pgfouine-1.2/pgfouine.php -logtype stderr -file $filename -top 30 \
-report /tmp/queries.html=overall,bytype,slowest,n-mosttime,n-mostfrequent,n-slowestaverage \
-report /tmp/hourly.html=overall,hourly \
-report /tmp/errors.html=overall,n-mostfrequenterrors \
-database $1

 

 

 

pgBadger

 

perl pgbadger  -t 30 --disable-session --disable-connection --disable-lock --disable-temporary --disable-checkpoint  --disable-autovacuum -o pgbadger_out_3.html -d <DBNAME>  /home/postgres/postgresql-2018-01-11_000000.log

 

 

 

 

 

pgFouine => Overall statistics

 

·  Overall statistics ^

  • Number of unique normalized queries: 208
  • Number of queries: 2,111
  • Total query duration: 2h36m45s
  • First query: 2018-01-11 00:01:06
  • Last query: 2018-01-11 23:20:07
  • Query peak: 6 queries/s at 2018-01-11 14:26:18
  • Number of errors: 70,043
  • Number of unique normalized errors: 19

 

 

pgBadger =>  Global Stats

 

Global Stats

  • Queries
  • Events
  • Vacuums
  • Temporary files
  • Sessions
  • Connections
  • 207 Number of unique normalized queries
  • 2,113 Number of queries
  • 4h41m48s Total query duration
  • 2018-01-11 00:01:06 First query
  • 2018-01-11 23:20:07 Last query
  • 6 queries/s at 2018-01-11 17:41:30 Query peak



--

Regards

Sreekanth

Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the document and all copies thereof.

Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.

If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe@fast.au.fujitsu.com

Attachment