Thread: fishing out LOG: duration lines from the logfile.
Hi,
I think its not going to a trivial task to take out only the LOG duration
lines from a PostgreSQL logfile. We need to extract the duration and
the actual statement. I think i will put a custom delimeters around the
statements for the time being so that the log parser can parse it
unambigiously. Is there any better method?
Regds
Mallah.
Example Log file:
LOG: duration: 3.725 ms statement: SELECT c.oid,
n.nspname,
c.relname
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
WHERE pg_catalog.pg_table_is_visible(c.oid)
AND c.relname ~ '^users$'
ORDER BY 2, 3;
other log messages
other log messages
other log messages
LOG: duration: 0.705 ms statement: SELECT relhasindex, relkind, relchecks, reltriggers, relhasrules
FROM pg_catalog.pg_class WHERE oid = '17411'
LOG: duration: 5.929 ms statement: SELECT a.attname,
pg_catalog.format_type(a.atttypid, a.atttypmod),
a.attnotnull, a.atthasdef, a.attnum
FROM pg_catalog.pg_attribute a
WHERE a.attrelid = '17411' AND a.attnum > 0 AND NOT a.attisdropped
ORDER BY a.attnum
LOG: duration: 0.799 ms statement: SELECT substring(d.adsrc for 128) FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = '17411' AND d.adnum = 1
LOG: duration: 0.965 ms statement: SELECT substring(d.adsrc for 128) FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = '17411' AND d.adnum = 31
LOG: duration: 0.998 ms statement: SELECT substring(d.adsrc for 128) FROM pg_catalog.pg_attrdef d
WHERE d.adrelid = '17411' AND d.adnum = 33
LOG: duration: 2.288 ms statement: SELECT c2.relname, i.indisprimary, i.indisunique, pg_catalog.pg_get_indexdef(i.indexrelid)
FROM pg_catalog.pg_class c, pg_catalog.pg_class c2, pg_catalog.pg_index i
WHERE c.oid = '17411' AND c.oid = i.indrelid AND i.indexrelid = c2.oid
ORDER BY i.indisprimary DESC, i.indisunique DESC, c2.relname
LOG: duration: 2.288 ms statement: SELECT c2.relname, i.indisprimary, i.indisunique, pg_catalog.pg_get_indexdef(i.indexrelid)
FROM pg_catalog.pg_class c, pg_catalog.pg_class c2, pg_catalog.pg_index i
WHERE c.oid = '17411' AND c.oid = i.indrelid AND i.indexrelid = c2.oid
ORDER BY i.indisprimary DESC, i.indisunique DESC, c2.relname :statement end (<-- custom delimit)
Rajesh Kumar Mallah wrote:
>
> Hi,
>
> I think its not going to a trivial task to take out only the LOG duration
> lines from a PostgreSQL logfile. We need to extract the duration and
> the actual statement. I think i will put a custom delimeters around the
> statements for the time being so that the log parser can parse it
> unambigiously. Is there any better method?
Seeing that I was involved in implementing this behavior, I felt I
should write a script to pull out this information to see how hard it
would be. Here is an awk script:
awk ' BEGIN {in_statement = "N"}
{
while (getline > 0)
{
if ($1 == "LOG:" && $2 == "duration:" && $5 == "statement:")
{
print $0;
in_statement = "Y";
}
else if (in_statement == "Y" && $0 ~ /^ /) # <-- tab
{
print $0;
}
else
in_statement = "N";
}
}' "$@"
I tested this with the log file you included and it seeme to work fine,
though the email had some line wraps I had to remove.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073