[BUGS] BUG #14687: pg_xlogdump does only count "main data" for record lengthand leading to incorrect statistics - Mailing list pgsql-bugs
| From | chjischj@163.com |
|---|---|
| Subject | [BUGS] BUG #14687: pg_xlogdump does only count "main data" for record lengthand leading to incorrect statistics |
| Date | |
| Msg-id | 20170603165939.1436.58887@wrigleys.postgresql.org Whole thread Raw |
| Responses |
Re: [BUGS] BUG #14687: pg_xlogdump does only count "main data" forrecord length and leading to incorrect statistics
(Andres Freund <andres@anarazel.de>)
Re: [BUGS] BUG #14687: pg_xlogdump does only count "main data" forrecord length and leading to incorrect statistics (Andres Freund <andres@anarazel.de>) |
| List | pgsql-bugs |
The following bug has been logged on the website:
Bug reference: 14687
Logged by: Chen huajun
Email address: chjischj@163.com
PostgreSQL version: 9.5.0
Operating system: ALL
Description:
Currently, record length in pg_xlogdump/pg_waldump's output only count
length of "main data",
This results in pg_xlogdump/pg_waldump can not accurately count the size of
the various types of WAL records.
for example
[postgres@node1 ~]$ pg_waldump --stats=record
pg10data/pg_wal/00000001000000000000001F Type N (%)
Record size (%) FPI size (%) Combined size
(%) ---- - ---
----------- --- -------- --- -------------
--- XLOG/FPI 939 ( 34.68)
22536 ( 27.15) 6897764 ( 99.75) 6920300 ( 98.89) Transaction/COMMIT
6 ( 0.22) 976 ( 1.18) 0 ( 0.00) 976 ( 0.01) Storage/CREATE
3 ( 0.11) 120 ( 0.14) 0 ( 0.00) 120
( 0.00) Standby/LOCK 6 ( 0.22) 240 ( 0.29) 0 (
0.00) 240 ( 0.00) Standby/INVALIDATIONS 3 ( 0.11) 264 (
0.32) 0 ( 0.00) 264 ( 0.00) Heap2/VISIBLE 1 (
0.04) 29 ( 0.03) 8192 ( 0.12) 8221 ( 0.12) Heap2/MULTI_INSERT
165 ( 6.09)
14606 ( 17.60) 0 ( 0.00) 14606 ( 0.21) Heap2/MULTI_INSERT+INIT
1493 ( 55.13)
41804 ( 50.37) 0 ( 0.00) 41804 ( 0.60) Heap/INSERT
29 ( 1.07) 783 ( 0.94) 0 ( 0.00) 783 ( 0.01) Heap/INPLACE
7 ( 0.26) 182 ( 0.22) 0 ( 0.00) 182
( 0.00) Btree/INSERT_LEAF 56 ( 2.07) 1456 ( 1.75) 8888 (
0.13) 10344 ( 0.15) --------
-------- -------- -------- Total 2708
82996 [1.19%] 6914844 [98.81%] 6997840 [100%]
Total Combined size above is only 6997840 byte,but should be 16MB.
So, I modified one line of code in pg_waldump.c
src/bin/pg_waldump/pg_waldump.c:390
rec_len = XLogRecGetDataLen(record) + SizeOfXLogRecord;
/* * Calculate the amount of FPI data in the record. * * XXX: We peek into xlogreader's
privatedecoded backup blocks for
the * bimg_len indicating the length of FPI data. It doesn't seem worth
it to * add an accessor macro for this. */ fpi_len = 0; for (block_id = 0; block_id <=
record->max_block_id;block_id++) { if (XLogRecHasBlockImage(record, block_id))
fpi_len += record->blocks[block_id].bimg_len; }
==> /* * Calculate the amount of FPI data in the record. * * XXX: We peek into
xlogreader'sprivate decoded backup blocks for
the * bimg_len indicating the length of FPI data. It doesn't seem worth
it to * add an accessor macro for this. */ fpi_len = 0; for (block_id = 0; block_id <=
record->max_block_id;block_id++) { if (XLogRecHasBlockImage(record, block_id))
fpi_len += record->blocks[block_id].bimg_len; } rec_len = XLogRecGetTotalLen(record) - fpi_len;
and got more precise output(correct FPI ratio is 41.37% instead of 98.81%)
[postgres@node1 ~]$ pg_waldump --stats=record
pg10data/pg_wal/00000001000000000000001F Type N (%)
Record size (%) FPI size (%) Combined size
(%) ---- - ---
----------- --- -------- --- -------------
--- XLOG/FPI 939 ( 34.68)
46011 ( 0.47) 6897764 ( 99.75) 6943775 ( 41.54) Transaction/COMMIT
6 ( 0.22) 988 ( 0.01) 0 ( 0.00) 988 ( 0.01) Storage/CREATE
3 ( 0.11) 126 ( 0.00) 0 ( 0.00) 126
( 0.00) Standby/LOCK 6 ( 0.22) 252 ( 0.00) 0 (
0.00) 252 ( 0.00) Standby/INVALIDATIONS 3 ( 0.11) 270 (
0.00) 0 ( 0.00) 270 ( 0.00) Heap2/VISIBLE 1 (
0.04) 64 ( 0.00) 8192 ( 0.12) 8256 ( 0.05) Heap2/MULTI_INSERT
165 ( 6.09)
547329 ( 5.59) 0 ( 0.00) 547329 (
3.27) Heap2/MULTI_INSERT+INIT 1493 ( 55.13)
9194033 ( 93.82) 0 ( 0.00) 9194033 (
55.01) Heap/INSERT 29 ( 1.07) 5655 ( 0.06) 0 (
0.00) 5655 ( 0.03) Heap/INPLACE 7 ( 0.26) 1493 (
0.02) 0 ( 0.00) 1493 ( 0.01) Btree/INSERT_LEAF 56 (
2.07) 3723 ( 0.04) 8888 ( 0.13) 12611 ( 0.08)
--------
-------- -------- -------- Total 2708
9799944 [58.63%] 6914844 [41.37%] 16714788 [100%]
--
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs
pgsql-bugs by date: