Re: NDirectFileRead and Write - Mailing list pgsql-hackers

2008/8/5 ITAGAKI Takahiro <itagaki.takahiro@oss.ntt.co.jp>:
> Here is a patch to user NDirectFileRead/Write counters to get I/O counts
> in BufFile module. We can see the counters when log_statement_stats is on.
>
> The information is different from trace_sort; trace_sort shows used blocks
> in external sort, and log_statement_stats shows how many I/Os are submitted
> during sorts.
>
> I wrote:
>> I'd like to use NDirectFileRead and NDirectFileWrite statistics counters
>> for counting reads and writes in BufFile. They are defined, but not used
>> now. BufFile is used for tuple sorting or materializing, so we could use
>> NDirectFileRead/Write to retrieve how many I/Os are done in temp tablespace.
>
> =# SET client_min_messages = log;
> =# SET trace_sort = on;
> =# SET log_statement_stats = on;
> =# EXPLAIN ANALYZE SELECT * FROM generate_series(1, 1000000) AS i ORDER BY i;
> LOG:  begin tuple sort: nkeys = 1, workMem = 1024, randomAccess = f
> LOG:  switching to external sort with 7 tapes: CPU 0.09s/0.26u sec elapsed 0.35 sec
> LOG:  performsort starting: CPU 0.48s/1.68u sec elapsed 2.20 sec
> LOG:  finished writing final run 1 to tape 0: CPU 0.48s/1.70u sec elapsed 2.21 sec
> LOG:  performsort done: CPU 0.48s/1.70u sec elapsed 2.21 sec
> LOG:  external sort ended, 2444 disk blocks used: CPU 0.79s/2.23u sec elapsed 3.06 sec
> LOG:  QUERY STATISTICS
> DETAIL:  ! system usage stats:
> !       3.078000 elapsed 2.234375 user 0.812500 system sec
> !       [3.328125 user 1.281250 sys total]
> ! buffer usage stats:
> !       Shared blocks:          0 read,          0 written, buffer hit rate = 0.00%
> !       Local  blocks:          0 read,          0 written, buffer hit rate = 0.00%
> !       Direct blocks:       5375 read,       5374 written
>                                                             QUERY PLAN
>
------------------------------------------------------------------------------------------------------------------------------------
>  Sort  (cost=62.33..64.83 rows=1000 width=4) (actual time=2221.485..2743.831 rows=1000000 loops=1)
>   Sort Key: i
>   Sort Method:  external sort  Disk: 19552kB
>   ->  Function Scan on generate_series i  (cost=0.00..12.50 rows=1000 width=4) (actual time=349.065..892.907
rows=1000000loops=1)
 
>  Total runtime: 3087.305 ms
> (5 rows)
>
> Regards,
> ---
> ITAGAKI Takahiro
> NTT Open Source Software Center
>
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
>

Though I'm not quite good around buffers, I reviewed this patch and
found that it seems strange about the declaring position of the two
variables. In ResetBufferUsage() of bufmgr.c it says:

void
ResetBufferUsage(void)
{BufferHitCount = 0;ReadBufferCount = 0;BufferFlushCount = 0;LocalBufferHitCount = 0;ReadLocalBufferCount =
0;LocalBufferFlushCount= 0;NDirectFileRead = 0;NDirectFileWrite = 0;
 
}

so I guess all of these variables should be defined together but
actually you put the two in buffile.h while the others in
buf_iinnternals.h. Is there clear reason for that?

Regards,


-- 
Hitoshi Harada