Thread: more detailed description of tup_returned and tup_fetched

more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:
Hi,

I worried the difference between "tup_returned" and "tup_fetched" in
pg_stat_database. I assumed that "tup_returned" means the number of tuples
that returned to clients. Of course, this is wrong.

So, why don't you describe in more detail? If my understanding is right, they
correspond to "seq_tup_read" and "idx_tup_fetch" in pg_stat_all_tables.

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

Attachment

Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/14 17:00, Masahiro Ikeda wrote:
> Hi,
> 
> I worried the difference between "tup_returned" and "tup_fetched" in
> pg_stat_database. I assumed that "tup_returned" means the number of tuples
> that returned to clients. Of course, this is wrong.

-       Number of rows returned by queries in this database
+       Number of live rows returned by sequential scans of queries in this database

-       Number of rows fetched by queries in this database
+       Number of live rows fetched by index scan of queries in this database

I found the following comments in pgstat.h. So maybe even these
new descriptions are incorrect?

  * Note: for a table, tuples_returned is the number of tuples successfully
  * fetched by heap_getnext, while tuples_fetched is the number of tuples
  * successfully fetched by heap_fetch under the control of bitmap indexscans.
  * For an index, tuples_returned is the number of index entries returned by
  * the index AM, while tuples_fetched is the number of tuples successfully
  * fetched by heap_fetch under the control of simple indexscans for this index.

Regards,


-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/05/17 15:32, Fujii Masao wrote:
> 
> 
> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>> Hi,
>>
>> I worried the difference between "tup_returned" and "tup_fetched" in
>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>> that returned to clients. Of course, this is wrong.
> 
> -       Number of rows returned by queries in this database
> +       Number of live rows returned by sequential scans of queries in this
> database
> 
> -       Number of rows fetched by queries in this database
> +       Number of live rows fetched by index scan of queries in this database
> 
> I found the following comments in pgstat.h. So maybe even these
> new descriptions are incorrect?
> 
>  * Note: for a table, tuples_returned is the number of tuples successfully
>  * fetched by heap_getnext, while tuples_fetched is the number of tuples
>  * successfully fetched by heap_fetch under the control of bitmap indexscans.
>  * For an index, tuples_returned is the number of index entries returned by
>  * the index AM, while tuples_fetched is the number of tuples successfully
>  * fetched by heap_fetch under the control of simple indexscans for this index.

Oh, Thanks!

I updated the sentences using the descriptions of
"pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
"pg_stat_all_index.idx_tup_read".

-       Number of rows returned by queries in this database
+       Number of rows returned by queries in this database. The rows
correspond to the live rows fetched by sequential scans and index entries
returned by scans on indexes

-       Number of rows fetched by queries in this database
+       Number of rows fetched by queries in this database. The rows
correspond to the live rows fetched by index scans

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/17 18:58, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/17 15:32, Fujii Masao wrote:
>>
>>
>> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>>> Hi,
>>>
>>> I worried the difference between "tup_returned" and "tup_fetched" in
>>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>>> that returned to clients. Of course, this is wrong.
>>
>> -       Number of rows returned by queries in this database
>> +       Number of live rows returned by sequential scans of queries in this
>> database
>>
>> -       Number of rows fetched by queries in this database
>> +       Number of live rows fetched by index scan of queries in this database
>>
>> I found the following comments in pgstat.h. So maybe even these
>> new descriptions are incorrect?
>>
>>   * Note: for a table, tuples_returned is the number of tuples successfully
>>   * fetched by heap_getnext, while tuples_fetched is the number of tuples
>>   * successfully fetched by heap_fetch under the control of bitmap indexscans.
>>   * For an index, tuples_returned is the number of index entries returned by
>>   * the index AM, while tuples_fetched is the number of tuples successfully
>>   * fetched by heap_fetch under the control of simple indexscans for this index.
> 
> Oh, Thanks!
> 
> I updated the sentences using the descriptions of
> "pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
> "pg_stat_all_index.idx_tup_read".
> 
> -       Number of rows returned by queries in this database
> +       Number of rows returned by queries in this database. The rows
> correspond to the live rows fetched by sequential scans and index entries
> returned by scans on indexes

This is still not correct because this counter is incremented even when
other scan like TidScan happens?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/05/17 20:46, Fujii Masao wrote:
> 
> 
> On 2021/05/17 18:58, Masahiro Ikeda wrote:
>>
>>
>> On 2021/05/17 15:32, Fujii Masao wrote:
>>>
>>>
>>> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>>>> Hi,
>>>>
>>>> I worried the difference between "tup_returned" and "tup_fetched" in
>>>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>>>> that returned to clients. Of course, this is wrong.
>>>
>>> -       Number of rows returned by queries in this database
>>> +       Number of live rows returned by sequential scans of queries in this
>>> database
>>>
>>> -       Number of rows fetched by queries in this database
>>> +       Number of live rows fetched by index scan of queries in this database
>>>
>>> I found the following comments in pgstat.h. So maybe even these
>>> new descriptions are incorrect?
>>>
>>>   * Note: for a table, tuples_returned is the number of tuples successfully
>>>   * fetched by heap_getnext, while tuples_fetched is the number of tuples
>>>   * successfully fetched by heap_fetch under the control of bitmap indexscans.
>>>   * For an index, tuples_returned is the number of index entries returned by
>>>   * the index AM, while tuples_fetched is the number of tuples successfully
>>>   * fetched by heap_fetch under the control of simple indexscans for this
>>> index.
>>
>> Oh, Thanks!
>>
>> I updated the sentences using the descriptions of
>> "pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
>> "pg_stat_all_index.idx_tup_read".
>>
>> -       Number of rows returned by queries in this database
>> +       Number of rows returned by queries in this database. The rows
>> correspond to the live rows fetched by sequential scans and index entries
>> returned by scans on indexes
> 
> This is still not correct because this counter is incremented even when
> other scan like TidScan happens?

Sorry, I couldn't find the way to increment tup_returned by TidScan.
Do you mean that Tid Range Scan increments the counter?

Tid Range Scan increments the tup_returned, and
pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
Tid Range Scan is like sequential scan. That's the reason why the document of
pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
sequential scans"

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/18 13:20, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/17 20:46, Fujii Masao wrote:
>>
>>
>> On 2021/05/17 18:58, Masahiro Ikeda wrote:
>>>
>>>
>>> On 2021/05/17 15:32, Fujii Masao wrote:
>>>>
>>>>
>>>> On 2021/05/14 17:00, Masahiro Ikeda wrote:
>>>>> Hi,
>>>>>
>>>>> I worried the difference between "tup_returned" and "tup_fetched" in
>>>>> pg_stat_database. I assumed that "tup_returned" means the number of tuples
>>>>> that returned to clients. Of course, this is wrong.
>>>>
>>>> -       Number of rows returned by queries in this database
>>>> +       Number of live rows returned by sequential scans of queries in this
>>>> database
>>>>
>>>> -       Number of rows fetched by queries in this database
>>>> +       Number of live rows fetched by index scan of queries in this database
>>>>
>>>> I found the following comments in pgstat.h. So maybe even these
>>>> new descriptions are incorrect?
>>>>
>>>>    * Note: for a table, tuples_returned is the number of tuples successfully
>>>>    * fetched by heap_getnext, while tuples_fetched is the number of tuples
>>>>    * successfully fetched by heap_fetch under the control of bitmap indexscans.
>>>>    * For an index, tuples_returned is the number of index entries returned by
>>>>    * the index AM, while tuples_fetched is the number of tuples successfully
>>>>    * fetched by heap_fetch under the control of simple indexscans for this
>>>> index.
>>>
>>> Oh, Thanks!
>>>
>>> I updated the sentences using the descriptions of
>>> "pg_stat_all_tables.seq_tup_read", "pg_stat_all_tables.idx_tup_fetch", and
>>> "pg_stat_all_index.idx_tup_read".
>>>
>>> -       Number of rows returned by queries in this database
>>> +       Number of rows returned by queries in this database. The rows
>>> correspond to the live rows fetched by sequential scans and index entries
>>> returned by scans on indexes
>>
>> This is still not correct because this counter is incremented even when
>> other scan like TidScan happens?
> 
> Sorry, I couldn't find the way to increment tup_returned by TidScan.
> Do you mean that Tid Range Scan increments the counter?

Yes, what I tried to mean is Tid Range Scan.

> 
> Tid Range Scan increments the tup_returned, and
> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
> Tid Range Scan is like sequential scan.

Yes, you're right. One interesting thing I found is;
when Tid Range Scan happens, seq_tup_read is incremented
but seq_scan is not. I'm not sure if this is expected behavior or not.

> That's the reason why the document of
> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
> sequential scans"

Regarding the original issue, as far as I understand correctly,

* pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) + sum(pg_stat_all_indexes.idx_tup_read)
* pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)

But the counters for some system catalogs like pg_database shared
across all databases of a cluster are excluded from that calculation.
Is this my understanding right? If right, probably we can reuse
the existing descriptions for those counters to document
pg_stat_database counters. For example,

pg_stat_database.tup_returned:
Number of live rows fetched by sequential and index scans in this database

pg_stat_database.tup_fetched:
Number of index entries returned by scans on indexes in this database

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/05/18 16:01, Fujii Masao wrote:
> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>> Tid Range Scan increments the tup_returned, and
>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
>> Tid Range Scan is like sequential scan.
> 
> Yes, you're right. One interesting thing I found is;
> when Tid Range Scan happens, seq_tup_read is incremented
> but seq_scan is not. I'm not sure if this is expected behavior or not.

The following comment says that this behavior is expected. But, I agree it's
odd and it's natural both seq_tup_read and seq_scan are incremented at the
same time or not...

/*
 * Currently, we only have a stats counter for sequential heap scans (but
 * e.g for bitmap scans the underlying bitmap index scans will be counted,
 * and for sample scans we update stats for tuple fetches).
 */
if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
    pgstat_count_heap_scan(scan->rs_base.rs_rd);


>> That's the reason why the document of
>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>> sequential scans"
> 
> Regarding the original issue, as far as I understand correctly,
> 
> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
> sum(pg_stat_all_indexes.idx_tup_read)
> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
> 
> But the counters for some system catalogs like pg_database shared
> across all databases of a cluster are excluded from that calculation.
> Is this my understanding right? If right, probably we can reuse
> the existing descriptions for those counters to document
> pg_stat_database counters. For example,

Yes, my understanding is same now.


> pg_stat_database.tup_returned:> Number of live rows fetched by sequential and index scans in this database

I wonder "live rows fetched by index scans" may mislead. I think "live" means
it's not dead tuple and "rows" mean the tuple user want to get.

But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
scans on this index". There is no meaning of "live" and "rows", so I thought
it's better to distinguish them.

So, why don't you change to "Number of live rows fetched by sequential scans
and index entries returned by index scans in this database"?


> pg_stat_database.tup_fetched:
> Number of index entries returned by scans on indexes in this database
Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
pg_stat_database.tup_returned.

"Number of live rows fetched by index scans in this database" seems to be correct.


Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/18 18:23, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/18 16:01, Fujii Masao wrote:
>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>> Tid Range Scan increments the tup_returned, and
>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok because
>>> Tid Range Scan is like sequential scan.
>>
>> Yes, you're right. One interesting thing I found is;
>> when Tid Range Scan happens, seq_tup_read is incremented
>> but seq_scan is not. I'm not sure if this is expected behavior or not.
> 
> The following comment says that this behavior is expected. But, I agree it's
> odd and it's natural both seq_tup_read and seq_scan are incremented at the
> same time or not...
> 
> /*
>   * Currently, we only have a stats counter for sequential heap scans (but
>   * e.g for bitmap scans the underlying bitmap index scans will be counted,
>   * and for sample scans we update stats for tuple fetches).
>   */
> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
>     pgstat_count_heap_scan(scan->rs_base.rs_rd);
> 
> 
>>> That's the reason why the document of
>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>> sequential scans"
>>
>> Regarding the original issue, as far as I understand correctly,
>>
>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>> sum(pg_stat_all_indexes.idx_tup_read)
>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>
>> But the counters for some system catalogs like pg_database shared
>> across all databases of a cluster are excluded from that calculation.
>> Is this my understanding right? If right, probably we can reuse
>> the existing descriptions for those counters to document
>> pg_stat_database counters. For example,
> 
> Yes, my understanding is same now.
> 
> 
>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential and index scans in this database
> 
> I wonder "live rows fetched by index scans" may mislead. I think "live" means
> it's not dead tuple and "rows" mean the tuple user want to get.
> 
> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
> scans on this index". There is no meaning of "live" and "rows", so I thought
> it's better to distinguish them.
> 
> So, why don't you change to "Number of live rows fetched by sequential scans
> and index entries returned by index scans in this database"?

Yes, LGTM.


>> pg_stat_database.tup_fetched:
>> Number of index entries returned by scans on indexes in this database
> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
> pg_stat_database.tup_returned.

I was thinking that pg_stat_database.tup_fetched is the same as
the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
is not.

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/05/18 20:10, Fujii Masao wrote:
> 
> 
> On 2021/05/18 18:23, Masahiro Ikeda wrote:
>>
>>
>> On 2021/05/18 16:01, Fujii Masao wrote:
>>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>>> Tid Range Scan increments the tup_returned, and
>>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok
>>>> because
>>>> Tid Range Scan is like sequential scan.
>>>
>>> Yes, you're right. One interesting thing I found is;
>>> when Tid Range Scan happens, seq_tup_read is incremented
>>> but seq_scan is not. I'm not sure if this is expected behavior or not.
>>
>> The following comment says that this behavior is expected. But, I agree it's
>> odd and it's natural both seq_tup_read and seq_scan are incremented at the
>> same time or not...
>>
>> /*
>>   * Currently, we only have a stats counter for sequential heap scans (but
>>   * e.g for bitmap scans the underlying bitmap index scans will be counted,
>>   * and for sample scans we update stats for tuple fetches).
>>   */
>> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
>>     pgstat_count_heap_scan(scan->rs_base.rs_rd);
>>
>>
>>>> That's the reason why the document of
>>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>>> sequential scans"
>>>
>>> Regarding the original issue, as far as I understand correctly,
>>>
>>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>>> sum(pg_stat_all_indexes.idx_tup_read)
>>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>>
>>> But the counters for some system catalogs like pg_database shared
>>> across all databases of a cluster are excluded from that calculation.
>>> Is this my understanding right? If right, probably we can reuse
>>> the existing descriptions for those counters to document
>>> pg_stat_database counters. For example,
>>
>> Yes, my understanding is same now.
>>
>>
>>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential
>>> and index scans in this database
>>
>> I wonder "live rows fetched by index scans" may mislead. I think "live" means
>> it's not dead tuple and "rows" mean the tuple user want to get.
>>
>> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
>> scans on this index". There is no meaning of "live" and "rows", so I thought
>> it's better to distinguish them.
>>
>> So, why don't you change to "Number of live rows fetched by sequential scans
>> and index entries returned by index scans in this database"?
> 
> Yes, LGTM.
> 
> 
>>> pg_stat_database.tup_fetched:
>>> Number of index entries returned by scans on indexes in this database
>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>> pg_stat_database.tup_returned.
> 
> I was thinking that pg_stat_database.tup_fetched is the same as
> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
> is not.

Yes. So, "Number of index entries returned by scans on indexes in this
database" is incorrect, and "Number of live rows fetched by index scans in
this database" is correct?

Regards,

-- 
Masahiro Ikeda
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/20 9:46, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/18 20:10, Fujii Masao wrote:
>>
>>
>> On 2021/05/18 18:23, Masahiro Ikeda wrote:
>>>
>>>
>>> On 2021/05/18 16:01, Fujii Masao wrote:
>>>> On 2021/05/18 13:20, Masahiro Ikeda wrote:
>>>>> Tid Range Scan increments the tup_returned, and
>>>>> pg_stat_all_tables.seq_tup_read is also incremented. I thought it's ok
>>>>> because
>>>>> Tid Range Scan is like sequential scan.
>>>>
>>>> Yes, you're right. One interesting thing I found is;
>>>> when Tid Range Scan happens, seq_tup_read is incremented
>>>> but seq_scan is not. I'm not sure if this is expected behavior or not.
>>>
>>> The following comment says that this behavior is expected. But, I agree it's
>>> odd and it's natural both seq_tup_read and seq_scan are incremented at the
>>> same time or not...
>>>
>>> /*
>>>    * Currently, we only have a stats counter for sequential heap scans (but
>>>    * e.g for bitmap scans the underlying bitmap index scans will be counted,
>>>    * and for sample scans we update stats for tuple fetches).
>>>    */
>>> if (scan->rs_base.rs_flags & SO_TYPE_SEQSCAN)
>>>      pgstat_count_heap_scan(scan->rs_base.rs_rd);
>>>
>>>
>>>>> That's the reason why the document of
>>>>> pg_stat_all_tables.seq_tup_read says "Number of live rows fetched by
>>>>> sequential scans"
>>>>
>>>> Regarding the original issue, as far as I understand correctly,
>>>>
>>>> * pg_stat_database.tup_returned = sum(pg_stat_all_tables.seq_tup_read) +
>>>> sum(pg_stat_all_indexes.idx_tup_read)
>>>> * pg_stat_database.tup_fetched = sum(pg_stat_all_tables.idx_tup_fetch)
>>>>
>>>> But the counters for some system catalogs like pg_database shared
>>>> across all databases of a cluster are excluded from that calculation.
>>>> Is this my understanding right? If right, probably we can reuse
>>>> the existing descriptions for those counters to document
>>>> pg_stat_database counters. For example,
>>>
>>> Yes, my understanding is same now.
>>>
>>>
>>>> pg_stat_database.tup_returned:> Number of live rows fetched by sequential
>>>> and index scans in this database
>>>
>>> I wonder "live rows fetched by index scans" may mislead. I think "live" means
>>> it's not dead tuple and "rows" mean the tuple user want to get.
>>>
>>> But, pg_stat_all_indexes.idx_tup_read says that "index entires returned by
>>> scans on this index". There is no meaning of "live" and "rows", so I thought
>>> it's better to distinguish them.
>>>
>>> So, why don't you change to "Number of live rows fetched by sequential scans
>>> and index entries returned by index scans in this database"?
>>
>> Yes, LGTM.
>>
>>
>>>> pg_stat_database.tup_fetched:
>>>> Number of index entries returned by scans on indexes in this database
>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>> pg_stat_database.tup_returned.
>>
>> I was thinking that pg_stat_database.tup_fetched is the same as
>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>> is not.
> 
> Yes. So, "Number of index entries returned by scans on indexes in this
> database" is incorrect, and "Number of live rows fetched by index scans in
> this database" is correct?

Yes, I think so!

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/05/20 17:00, Fujii Masao wrote:
> On 2021/05/20 9:46, Masahiro Ikeda wrote:
>> On 2021/05/18 20:10, Fujii Masao wrote:
>>>>> pg_stat_database.tup_fetched:
>>>>> Number of index entries returned by scans on indexes in this database
>>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>>> pg_stat_database.tup_returned.
>>>
>>> I was thinking that pg_stat_database.tup_fetched is the same as
>>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>>> is not.
>>
>> Yes. So, "Number of index entries returned by scans on indexes in this
>> database" is incorrect, and "Number of live rows fetched by index scans in
>> this database" is correct?
> 
> Yes, I think so!

Thanks!
I updated the patch for summarizing this thread.

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION

Attachment

Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/20 17:38, Masahiro Ikeda wrote:
> 
> 
> On 2021/05/20 17:00, Fujii Masao wrote:
>> On 2021/05/20 9:46, Masahiro Ikeda wrote:
>>> On 2021/05/18 20:10, Fujii Masao wrote:
>>>>>> pg_stat_database.tup_fetched:
>>>>>> Number of index entries returned by scans on indexes in this database
>>>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>>>> pg_stat_database.tup_returned.
>>>>
>>>> I was thinking that pg_stat_database.tup_fetched is the same as
>>>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>>>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>>>> is not.
>>>
>>> Yes. So, "Number of index entries returned by scans on indexes in this
>>> database" is incorrect, and "Number of live rows fetched by index scans in
>>> this database" is correct?
>>
>> Yes, I think so!
> 
> Thanks!
> I updated the patch for summarizing this thread.

Thanks for updating the patch! LGTM.

This is an improvement of documentation, so this should be applied in
v15 dev cycle? If so, could you add the patch to the next CF? Or you think
this is a bug fix and needs to be back-patched?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/05/21 22:26, Fujii Masao wrote:
> 
> 
> On 2021/05/20 17:38, Masahiro Ikeda wrote:
>>
>>
>> On 2021/05/20 17:00, Fujii Masao wrote:
>>> On 2021/05/20 9:46, Masahiro Ikeda wrote:
>>>> On 2021/05/18 20:10, Fujii Masao wrote:
>>>>>>> pg_stat_database.tup_fetched:
>>>>>>> Number of index entries returned by scans on indexes in this database
>>>>>> Is this the sum of pg_stat_all_indexes.idx_tup_read? This is accounted to
>>>>>> pg_stat_database.tup_returned.
>>>>>
>>>>> I was thinking that pg_stat_database.tup_fetched is the same as
>>>>> the sum of pg_stat_all_tables.idx_tup_fetch. Because they both
>>>>> are incremented by bitmap index scans, but pg_stat_all_indexes.idx_tup_read
>>>>> is not.
>>>>
>>>> Yes. So, "Number of index entries returned by scans on indexes in this
>>>> database" is incorrect, and "Number of live rows fetched by index scans in
>>>> this database" is correct?
>>>
>>> Yes, I think so!
>>
>> Thanks!
>> I updated the patch for summarizing this thread.
> 
> Thanks for updating the patch! LGTM.
> 
> This is an improvement of documentation, so this should be applied in
> v15 dev cycle? If so, could you add the patch to the next CF? Or you think
> this is a bug fix and needs to be back-patched?

Thanks for checking the patch!
I thought this patch will be applied in V15 dev cycle.

OK. I added the patch to the next CF.
https://commitfest.postgresql.org/33/3130/

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Fujii Masao
Date:

On 2021/05/24 11:37, Masahiro Ikeda wrote:
> Thanks for checking the patch!
> I thought this patch will be applied in V15 dev cycle.
> 
> OK. I added the patch to the next CF.
> https://commitfest.postgresql.org/33/3130/

I pushed the patch to 15dev. Thanks!

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Re: more detailed description of tup_returned and tup_fetched

From
Masahiro Ikeda
Date:

On 2021/06/30 20:59, Fujii Masao wrote:
> 
> 
> On 2021/05/24 11:37, Masahiro Ikeda wrote:
>> Thanks for checking the patch!
>> I thought this patch will be applied in V15 dev cycle.
>>
>> OK. I added the patch to the next CF.
>> https://commitfest.postgresql.org/33/3130/
> 
> I pushed the patch to 15dev. Thanks!

Thanks!

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION