Thread: Printing bitmap objects in the debugger

Hi All,
While working on partition-wise join, I had to examine Relids objects
many times. Printing the Bitmapset::words[] in binary format and then
inferring the relids takes time and is error prone. Instead I wrote a
function bms_to_char() which allocates a StringInfo, calls
outBitmapset() to decode Bitmapset as a set of integers and returns
the string. In order to examine a Relids object all one has to do is
execute 'p bms_to_char(bms_object) under gdb.

Is there a way, this can be included in the code? If it's available in
the code, developers don't have to apply the patch and compile it for
debugging.

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Ashutosh Bapat wrote:
> Hi All,
> While working on partition-wise join, I had to examine Relids objects
> many times. Printing the Bitmapset::words[] in binary format and then
> inferring the relids takes time and is error prone. Instead I wrote a
> function bms_to_char() which allocates a StringInfo, calls
> outBitmapset() to decode Bitmapset as a set of integers and returns
> the string. In order to examine a Relids object all one has to do is
> execute 'p bms_to_char(bms_object) under gdb.

I don't understand.  Why don't you just use "call pprint(the bitmapset)"
in the debugger?

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> writes:
> While working on partition-wise join, I had to examine Relids objects
> many times. Printing the Bitmapset::words[] in binary format and then
> inferring the relids takes time and is error prone.

FWIW, I generally rely on pprint() to look at planner data structures
from the debugger.
        regards, tom lane

Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> I don't understand.  Why don't you just use "call pprint(the bitmapset)"
> in the debugger?

Bitmapsets aren't Nodes, so pprint doesn't work directly on them.
I usually find that I can pprint some node containing the value(s)
I'm interested in, but maybe that isn't working for Ashutosh's
particular case.
        regards, tom lane

Tom Lane wrote:
> Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> writes:
> > While working on partition-wise join, I had to examine Relids objects
> > many times. Printing the Bitmapset::words[] in binary format and then
> > inferring the relids takes time and is error prone.
> 
> FWIW, I generally rely on pprint() to look at planner data structures
> from the debugger.

Also:
http://blog.pgaddict.com/posts/making-debugging-with-gdb-a-bit-easier
This may not address the issue directly, but it's probably very helpful.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 2016/09/15 0:04, Tom Lane wrote:
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>> I don't understand.  Why don't you just use "call pprint(the bitmapset)"
>> in the debugger?
> 
> Bitmapsets aren't Nodes, so pprint doesn't work directly on them.
> I usually find that I can pprint some node containing the value(s)
> I'm interested in, but maybe that isn't working for Ashutosh's
> particular case.

There are many loose (ie, not inside any Node) Relids variables within the
optimizer code.  Perhaps Ashutosh ended up needing to look at those a lot.

Thanks,
Amit

On Wed, Sep 14, 2016 at 5:31 PM, Pavan Deolasee
<pavan.deolasee@gmail.com> wrote:
>
> On Wed, Sep 14, 2016 at 3:46 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
> wrote:
>>
>>
>>
>>  lately I'm using LVM debugger (which probably does not have something
>> equivalent),
>
>
> And I was so clueless about lldb's powerful scripting interface. For
> example, you can write something like this in bms_utils.py:
>
> import lldb
>
> def print_bms_members (bms):
>     words = bms.GetChildMemberWithName("words")
>     nwords = int(bms.GetChildMemberWithName("nwords").GetValue())
>
>     ret = 'nwords = {0} bitmap: '.format(nwords,)
>     for i in range(0, nwords):
>         ret += hex(int(words.GetChildAtIndex(0, lldb.eNoDynamicValues,
> True).GetValue()))
>
>     return ret
>

Thanks a lot for digging into it.

> And then do this while attached to lldb debugger:
>
> Process 99659 stopped
> * thread #1: tid = 0x59ba69, 0x00000001090b012f
> postgres`bms_add_member(a=0x00007fe60a0351f8, x=10) + 15 at bitmapset.c:673,
> queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
>     frame #0: 0x00000001090b012f
> postgres`bms_add_member(a=0x00007fe60a0351f8, x=10) + 15 at bitmapset.c:673
>    670 int wordnum,
>    671 bitnum;
>    672
> -> 673 if (x < 0)
>    674 elog(ERROR, "negative bitmapset member not allowed");
>    675 if (a == NULL)
>    676 return bms_make_singleton(x);
> (lldb) script
> Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.
>>>> from bms_utils import *
>>>> bms = lldb.frame.FindVariable ("a")
>>>> print print_bms_members(bms)
> nwords = 1 bitmap: 0x200
>

I can get that kind of output by command
p *bms
p/x (or p/b) *bms->words@(bms->nwords) in gdb.

But I can certainly extend the script you wrote above to print more
meaningful output similar to outBitmapset(). But then this would be
specific to LLVM. GDB too seems to have a similar interface
https://sourceware.org/gdb/wiki/PythonGdbTutorial, so I can probably
use above script with some modifications with GDB as well. Python
script will be easier to maintain as compared to maintaining a patch
that needs to be applied and compiled.

Said that, I am not sure if every debugger supported on every platform
we support has these features. Or may be developers work on only those
platforms which have such powerful debuggers, so it's ok.

Every debugger usually has much easier way to call a function and
print its output, so having a function like the one I have in the
patch makes things easy for all the debuggers and may be developers
not familiar with python.

>
> The complete API reference is available here
> http://lldb.llvm.org/python_reference/index.html
>
> Looks like an interesting SoC project to write useful lldb/gdb scripts to
> print internal structures for ease of debugging :-)
>

+1, if we can include something like that in the repository so as to
avoid every developer maintaining a script of his/her own.

-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

On Thu, Sep 15, 2016 at 2:58 PM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:
> On Wed, Sep 14, 2016 at 5:31 PM, Pavan Deolasee
> <pavan.deolasee@gmail.com> wrote:
>> The complete API reference is available here
>> http://lldb.llvm.org/python_reference/index.html
>>
>> Looks like an interesting SoC project to write useful lldb/gdb scripts to
>> print internal structures for ease of debugging :-)
>>
>
> +1, if we can include something like that in the repository so as to
> avoid every developer maintaining a script of his/her own.

+1. Even if one finishes by doing manual modifications of some of
them, it is always good to have a central point of reference.
-- 
Michael

>> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>>> I don't understand.  Why don't you just use "call pprint(the bitmapset)"
>>> in the debugger?
>>
>> Bitmapsets aren't Nodes, so pprint doesn't work directly on them.
>> I usually find that I can pprint some node containing the value(s)
>> I'm interested in, but maybe that isn't working for Ashutosh's
>> particular case.

that's right.
>
> There are many loose (ie, not inside any Node) Relids variables within the
> optimizer code.  Perhaps Ashutosh ended up needing to look at those a lot.

that's right too.

In joinrels.c for example we are manipulating Relids so many times.
[ashutosh@ubuntu pg_head]grep bms_ src/backend/optimizer/path/joinrels.c | wc -l
69
There are many other instances of this in other optimizer and planner
files. There are other places where we manipulate Bitmapsets.

And not every Relids object computed is contained in a Node. So,
pprint() doesn't help much.

-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

On Wed, Sep 14, 2016 at 8:45 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Tom Lane wrote:
>> Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> writes:
>> > While working on partition-wise join, I had to examine Relids objects
>> > many times. Printing the Bitmapset::words[] in binary format and then
>> > inferring the relids takes time and is error prone.
>>
>> FWIW, I generally rely on pprint() to look at planner data structures
>> from the debugger.
>
> Also:
> http://blog.pgaddict.com/posts/making-debugging-with-gdb-a-bit-easier
> This may not address the issue directly, but it's probably very helpful.

Thanks for the reference. I think this is something similar to what
Pavan suggested in the mail thread.

-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

On Wed, Sep 14, 2016 at 8:01 AM, Pavan Deolasee
<pavan.deolasee@gmail.com> wrote:
> On Wed, Sep 14, 2016 at 3:46 PM, Pavan Deolasee <pavan.deolasee@gmail.com>
> wrote:
>>  lately I'm using LVM debugger (which probably does not have something
>> equivalent),
>
>
> And I was so clueless about lldb's powerful scripting interface. For
> example, you can write something like this in bms_utils.py:
>
> import lldb
>
> def print_bms_members (bms):
>     words = bms.GetChildMemberWithName("words")
>     nwords = int(bms.GetChildMemberWithName("nwords").GetValue())
>
>     ret = 'nwords = {0} bitmap: '.format(nwords,)
>     for i in range(0, nwords):
>         ret += hex(int(words.GetChildAtIndex(0, lldb.eNoDynamicValues,
> True).GetValue()))
>
>     return ret
>
> And then do this while attached to lldb debugger:
>
> Process 99659 stopped
> * thread #1: tid = 0x59ba69, 0x00000001090b012f
> postgres`bms_add_member(a=0x00007fe60a0351f8, x=10) + 15 at bitmapset.c:673,
> queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
>     frame #0: 0x00000001090b012f
> postgres`bms_add_member(a=0x00007fe60a0351f8, x=10) + 15 at bitmapset.c:673
>    670 int wordnum,
>    671 bitnum;
>    672
> -> 673 if (x < 0)
>    674 elog(ERROR, "negative bitmapset member not allowed");
>    675 if (a == NULL)
>    676 return bms_make_singleton(x);
> (lldb) script
> Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.
>>>> from bms_utils import *
>>>> bms = lldb.frame.FindVariable ("a")
>>>> print print_bms_members(bms)
> nwords = 1 bitmap: 0x200
>
>
> The complete API reference is available here
> http://lldb.llvm.org/python_reference/index.html
>
> Looks like an interesting SoC project to write useful lldb/gdb scripts to
> print internal structures for ease of debugging :-)

This seems like a very complicated mechanism of substituting for a
very simple patch.  Your LLDB script is about the same number of lines
as Ashutosh's patch and only works for people who use LLDB.  Plus,
once you write it, you've got to enter the Python interpreter to use
it and then run three more lines of code that aren't easy to remember.
In contrast, with Ashutosh's proposed patch, you just write:

p bms_to_char(bms_object)

...and you're done.  Now, I grant that his approach bloats the binary
and yours does not, but nobody complains about pprint() bloating the
binary.  If that's actually an issue people are really concerned about
then let's just reject this and Ashutosh can patch his local copy.  If
that's not a serious problem then let's take the patch.  If anything,
I think this discussion shows that LLDB macros are probably too much
of a pain to be seriously considered for everyday use, unless perhaps
you're the sort of person who plans to spend the day inside the Python
shell anyway.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> This seems like a very complicated mechanism of substituting for a
> very simple patch.

Well, if we're gonna do it, then let's just do it, but please let's
have a patch that doesn't look like somebody's temporary debugging kluge.

I'd suggest that this is parallel to nodeToString() and therefore
(a) should be placed beside it, (b) should be named like it,
bmsToString() perhaps, and (c) should look more like it internally.
        regards, tom lane

>
> I'd suggest that this is parallel to nodeToString() and therefore
> (a) should be placed beside it,

Done. Added it after nodeToString().

> (b) should be named like it,
> bmsToString() perhaps,

bmsToString() is fine. Used that name.

> and (c) should look more like it internally.
>

Done.

I have also added a declaration for this function in nodes.h after
definition of struct Bitmapset. WIthout this declaration compiler
gives warning "no previous declaration" of this function.

Tested it under the debugger

Breakpoint 1, make_join_rel (root=0x20cafb0, rel1=0x20e2998,
rel2=0x20dd2c0) at joinrels.c:664
(gdb) p bmsToString(rel1->relids)
$1 = 0x2102fd0 "(b 1 3)"
(gdb) p bmsToString(rel2->relids)
$2 = 0x2104bc0 "(b 4)"
(gdb) p bmsToString(joinrelids)
$3 = 0x2104fd8 "(b 1 3 4)"
(gdb) p bmsToString(joinrel->relids)
$4 = 0x2105998 "(b 1 3 4)"
(gdb) p bmsToString(joinrel->lateral_relids)
$5 = 0x2105db0 "(b)"
(gdb) p joinrel->lateral_relids
$6 = (Relids) 0x0

--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> writes:
>> I'd suggest that this is parallel to nodeToString() and therefore
>> (a) should be placed beside it,

> Done. Added it after nodeToString().

Pushed, thanks.
        regards, tom lane

Thanks a lot.

On Fri, Sep 16, 2016 at 7:07 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Ashutosh Bapat <ashutosh.bapat@enterprisedb.com> writes:
>>> I'd suggest that this is parallel to nodeToString() and therefore
>>> (a) should be placed beside it,
>
>> Done. Added it after nodeToString().
>
> Pushed, thanks.
>
>                         regards, tom lane



-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company