Thread: It seems no Windows buildfarm members are running find_typedefs
The current typedefs list seems to be lacking any Windows-only typedefs.
Noticed while trying to pgindent postmaster.c.
regards, tom lane
On 04/01/2014 08:53 PM, Tom Lane wrote: > The current typedefs list seems to be lacking any Windows-only typedefs. > Noticed while trying to pgindent postmaster.c. > > Hmm. odd. will check. cheers andrew
On 04/01/2014 09:22 PM, Andrew Dunstan wrote: > > On 04/01/2014 08:53 PM, Tom Lane wrote: >> The current typedefs list seems to be lacking any Windows-only typedefs. >> Noticed while trying to pgindent postmaster.c. >> >> > > > > Hmm. odd. will check. > > It's apparently causing the buildfarm to crash, which is why I must have disabled it. I'll chase that down tomorrow. cheers andrew
On 04/02/2014 12:25 AM, Andrew Dunstan wrote: > > On 04/01/2014 09:22 PM, Andrew Dunstan wrote: >> >> On 04/01/2014 08:53 PM, Tom Lane wrote: >>> The current typedefs list seems to be lacking any Windows-only >>> typedefs. >>> Noticed while trying to pgindent postmaster.c. >>> >>> >> >> >> >> Hmm. odd. will check. >> >> > > It's apparently causing the buildfarm to crash, which is why I must > have disabled it. I'll chase that down tomorrow. OK, we're back: <http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=frogmouth&dt=2014-04-02%2016%3A08%3A12&stg=typedefs> BTW, three animals are currently trying to contribute typedefs but aren't in fact contributing anything: okapi, dromedary and prairiedog. See <http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1> I can't really help much on these as my Gentoo facilities are non-existent, and my OSX facilities are not much better. I do recall trying to find a way to get typedefs on OSX a few years ago, without success. cheers andrew > > cheers > > andrew > > > >
Andrew Dunstan <andrew@dunslane.net> writes: > BTW, three animals are currently trying to contribute typedefs but > aren't in fact contributing anything: okapi, dromedary and prairiedog. > See <http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1> Man, that's a short list. I wonder if we need to encourage more people to do this. > I can't really help much on these as my Gentoo facilities are > non-existent, and my OSX facilities are not much better. I do recall > trying to find a way to get typedefs on OSX a few years ago, without > success. I poked around a bit, and so far as I can tell, OS X does not store debug symbol tables in executables. It looks like gdb goes to the .o files when it wants debug info. What's in the .o files is good ol' DWARF (at least in reasonably recent OS X releases), so it's not any harder to pull out the typedef names than it is on Linux. The problem is that you gotta iterate over all the .o files in the build tree rather than the installed executables. I looked at fixing find_typedefs but it seems like it's pretty fixated on the latter approach; any thoughts on how to revise it? regards, tom lane
On 04/02/2014 08:43 PM, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> BTW, three animals are currently trying to contribute typedefs but >> aren't in fact contributing anything: okapi, dromedary and prairiedog. >> See <http://www.pgbuildfarm.org/cgi-bin/typedefs.pl?show_list=1> > Man, that's a short list. I wonder if we need to encourage more people > to do this. > >> I can't really help much on these as my Gentoo facilities are >> non-existent, and my OSX facilities are not much better. I do recall >> trying to find a way to get typedefs on OSX a few years ago, without >> success. > I poked around a bit, and so far as I can tell, OS X does not store debug > symbol tables in executables. It looks like gdb goes to the .o files when > it wants debug info. What's in the .o files is good ol' DWARF (at least > in reasonably recent OS X releases), so it's not any harder to pull out > the typedef names than it is on Linux. The problem is that you gotta > iterate over all the .o files in the build tree rather than the installed > executables. I looked at fixing find_typedefs but it seems like it's > pretty fixated on the latter approach; any thoughts on how to revise it? > > Well, the reason it's that way is that that's the way it was done before the buildfarm took over the task. But it's not holy writ. Doing something else would be a SMOC. Essentially, I think the part that would need to change is this: foreach my $bin ( glob("$installdir/bin/*"), glob("$installdir/lib/*"), glob("$installdir/lib/postgresql/*") ) For OSX we'd construct the list via File::Find to recurse through the directories. So, something like this: my $using_osx = [some test for OSX]; my @testfiles; my $obj_wanted = sub { /^.*\.o\z/s && push(@testfiles,$File::Find::name); }; if ($using_osx) { File::Find::find($obj_wanted,$pgsql); } else { @testfiles = ( glob("$installdir/bin/*"), glob("$installdir/lib/*"), glob("$installdir/lib/postgresql/*"); } foreach my $bin (@testfiles) cheers andrew
On 2 Apr 2014, at 5:43 PM, Tom Lane wrote: > I poked around a bit, and so far as I can tell, OS X does not store debug > symbol tables in executables. It looks like gdb goes to the .o files when > it wants debug info. What's in the .o files is good ol' DWARF (at least > in reasonably recent OS X releases), so it's not any harder to pull out > the typedef names than it is on Linux. The problem is that you gotta > iterate over all the .o files in the build tree rather than the installed > executables. I looked at fixing find_typedefs but it seems like it's > pretty fixated on the latter approach; any thoughts on how to revise it? The Apple development tools gather the debug information during the final link stage (the one that produces the executableor shared object) using "dsymutil", which simply iterates over all of the .o files and links the debug info intoa separate object, foo.dSYM. Apple's gdb and lldb then find the relevant .dSYM file using a per-build UUID embedded inthe executable/library/debug symbol file. The dSYM file is a normal object file which has the DWARF sections but not the usual text/data sections, so if it can begenerated/found, it should be possible to dump the DWARF data from it and look for typedefs that way. (I'm pretty sure that if you were to run dsymutil and then merge the resulting object file's sections into the executable/shlib,you'd get a perfectly functional and debuggable result without having to look for or cart around the extradSYM file--- I haven't tried this though.)
Wim Lewis <wiml@omnigroup.com> writes:
> On 2 Apr 2014, at 5:43 PM, Tom Lane wrote:
>> I poked around a bit, and so far as I can tell, OS X does not store debug
>> symbol tables in executables.
> The Apple development tools gather the debug information during the final link stage (the one that produces the
executableor shared object) using "dsymutil", which simply iterates over all of the .o files and links the debug info
intoa separate object, foo.dSYM. Apple's gdb and lldb then find the relevant .dSYM file using a per-build UUID embedded
inthe executable/library/debug symbol file.
Ah. I've forgotten the details, but I'm pretty sure that we have
deliberately arranged our build process so that the .dSYM files don't get
built during link steps. Debugging seems to work all right anyway, at
least if the build tree is available, so I think Apple's gdb is able to
work from the symbol tables in the .o files.
While perhaps that approach should be rethought, I'm disinclined to
mess with it just for the benefit of find_typedefs.
regards, tom lane
Andrew Dunstan <andrew@dunslane.net> writes: > For OSX we'd construct the list via File::Find to recurse through the > directories. > So, something like this: Thanks for the tips. The attached patch against buildfarm 4.11 seems to work, cf http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedary&dt=2014-04-03%2003%3A33%3A16&stg=typedefs I tried it on current OSX (10.9.2) as well as dromedary's 10.6.8. It does not work on prairiedog (10.4.11) --- the debug info format seems to be different that far back. Can't say that's worth worrying about. regards, tom lane *** build-farm-4.11/run_build.pl~ Fri Jun 14 09:05:52 2013 --- build-farm-4.11/run_build.pl Wed Apr 2 23:31:27 2014 *************** *** 1506,1520 **** my $objdump = $host || 'objdump'; my @err = `$objdump -W 2>&1`; my @readelferr = `readelf -w 2>&1`; my %syms; my @dumpout; my @flds; ! foreach my $bin ( ! glob("$installdir/bin/*"), ! glob("$installdir/lib/*"), ! glob("$installdir/lib/postgresql/*") ! ) { next if $bin =~ m!bin/(ipcclean|pltcl_)!; next unless -f $bin; --- 1506,1536 ---- my $objdump = $host || 'objdump'; my @err = `$objdump -W 2>&1`; my @readelferr = `readelf -w 2>&1`; + my $using_osx = (`uname` eq "Darwin\n"); + my @testfiles; my %syms; my @dumpout; my @flds; ! if ($using_osx) ! { ! # On OS X, we need to examine the .o files ! my $obj_wanted = sub { ! /^.*\.o\z/s && push(@testfiles, $File::Find::name); ! }; ! ! File::Find::find($obj_wanted,$pgsql); ! } ! else ! { ! # Elsewhere, look at the installed executables and shared libraries ! @testfiles = ( ! glob("$installdir/bin/*"), ! glob("$installdir/lib/*"), ! glob("$installdir/lib/postgresql/*") ! ); ! } ! foreach my $bin (@testfiles) { next if $bin =~ m!bin/(ipcclean|pltcl_)!; next unless -f $bin; *************** *** 1534,1540 **** } elsif ( @readelferr > 10 ) { - # FreeBSD, similar output to Linux @dumpout = `readelf -w $bin 2>/dev/null | egrep -A3 DW_TAG_typedef 2>/dev/null`; --- 1550,1555 ---- *************** *** 1546,1551 **** --- 1561,1579 ---- $syms{$flds[-1]} =1; } } + elsif ($using_osx) + { + @dumpout = + `dwarfdump $bin 2>/dev/null | egrep -A2 TAG_typedef 2>/dev/null`; + foreach (@dumpout) + { + @flds = split; + next unless (@flds == 3); + next unless ($flds[0] eq "AT_name("); + next unless ($flds[1] =~ m/^"(.*)"$/); + $syms{$1} =1; + } + } else { @dumpout = `$objdump --stabs $bin 2>/dev/null`;
I wrote: > Thanks for the tips. The attached patch against buildfarm 4.11 seems > to work, cf > http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedary&dt=2014-04-03%2003%3A33%3A16&stg=typedefs BTW, after looking a bit more closely at what this added to the typedefs list, I realize that this mechanism also collects typedefs that appear in files that are compiled but never installed, for example src/timezone/zic and the ecpg regression tests. This seems like a Good Thing, since certainly pgindent is going to hit the source files for those programs too. I wonder if we ought to switch over to scanning the .o files on all platforms? regards, tom lane
On 04/03/2014 12:01 AM, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: >> For OSX we'd construct the list via File::Find to recurse through the >> directories. >> So, something like this: > Thanks for the tips. The attached patch against buildfarm 4.11 seems > to work, cf > http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=dromedary&dt=2014-04-03%2003%3A33%3A16&stg=typedefs > I tried it on current OSX (10.9.2) as well as dromedary's 10.6.8. > It does not work on prairiedog (10.4.11) --- the debug info format > seems to be different that far back. Can't say that's worth worrying > about. > > Thanks, applied. > BTW, after looking a bit more closely at what this added to the typedefs > list, I realize that this mechanism also collects typedefs that appear in > files that are compiled but never installed, for example src/timezone/zic > and the ecpg regression tests. This seems like a Good Thing, since > certainly pgindent is going to hit the source files for those programs > too. I wonder if we ought to switch over to scanning the .o files on > all platforms? Sure, if someone wants to work out what's involved. Do the MSVC tools have some gadget for extracting symbols from .o files in a similar way? It would be nice to get them into the mix too. cheers andrew