Thread: Re: [GENERAL] Symbols and versioning of binary releases; running a symbol server

On Thu, Jun 16, 2011 at 02:52, Craig Ringer <craig@postnewspapers.com.au> wrote:
> Hi (EnterpriseDB) folks
>
> I've been working with someone off list to get some information about a
> crash they encounter during a batch run. We're generating a crash dump, but
> I'm having some issues getting matching symbols so I can examine it.
>
> One thing that would help with this would be if the EnterpriseDB releases
> included their build revision in the output of "SELECT version()", so it's
> always clear exactly what build is in use.
>
> I've also noticed in this process that the "File version" on postgres.exe
> bears no apparent relationship to the EnterpriseDB release number. For
> example, postgresql 8.4.2-2 has a File Version of 8.4.2-104 while 8.4.2-1
> has a file version of (IIRC) 8.4.2-9343 . Is there any way that can be
> improved?

1) This is not actually an EnterpriseDB thing - those versions and
stamps are set by the build system
2) Why -general, and not -hackers? ;) I'll move it...


To get to your points:

The last digit of the version number is actually the build *day*. It's
calculated by the formula:
    my $d = ($year - 100) . "$yday";

I have a feeling we've overflowed that field. The value today
should'äve been 11166. I think we overflowed it when the year turned
2000, without noticing! The docs claim it's a 16 bit integer though,
which should've worked.

We could (once we've figured out why it's wrong) put that number in
the version string as well. Or some other number - if we can pick a
good one.

I don't think the EDB installers should have a *different* string than
what you'd get if you built the same thing manually...


> Even better would be to put zipped symbols directories onto the EDB download
> site, arranged by Pg version. Bonus points for having symlinks from the

Or right alongside the downloads themselves.

> md5sum of postgres.exe to the matching symbols. Better again would be to run
> a public symbol server with symbols for all builds EnterpriseDB releases:
>
> http://chadaustin.me/2009/03/reporting-crashes-in-imvu-creating-your-very-own-symbol-server/
>
> ... so there's no need to play version guessing games, you just point your
> debugger at the symbol server and it fetches what it needs on demand.
>
> Come to think of it, I can probably run a public symbol server myself if the
> EDB folks don't want to, but it'd be lovely if they were willing to do so
> because it could be integrated into the release process to ensure symbols
> were never missing for a build that hit public release.

Hmm. That site talks about sharing them over a windows fileshar,e I
doubt anybody wants to do that publically.

Now, if this can be made to serve off a simple http (or ftp) server,
we could probably run a server for it off the infrastructure - but
that's assuming someone actually uploads the symbols as builds are
created ;) And it requires the server not to be windows, and using
simple protocols...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 16/06/2011 2:47 PM, Magnus Hagander wrote:

> We could (once we've figured out why it's wrong) put that number in
> the version string as well. Or some other number - if we can pick a
> good one.
>
> I don't think the EDB installers should have a *different* string than
> what you'd get if you built the same thing manually...

In this case a different (or additional) string for EDB releases is 
exactly what I'm after - a string that uniquely identifies a release 
made with an EDB installer, so that "8.4.2-1" can be reliably identified 
as separate from "8.4.2-2" and separate again from a hand-built Pg or a 
.zip release.

Maybe an attribute other than File Version can be used, as custom 
attributes can IIRC be defined easily.

That said, it turns out it doesn't matter much if you have a proper 
symbols cache maintained by symstore.exe . It just figures it out based 
on the hash of the binary, so you don't need to know exactly what 
release you're using.

>> Even better would be to put zipped symbols directories onto the EDB download
>> site, arranged by Pg version. Bonus points for having symlinks from the
>
> Or right alongside the downloads themselves.

Well, they're already in the installer bundles. It'd just be nice to 
have an easily browsed directory of all the old versions, one that 
didn't require downloading and unpacking the installer for that version.

To download 8.4.2-2 at all I had to go to download the latest EDB 
release, note the URL and play substitution games. Maybe I just didn't 
find the "old versions" link that'll turn out to be lurking somewhere.

Again, though, a proper symbols server would pretty much take care of 
this problem.

> Hmm. That site talks about sharing them over a windows fileshar,e I
> doubt anybody wants to do that publically.

Oops.

The Microsoft symbol server serves over HTTP and I hadn't noticed that 
the proposed method in that article was SMB as I was rushing off to work 
when I finished that post.

It looks like running your own symbol over http is simple, though. Just 
expose the directory tree created by symstore.exe over http. No 
Windows-specific stuff required. I just tested it out with a local 
Apache instance and it works fine.

Most instructions talk about symproxy.exe but it turns out you don't 
need that unless you want to run a proxy for Microsoft's symbol server. 
That's is undesirable and unnecessary in a public symbol server.

It'd be pretty trivial to rsync a copy of the symbols directory created 
by symstore.exe from the build host over to a public web server, so 
offering an http symbol server looks quite painless.

As for adding the symbols, that just needs to be part of the release 
process, such that an EDB release can't get to the download page without 
first having had its symbols added to the store.

The Windows "symstore" command from Debugging Tools for Windows is 
requried to update the symbol store. It's invoked as:

C:\Program Files\PostgreSQL\8.4>symstore add /s 
"C:\Users\Craig\Developer\PgSymbols" /t PostgreSQL /v "9.0.0-1" 
/compress /r /f "C:\Program Files\PostgreSQL\9.0"

to store to a local symbol store C:\Users\Craig\Developer\PgSymbols . If 
used for a public server that would be a local or UNC path to a 
directory that was getting rsync'd to a public web server instead.

-- 
Craig Ringer

Tech-related writing at http://soapyfrogs.blogspot.com/

On Thu, Jun 16, 2011 at 11:31 AM, Craig Ringer
<craig@postnewspapers.com.au> wrote:
> In this case a different (or additional) string for EDB releases is exactly
> what I'm after - a string that uniquely identifies a release made with an
> EDB installer, so that "8.4.2-1" can be reliably identified as separate from
> "8.4.2-2" and separate again from a hand-built Pg or a .zip release.

That's not feasible, as the -x part of the number is the installer
build number and is unrelated to the build of the binaries. We
regularly rebuild the installers to fix minor issues without
rebuilding the binaries.

> To download 8.4.2-2 at all I had to go to download the latest EDB release,
> note the URL and play substitution games. Maybe I just didn't find the "old
> versions" link that'll turn out to be lurking somewhere.

I'm not sure I understand the problem. We ship the symbols in the
package, so why did you have to go looking for an old version to debug
an issue?


-- 
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 16/06/2011 6:38 PM, Dave Page wrote:

>> To download 8.4.2-2 at all I had to go to download the latest EDB release,
>> note the URL and play substitution games. Maybe I just didn't find the "old
>> versions" link that'll turn out to be lurking somewhere.
>
> I'm not sure I understand the problem. We ship the symbols in the
> package, so why did you have to go looking for an old version to debug
> an issue?

Because I was debugging a dump. I didn't have that version installed on 
my machine.

I could've just got them to send a zipped copy of their symbols 
directory, too.

It turns out that it doesn't matter whether individual releases can be 
perfectly identified; the symbol server indexer takes care of that. I'll 
just collect symbols from each release and pop them in my symbol cache 
so I don't have to worry in future.

Publishing a complete symbol cache via a symbol server would be pretty 
handy, though. If needs be it's something I'll do, it'll just be harder 
for me to keep it up to date and not miss any releases.

-- 
Craig Ringer

Tech-related writing at http://soapyfrogs.blogspot.com/