Re: How to build statically on Windows - Mailing list psycopg

From Dan Davis
Subject Re: How to build statically on Windows
Date
Msg-id CAFzonYb+Twnk9Zwzw56gQyiGL6piJA3hN0SsMRJfFe4SR1-Djw@mail.gmail.com
Whole thread Raw
In response to Re: How to build statically on Windows  (Dan Davis <dansmood@gmail.com>)
Responses Re: How to build statically on Windows  (Dan Davis <dansmood@gmail.com>)
Re: How to build statically on Windows  (Daniele Varrazzo <daniele.varrazzo@gmail.com>)
List psycopg

I am going to tell my team this is much, much harder than cx_Oracle, and we may have no choice but to coordinate updates to more recent copies. I would never ask you guys to go back and backfill, e.g. build 2.8.5 for Python 3.9 just for us.

I decided to at least try the "easy button" and setup a fork of psycopg2 with appveyor.  I changed the files in my fork for the .appveyor sub-directory to only build for Python 3.9 and Python 3.6 64 bit.  However, it seems that appveyor has too many settings for me to easily duplicate the environment - not enough is controlled simply from the files in .appveyor to make this easy button super easy.  If it is possible for someone to document the settings, that probably would be of benefit for you guys as well.

I know most people creating pull requests can simply rely on github.com and appveyor to test that everything works, however.




On Tue, Oct 5, 2021 at 4:40 PM Dan Davis <dansmood@gmail.com> wrote:
Jason,

I tried to call the appveyor script, but it was looking for a number of Environment variables I did not have.  I got through PY_VER=39 and then thought better of it. 

Do you think it would be easier to do it this way:
* fork psycopg2
* make my own appveyor account that builds the versions I need...

Simulate the appveyor environment on my own workstation

Wow, what a lot I've forgotten since before manylinux1 - I used to do this for lxml, PyCrypto, and a lot of other packages :).   Still, they were all more or less one-offs - I never got as automated as modern DevOps allows these days.



On Tue, Oct 5, 2021 at 2:11 PM Jason Erickson <jerickso@stickpeople.com> wrote:
Hi Dan,

Yeah, unfortunately --static-libpq doesn't do what it should anymore.  We should remove that option.

When building pyscopg2 from source, the binary Windows Postgres packages does not include the statically linked library (In fairness, I haven't checked the latest packages, but that is how it has been in the past).  The library included with the binary packages was the DLL import library and it was named libpq.lib.  This is an issue, because originally when you built from source on windows, the file libpq.lib was the static link library, whereas the DLL import library was named libpqdll.lib.  So we have a name inconsistency and no way to link statically with the Postgres binary distribution.

This got even more convoluted a few major versions back with the source of Postgres, as the Windows perl build scripts quit creating the build file for the static link library, only building the DLL import library AND naming it libpq.lib.  With our Appveyor build script, I cheated and modified the perl script to build the library instead of the DLL at this line:
      file_replace('Mkvcbuild.pm', "'libpq', 'dll'", "'libpq', 'lib'")

With that said, I am not happy with that solution and always intended to revisit the setup script portion for windows, but always had more questions than answers, some of them:
* If static libraries are not part of the Postgres binary distribution (or even the build from source option by default), do we concern ourselves with them? Personally I prefer static libraries because I think it has less support issues, but???
* If people are building psycopg from source, what libraries do we assume they have installed?  pg_config probably would not be working, so include/library paths would have to be passed. What about other dependencies (ie openssl, still use the has_ssl flag?)?  There are slightly different link libraries between some Postgres, so versions matter, too.
* Do we try to differentiate between the DLL import library and the static library since the name can be the same between them (size?)?
* Or do we say heck with it and just link against a file named libpq.lib, letting the builder point to the libpq they want to use?

Maybe an option is to have setup.py on windows call the appveyor script (with some modifications) and build everything from scratch?  It takes about 40 minutes to build everything, tho, and does require some storage space for the source and build files, so not the best solution.

-jason


On Mon, Oct 4, 2021 at 5:13 PM Daniele Varrazzo <daniele.varrazzo@gmail.com> wrote:
Hi Dan,

On Tue, 5 Oct 2021 at 01:01, Dan Davis <dansmood@gmail.com> wrote:
>
> Can anyone give me a solution to build psycopg2 statically on Windows?
>
> I have succeeded in building it, but when I run dumpbin /dependents on the generated file (the PYD file), it still depends on libpq.dll even when I pass --static-libpq.

I haven't personally used --static-libpq in a long time, and neither
have I used windows for a while. As far as I know that part of the set
up might have bitrotten.

If anyone can help Dan it would be appreciated.

Dan, there is a ticket/MR in the tracker of which I've never been able
to make completely sense: https://github.com/psycopg/psycopg2/pull/758
Would you like to check if that's the issue that doesn't allow
building the lib statically and if so can you propose a MR or just
acknowledge that the proposed one works as expected?

Thank you everyone

-- Daniele


psycopg by date:

Previous
From: Dan Davis
Date:
Subject: Re: How to build statically on Windows
Next
From: Dan Davis
Date:
Subject: Re: How to build statically on Windows