Thread: Building psqlODBC installers
I found the procedure for building the installers quite cumbersome.
There are several ways to do it, using the PowerShell installers, or
Make.bat or Makex64.bat files, and you have to first build the right
drivers versions etc.
To simplify things, I invented yet another way to build the installers ;-).
I pushed a patch that adds new targets to the top-level win64.mak file.
The most interesting target is "world", which builds both x64 and x86
versions of the driver, both ANSI and Unicode versions. And then it
builds both x86 and x64 installers. All in one command:
nmake /f win64.mak world
I don't know how you've been building the official releases so far, but
I would strongly recommend using this method. I'd like to remove the
installer/make.bat and installer/makex64.bat files now. The only reason
I didn't do that yet is that I don't actually do the official releases
myself, so wanted to make sure this works for everyone.
So: Hiroshi, Hiroshi, Dave, does this method work for you, for building
the official builds? Would it be helpful to add new targets to build the
various .zip files, for pushing directly to the FTP site?
Any objections to removing the following files:
installer/Make.bat
installer/Makex64.bat
installer/buildX86-installer.ps1
installer/buildX64-installer.ps1
?
To be honest, I would also like to remove the MSBuild stuff that
Inoue-san did last summer. I prefer the nmake infrastructure, and don't
find it useful myself. But I know you spent a lot of effort on writing
that stuff... Do you use it actively? What would it take to convince you
to switch back to the nmake system? I volunteer to improve the nmake
system, if there's something missing.
One cumbersome thing left is bumping the version number. We have version
numbers in many different files (and my commit only made the situation
worse). For example, see the latest commit that bumped up the version
number to 09.03.0300:
commit dbd2a9eb2be8913bdfbe4331f8a1649966e39906
Author: Hiroshi Saito <hiroshi@winpg.jp>
Date: Wed May 14 00:24:27 2014 +0900
Prep release ver 09.03.0300
configure.ac | 2 +-
docs/release.html | 33 +++++++++++++++++++++++++++++++++
installer/Make.bat | 3 ++-
installer/MakeX64.bat | 3 ++-
installer/buildX64-installer.ps1 | 1 +
version.h | 6 +++---
winbuild/configuration_template.xml | 2 +-
7 files changed, 43 insertions(+), 7 deletions(-)
And that one forgot to update installer/buildX86-installer.ps1! And it's
not just a version number, many of those files contain product UUIDs
like this (from Make.bat):
+SET PRODUCTID["09.03.0300"]="D3527FA5-9C2B-4550-A59B-9534A78950F4"
these need to be kept in sync across all files.
So my next endeavor is to try to unify all that. There should be exactly
one file that contains all the version numbers and product codes, and
one script or makefile target to update it.
- Heikki
On Fri, Jun 13, 2014 at 9:33 PM, Heikki Linnakangas <hlinnakangas@vmware.com> wrote: > I found the procedure for building the installers quite cumbersome. There > are several ways to do it, using the PowerShell installers, or Make.bat or > Makex64.bat files, and you have to first build the right drivers versions > etc. > > To simplify things, I invented yet another way to build the installers ;-). > I pushed a patch that adds new targets to the top-level win64.mak file. The > most interesting target is "world", which builds both x64 and x86 versions > of the driver, both ANSI and Unicode versions. And then it builds both x86 > and x64 installers. All in one command: > > nmake /f win64.mak world Awesome +1 for the strategy with nmake. Just had a look at what you pushed and this is really straight-forward. While looking at that I am wondering as well about having localized installers with some embedded mst files created by torch. Perhaps this is too much thinking? -- Michael
On 06/13/2014 08:33 PM, Heikki Linnakangas wrote: > To be honest, I would also like to remove the MSBuild stuff that > Inoue-san did last summer. I prefer the nmake infrastructure, and don't > find it useful myself. But I know you spent a lot of effort on writing > that stuff... Do you use it actively? Personally, I just found it immensely useful. It also properly detects the SDK, which is nice. I don't really see the appeal of nmake. Why use it? Pg proper got rid of it (for good reasons, IMO) and it's generally pretty deprecated for MS tools. I don't pretend that MSBuild is the most beautiful thing around (more like the bastard child of CMake and Ant) but it does fit in with the tooling really well, plays well with MSVC, and is well supported. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
(2014/06/13 21:33), Heikki Linnakangas wrote: > I found the procedure for building the installers quite cumbersome. > There are several ways to do it, using the PowerShell installers, or > Make.bat or Makex64.bat files, and you have to first build the right > drivers versions etc. My intension was to use Powershell scripts. It is not difficult to make the script to make all binaries and installers but My main point is to use MSBuild and always type ./BuildAll.bat ... to build binaries. It's not necessary to build installers to test the changes of source code. I've already explained the advantages to use MSBuild. THe only disadvantage is it's unavailable for vc9 or before. I strongly object to use nmake other than vc9 or before. As for installers, other installers are welcome. regards, Hiroshi Inoue
On Fri, Jun 13, 2014 at 4:07 PM, Craig Ringer <craig@2ndquadrant.com> wrote: > On 06/13/2014 08:33 PM, Heikki Linnakangas wrote: > >> To be honest, I would also like to remove the MSBuild stuff that >> Inoue-san did last summer. I prefer the nmake infrastructure, and don't >> find it useful myself. But I know you spent a lot of effort on writing >> that stuff... Do you use it actively? > > Personally, I just found it immensely useful. > > It also properly detects the SDK, which is nice. > > I don't really see the appeal of nmake. Why use it? Pg proper got rid of > it (for good reasons, IMO) and it's generally pretty deprecated for MS > tools. > > I don't pretend that MSBuild is the most beautiful thing around (more > like the bastard child of CMake and Ant) but it does fit in with the > tooling really well, plays well with MSVC, and is well supported. I haven't worked on ODBC in a long while, but from a higher level perspective of someone who builds and packages multi-platform software for a living, I whole heartedly agree that MSBuild should be used in place of nmake, for essentially the same reasons to those Craig give. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company
(2014/06/13 21:33), Heikki Linnakangas wrote: > I found the procedure for building the installers quite cumbersome. > There are several ways to do it, using the PowerShell installers, or > Make.bat or Makex64.bat files, and you have to first build the right > drivers versions etc. > > To simplify things, I invented yet another way to build the installers ;-). > > I pushed a patch that adds new targets to the top-level win64.mak file. > The most interesting target is "world", which builds both x64 and x86 > versions of the driver, both ANSI and Unicode versions. And then it > builds both x86 and x64 installers. All in one command: > > nmake /f win64.mak world > > I don't know how you've been building the official releases so far, but > I would strongly recommend using this method. I'd like to remove the > installer/make.bat and installer/makex64.bat files now. The only reason > I didn't do that yet is that I don't actually do the official releases > myself, so wanted to make sure this works for everyone. > > So: Hiroshi, Hiroshi, Dave, does this method work for you, for building > the official builds? Would it be helpful to add new targets to build the > various .zip files, for pushing directly to the FTP site? It doesn't work under Visual Studio 2013 Express for Windows Desktop because 1. TARGET_CPU variable isn't set. 2. setenv.cmd doesn't exist. IIRC the only environment win64.mak works is Windows SDK 7.1. One of the reason why I switched to MSBuild is to avoid such kind of troubles. regards, Hiroshi Inoue
On 06/15/2014 06:06 AM, Inoue, Hiroshi wrote: >> > > It doesn't work under Visual Studio 2013 Express for Windows Desktop > because > 1. TARGET_CPU variable isn't set. > 2. setenv.cmd doesn't exist. > > IIRC the only environment win64.mak works is Windows SDK 7.1. > One of the reason why I switched to MSBuild is to avoid such > kind of troubles. Yeah, SDK detection is a nightmare. I didn't know about the great work you'd done with your Powershell scripts when I wrote pg_build_win (or maybe it wasn't around then). It has some mess in it for SDK detection, and that's just to determine the version and check the SDK, it doesn't actually try to find it. There aren't even standard registry keys for it, as different service pack levels and different versions (Enterprise, normal, Express, etc) have their own registry key locations. It's like Microsoft, for every SDK release, said: "Take the old packaging team out. Lock them outside the doors. Then find some random people from another team and make them package this SDK, with absolutely no knowledge of how the prior team did it." Anything we can use to make that easier is a win, and MSBuild is one of the few things that counts there. IMO, support for SDKs older than v10 (VS 2010, WinSDK 7.1) is completely uninteresting. If someone wants that, they can go get a matchingly ancient version of psqlODBC. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 06/16/2014 07:13 AM, Craig Ringer wrote: > On 06/15/2014 06:06 AM, Inoue, Hiroshi wrote: >>> >> >> It doesn't work under Visual Studio 2013 Express for Windows Desktop >> because >> 1. TARGET_CPU variable isn't set. >> 2. setenv.cmd doesn't exist. >> >> IIRC the only environment win64.mak works is Windows SDK 7.1. >> One of the reason why I switched to MSBuild is to avoid such >> kind of troubles. Oh, ouch. > Yeah, SDK detection is a nightmare. > > I didn't know about the great work you'd done with your Powershell > scripts when I wrote pg_build_win (or maybe it wasn't around then). It > has some mess in it for SDK detection, and that's just to determine the > version and check the SDK, it doesn't actually try to find it. > > There aren't even standard registry keys for it, as different service > pack levels and different versions (Enterprise, normal, Express, etc) > have their own registry key locations. > > It's like Microsoft, for every SDK release, said: "Take the old > packaging team out. Lock them outside the doors. Then find some random > people from another team and make them package this SDK, with absolutely > no knowledge of how the prior team did it." Ok, I agree we don't want to do SDK detection. > Anything we can use to make that easier is a win, and MSBuild is one of > the few things that counts there. Hmph. I don't particularly like the MSBuild format. But other seem to hate nmake even more, so MSBuild it is then. The important thing is to standardize on one build mechanism. I'll try to whip the MSbuild system into shape, so that it can build all the installers in one command. I'd also like to remove the dependency to PowerShell. Surely plain MSBuild can do what we want, so let's avoid the extra dependency if we can. > IMO, support for SDKs older than v10 (VS 2010, WinSDK 7.1) is completely > uninteresting. If someone wants that, they can go get a matchingly > ancient version of psqlODBC. Agreed. - Heikki
On 06/16/2014 07:00 PM, Heikki Linnakangas wrote: > I'll try to whip the MSbuild system into shape, so that it can build all > the installers in one command. I'd also like to remove the dependency to > PowerShell. Surely plain MSBuild can do what we want, so let's avoid the > extra dependency if we can. Ok, I came up with the attached. To build, just type: MSBuild at the top level of the source tree. This builds all four versions of the driver: x86_ANSI, x86_Unicode, x64_ANSI and x64_Unicode, and also builds the installers, both 32-bit and 64-bit versions. The build can be configured by copying winbuild/configuration-defaults.props file as winbuild/configuration-local.props, and modifying as needed. This completely removes all the other methods to build on Windows: the NMake files, the PowerShell scripts, the .bat files and the plain .vcproj/.sln files at the top level. The main project file that "MSBuild" launches is called winbuild.proj. I used winbuild/platformbuild.vcxproj as the model for winbuild.proj, but it has been heavily modified. There are some rough edges: building without MSDTC support doesn't work, for example. But the default configuration works. I'm using Microsoft Windows SDK 7.1 in my system, I haven't tested this with other SDK or Visual Studio versions. One notable change over the existing powershell system is that the this extracts the driver's version number from version.h, while the powershell scripts did it the other way 'round. This is better, because now you don't need to modify both files when bumping the version number. Question: Is everyone OK with this approach? Does this work for the SDK/compiler versions that you are actively using? If you have a problem with this, please speak up now. If there's some feature or configuration that's missing, I can work to add that, as long as we agree on this general direction of replacing all the other build mechanisms with plain MSBuild files. - Heikki
Attachment
On 06/18/2014 12:29 AM, Heikki Linnakangas wrote: > > Ok, I came up with the attached. To build, just type: > > MSBuild For the sake of easy exchange of test results and changes, do you have that in a public git branch anywhere? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On 06/18/2014 09:35 AM, Craig Ringer wrote: > On 06/18/2014 12:29 AM, Heikki Linnakangas wrote: >> >> Ok, I came up with the attached. To build, just type: >> >> MSBuild > > For the sake of easy exchange of test results and changes, do you have > that in a public git branch anywhere? Ah yeah: https://github.com/hlinnaka/psqlodbc/tree/winbuild - Heikki
On 06/18/2014 03:31 PM, Heikki Linnakangas wrote: >>> >> >> For the sake of easy exchange of test results and changes, do you have >> that in a public git branch anywhere? > > Ah yeah: https://github.com/hlinnaka/psqlodbc/tree/winbuild I should have time to test that out on VS2012 Express and VS 2010 Express tomorrow. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
(2014/06/17 22:14), Heikki Linnakangas wrote:
> On 06/16/2014 07:00 PM, Heikki Linnakangas wrote:
>> I'll try to whip the MSbuild system into shape, so that it can build all
>> the installers in one command. I'd also like to remove the dependency to
>> PowerShell. Surely plain MSBuild can do what we want, so let's avoid the
>> extra dependency if we can.
Unfortunately I don't prefer to pack all into one command(tool).
Though I chose instead of nmake, I don't love it so much.
Though I don't have much time to confirm the patch, I noticed
the followings for the time being.
1. The default target seems to build installers. Do we have to
build installers so often? I have never built installers for
a long time for purposes other than testing themselves.
2. ISTM there's no target which builds x86 and x64 drivers.
Is it right?
3. Why does the patch remove installer/psqlodbc-setup together?
> One notable change over the existing powershell system is that the this
> extracts the driver's version number from version.h, while the
> powershell scripts did it the other way 'round.
> This is better, because
> now you don't need to modify both files when bumping the version number.
I left version.h for nmake build. There's no need to maintain it when
switching to MSBuild is done.
regards,
Hiroshi Inoue
On 06/18/2014 03:10 PM, Hiroshi Inoue wrote: > (2014/06/17 22:14), Heikki Linnakangas wrote: >> On 06/16/2014 07:00 PM, Heikki Linnakangas wrote: >>> I'll try to whip the MSbuild system into shape, so that it can build all >>> the installers in one command. I'd also like to remove the dependency to >>> PowerShell. Surely plain MSBuild can do what we want, so let's avoid the >>> extra dependency if we can. > > Unfortunately I don't prefer to pack all into one command(tool). > Though I chose instead of nmake, I don't love it so much. > > Though I don't have much time to confirm the patch, I noticed > the followings for the time being. > > 1. The default target seems to build installers. Do we have to > build installers so often? I have never built installers for > a long time for purposes other than testing themselves. Ok, how about making BuildDriverBoth the default then? That builds the Unicode and ANSI drivers in the current Debug/Release configuration, for the current target Platform. > 2. ISTM there's no target which builds x86 and x64 drivers. > Is it right? No separate target, but you can pass the TARGET_CPU property: msbuild /t:BuildDriver /p:TARGET_CPU=x86 Or at least with Windows SDK 7.1, you can also use "setenv /x86" or "setenv /x64" to change the current target. > 3. Why does the patch remove installer/psqlodbc-setup together? > >> One notable change over the existing powershell system is that the this >> extracts the driver's version number from version.h, while the >> powershell scripts did it the other way 'round. >> This is better, because >> now you don't need to modify both files when bumping the version number. > > I left version.h for nmake build. There's no need to maintain it when > switching to MSBuild is done. version.h is also used by Unix builds. That's why I think we should also use version.h with the Windows builds - otherwise we end up with two different ways to derive the version number, which is bad. - Heikki
(forgot to answer one point) On 06/18/2014 03:10 PM, Hiroshi Inoue wrote: > 3. Why does the patch remove installer/psqlodbc-setup together? Because those files are not used for anything. You added them last July, but nothing's happened since, so it looks like a dead project to me. If you ever want to continue working on it, you can always resurrect it from the git history. - Heikki
(2014/06/18 21:26), Heikki Linnakangas wrote: > (forgot to answer one point) > > On 06/18/2014 03:10 PM, Hiroshi Inoue wrote: >> 3. Why does the patch remove installer/psqlodbc-setup together? > > Because those files are not used for anything. You added them last July, > but nothing's happened since, so it looks like a dead project to me. Isn't the stuff essentially irrelevant to this patch? Please don't remove them together with this patch. It's possible to distribute VC++ redistributables with drivers using the project. However I'm planning to use this project for another purpose. regards, Hiroshi Inoue -- I am using the free version of SPAMfighter. SPAMfighter has removed 10971 of my spam emails to date. Get the free SPAMfighter here: http://www.spamfighter.com/len Do you have a slow PC? Try a Free scan http://www.spamfighter.com/SLOW-PCfighter?cid=sigen
On 06/19/2014 06:26 AM, Inoue, Hiroshi wrote: > (2014/06/18 21:26), Heikki Linnakangas wrote: >> (forgot to answer one point) >> >> On 06/18/2014 03:10 PM, Hiroshi Inoue wrote: >>> 3. Why does the patch remove installer/psqlodbc-setup together? >> >> Because those files are not used for anything. You added them last July, >> but nothing's happened since, so it looks like a dead project to me. > > Isn't the stuff essentially irrelevant to this patch? > Please don't remove them together with this patch. Ok, fair enough. Would you mind if I remove it as a separate patch? - Heikki
(2014/06/19 14:51), Heikki Linnakangas wrote: > On 06/19/2014 06:26 AM, Inoue, Hiroshi wrote: >> (2014/06/18 21:26), Heikki Linnakangas wrote: >>> (forgot to answer one point) >>> >>> On 06/18/2014 03:10 PM, Hiroshi Inoue wrote: >>>> 3. Why does the patch remove installer/psqlodbc-setup together? >>> >>> Because those files are not used for anything. You added them last July, >>> but nothing's happened since, so it looks like a dead project to me. >> >> Isn't the stuff essentially irrelevant to this patch? >> Please don't remove them together with this patch. > > Ok, fair enough. Would you mind if I remove it as a separate patch? I think it's a useful project and I'm planning to use it in the next release. regards, Hiroshi Inoue