Thread: Running regression tests on Windows

On 05/20/2014 04:31 PM, Michael Paquier wrote:
> Now, I am trying to figure out how to run those regressions with
> pg_regress. On Linux, we use a given combination of odbcinst.ini and
> odbc.ini with a launcher that actually sets ODBCSYSINI to set the path
> where looking for the .ini files. I have been trying several combinations
> of --launcher as well as setting up ODBCSYSINI in either the makefile or as
> an environment viriable without success, pg_regress continues to complain
> with the following error:
> ! SQLDriverConnect failed.
> ! IM002=[Microsoft][ODBC Driver Manager] Data source name not found and no
> default driver specified
> So obviously pg_regress is not able to find the initialization files
> properly...

I believe the ODBCSYSINI environment variable is specific to unixodbc,
and not obeyed by the Windows ODBC library. I don't know the preferred
way to do it on Windows would be.

- Heikki

On 05/21/2014 08:53 AM, Michael Paquier wrote:
> On Wed, May 21, 2014 at 1:38 AM, Heikki Linnakangas <hlinnakangas@vmware.com
>> wrote:
>
>> On 05/20/2014 04:31 PM, Michael Paquier wrote:
>>
>>> Now, I am trying to figure out how to run those regressions with
>>> pg_regress. On Linux, we use a given combination of odbcinst.ini and
>>> odbc.ini with a launcher that actually sets ODBCSYSINI to set the path
>>> where looking for the .ini files. I have been trying several combinations
>>> of --launcher as well as setting up ODBCSYSINI in either the makefile or
>>> as
>>> an environment viriable without success, pg_regress continues to complain
>>> with the following error:
>>> ! SQLDriverConnect failed.
>>> ! IM002=[Microsoft][ODBC Driver Manager] Data source name not found and no
>>> default driver specified
>>> So obviously pg_regress is not able to find the initialization files
>>> properly...
>>>
>>
>> I believe the ODBCSYSINI environment variable is specific to unixodbc, and
>> not obeyed by the Windows ODBC library. I don't know the preferred way to
>> do it on Windows would be.
>>
> After some research on Redmond's website, It happens that you need to add
> manually an ODBC driver in the Windows registry to do that.

Ugh. Did you look at the ODBCCONF.exe program?

For the first version, I think we can punt on that. Let's just require
that you install the driver manually, using the installer for example,
and create the DSN manually.


> Btw, I have been able to do some hacking on that, finishing with the
> patch attached to support regressions on Windows.

Great!

> The following things are done:
> - nmake is used from a Windows SDK to run the tests.
> - Addition of documentation in test/README.txt.
> - test/src/common.c has been lightly patched to pass some custom parameters
> to run the tests with WIN32 as odbcinst.ini/odbc.ini cannot be used.
> Actually the driver name, server IP and database name are needed in the
> connection string.

Hmm. If we just require the user to create the DSN manually, we don't
need to do anything different in common.c.

> - Because Windows does not like much using non-full paths when invocating
> the *-test executables, I have moved all the tests to use input/ and
> output/ with @abd_srcdir@. This has the huge advantage to make the output
> transparent on all the OSes and all the development environments.

Oh. I don't think that's really necessary. Windows can handle relative
paths just fine. With the \! syntax, though, it seems that Windows wants
an extra pair of quotes. So this works:

\! "src/<testname>-test"

That also works on Unix, although the extra quotes are not necessary
there, so let's just do that.

> - A registry file that can be imported in the Windows registry has been
> added in the patch as it is troublesome to create new registry entries
> manually. A couple of clicks is enough to import the file and register a
> custom ODBC driver for the regression tests. That's specified in the docs
> as well.

The registry file didn't get through in the .diff, it just said "Binary
files /dev/null and b/test/win_registry_key.reg differ". Anyway, as I
said above I think the right approach is to give an odbc.conf command
line in the README that you can run. Or maybe run it from the makefile
automatically.

> - Some alternate outputs have been added for the unicode driver.

Oh, WVARCHAR vs VARCHAR, I see..

I committed a modified version of this patch. It's based on yours, with
a few changes:

- I didn't do the moving of files sql -> source and expected -> output.
- Added the double-quotes to the \! commands instead.
- I moved the list of tests to a separate file, called "tests", which is
included in both the Unix Makefile and win.mak. That makes it
unnecessary to update two makefiles when you add a new test (I bet about
50% of all commits that add new test case would forget to update both
files :-) ). It also makes it unnecessary to run ./configure every time
you add a new test, which is nice during development
- reworked win.mak somewhat, to avoid the moving of files (BTW, there is
no "mv" command on plain Windows; it's called "move".)
- I didn't add the alternate outputs yet - later..

> Even with this patch, I am still seeing some tests failing. Those bugs are
> different issues and the idea is to come back to fix them once this patch
> gets in.

I'm also getting some failures, and the numeric test program crashes.
But at least it's now relatively easy to run the regression tests.

- Heikki

On Fri, Jun 13, 2014 at 3:55 AM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> On 05/21/2014 08:53 AM, Michael Paquier wrote:
>>
>> On Wed, May 21, 2014 at 1:38 AM, Heikki Linnakangas
>> <hlinnakangas@vmware.com
>> After some research on Redmond's website, It happens that you need to add
>> manually an ODBC driver in the Windows registry to do that.
> Ugh. Did you look at the ODBCCONF.exe program?
>
> For the first version, I think we can punt on that. Let's just require that
> you install the driver manually, using the installer for example, and create
> the DSN manually.
Didn't know of it before you mentioned it. Looking at it using actions
of the type CONFIGDSN and CONFIGDRIVER would help a lot.

>> The following things are done:
>> - nmake is used from a Windows SDK to run the tests.
>> - Addition of documentation in test/README.txt.
>> - test/src/common.c has been lightly patched to pass some custom
>> parameters
>> to run the tests with WIN32 as odbcinst.ini/odbc.ini cannot be used.
>> Actually the driver name, server IP and database name are needed in the
>> connection string.
>
>
> Hmm. If we just require the user to create the DSN manually, we don't need
> to do anything different in common.c.
Yeah true.

>> - Because Windows does not like much using non-full paths when invocating
>> the *-test executables, I have moved all the tests to use input/ and
>> output/ with @abd_srcdir@. This has the huge advantage to make the output
>> transparent on all the OSes and all the development environments.
>
>
> Oh. I don't think that's really necessary. Windows can handle relative paths
> just fine. With the \! syntax, though, it seems that Windows wants an extra
> pair of quotes. So this works:
>
> \! "src/<testname>-test"
>
> That also works on Unix, although the extra quotes are not necessary there,
> so let's just do that.
OK good to know. I didn't think about that when writing this patch.

>> - A registry file that can be imported in the Windows registry has been
>> added in the patch as it is troublesome to create new registry entries
>> manually. A couple of clicks is enough to import the file and register a
>> custom ODBC driver for the regression tests. That's specified in the docs
>> as well.
>
>
> The registry file didn't get through in the .diff, it just said "Binary
> files /dev/null and b/test/win_registry_key.reg differ". Anyway, as I said
> above I think the right approach is to give an odbc.conf command line in the
> README that you can run. Or maybe run it from the makefile automatically.
>
>
>> - Some alternate outputs have been added for the unicode driver.
>
>
> Oh, WVARCHAR vs VARCHAR, I see..
>
> I committed a modified version of this patch.
Thanks!

>> Even with this patch, I am still seeing some tests failing. Those bugs are
>> different issues and the idea is to come back to fix them once this patch
>> gets in.
>
>
> I'm also getting some failures, and the numeric test program crashes. But at
> least it's now relatively easy to run the regression tests.
For the sake of the archives, I am attaching the failures I am currently seeing.
--
Michael

On Thu, Jun 12, 2014 at 11:55 AM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> On 05/21/2014 08:53 AM, Michael Paquier wrote:
>>
>> On Wed, May 21, 2014 at 1:38 AM, Heikki Linnakangas
>> <hlinnakangas@vmware.com
>>>
>>> wrote:
>>
>>
>>> On 05/20/2014 04:31 PM, Michael Paquier wrote:
>>>
>>>> Now, I am trying to figure out how to run those regressions with
>>>> pg_regress. On Linux, we use a given combination of odbcinst.ini and
>>>> odbc.ini with a launcher that actually sets ODBCSYSINI to set the path
>>>> where looking for the .ini files. I have been trying several
>>>> combinations
>>>> of --launcher as well as setting up ODBCSYSINI in either the makefile or
>>>> as
>>>> an environment viriable without success, pg_regress continues to
>>>> complain
>>>> with the following error:
>>>> ! SQLDriverConnect failed.
>>>> ! IM002=[Microsoft][ODBC Driver Manager] Data source name not found and
>>>> no
>>>> default driver specified
>>>> So obviously pg_regress is not able to find the initialization files
>>>> properly...
>>>>
>>>
>>> I believe the ODBCSYSINI environment variable is specific to unixodbc,
>>> and
>>> not obeyed by the Windows ODBC library. I don't know the preferred way to
>>> do it on Windows would be.
>>>
>> After some research on Redmond's website, It happens that you need to add
>> manually an ODBC driver in the Windows registry to do that.
>
>
> Ugh. Did you look at the ODBCCONF.exe program?
>
> For the first version, I think we can punt on that. Let's just require that
> you install the driver manually, using the installer for example, and create
> the DSN manually.
>
>
>
>> Btw, I have been able to do some hacking on that, finishing with the
>> patch attached to support regressions on Windows.
>
>
> Great!
>
>
>> The following things are done:
>> - nmake is used from a Windows SDK to run the tests.
>> - Addition of documentation in test/README.txt.
>> - test/src/common.c has been lightly patched to pass some custom
>> parameters
>> to run the tests with WIN32 as odbcinst.ini/odbc.ini cannot be used.
>> Actually the driver name, server IP and database name are needed in the
>> connection string.
>
>
> Hmm. If we just require the user to create the DSN manually, we don't need
> to do anything different in common.c.
>
>
>> - Because Windows does not like much using non-full paths when invocating
>> the *-test executables, I have moved all the tests to use input/ and
>> output/ with @abd_srcdir@. This has the huge advantage to make the output
>> transparent on all the OSes and all the development environments.
>
>
> Oh. I don't think that's really necessary. Windows can handle relative paths
> just fine. With the \! syntax, though, it seems that Windows wants an extra
> pair of quotes. So this works:
>
> \! "src/<testname>-test"
>
> That also works on Unix, although the extra quotes are not necessary there,
> so let's just do that.
>
>
>> - A registry file that can be imported in the Windows registry has been
>> added in the patch as it is troublesome to create new registry entries
>> manually. A couple of clicks is enough to import the file and register a
>> custom ODBC driver for the regression tests. That's specified in the docs
>> as well.
>
>
> The registry file didn't get through in the .diff, it just said "Binary
> files /dev/null and b/test/win_registry_key.reg differ". Anyway, as I said
> above I think the right approach is to give an odbc.conf command line in the
> README that you can run. Or maybe run it from the makefile automatically.
>
>
>> - Some alternate outputs have been added for the unicode driver.
>
>
> Oh, WVARCHAR vs VARCHAR, I see..
>
> I committed a modified version of this patch. It's based on yours, with a
> few changes:
>
> - I didn't do the moving of files sql -> source and expected -> output.
> - Added the double-quotes to the \! commands instead.
> - I moved the list of tests to a separate file, called "tests", which is
> included in both the Unix Makefile and win.mak. That makes it unnecessary to
> update two makefiles when you add a new test (I bet about 50% of all commits
> that add new test case would forget to update both files :-) ). It also
> makes it unnecessary to run ./configure every time you add a new test, which
> is nice during development
> - reworked win.mak somewhat, to avoid the moving of files (BTW, there is no
> "mv" command on plain Windows; it's called "move".)
> - I didn't add the alternate outputs yet - later..
>
>
>> Even with this patch, I am still seeing some tests failing. Those bugs are
>> different issues and the idea is to come back to fix them once this patch
>> gets in.
>
>
> I'm also getting some failures, and the numeric test program crashes. But at
> least it's now relatively easy to run the regression tests.
>
> - Heikki



--
Michael

On 06/13/2014 03:39 AM, Michael Paquier wrote:
>> >Oh. I don't think that's really necessary. Windows can handle relative paths
>> >just fine. With the \! syntax, though, it seems that Windows wants an extra
>> >pair of quotes. So this works:
>> >
>> >\! "src/<testname>-test"
>> >
>> >That also works on Unix, although the extra quotes are not necessary there,
>> >so let's just do that.
> OK good to know. I didn't think about that when writing this patch.

On further testing, that only works on Windows with psql version 9.4 or
later. Maybe we can live with that..

- Heikki

On Wed, Jun 18, 2014 at 9:03 PM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> On 06/13/2014 03:39 AM, Michael Paquier wrote:
>>>
>>> >Oh. I don't think that's really necessary. Windows can handle relative
>>> > paths
>>> >just fine. With the \! syntax, though, it seems that Windows wants an
>>> > extra
>>> >pair of quotes. So this works:
>>> >
>>> >\! "src/<testname>-test"
>>> >
>>> >That also works on Unix, although the extra quotes are not necessary
>>> > there,
>>> >so let's just do that.
>>
>> OK good to know. I didn't think about that when writing this patch.
>
>
> On further testing, that only works on Windows with psql version 9.4 or
> later. Maybe we can live with that..
Ouch OK. Is there any reason specific to psql for that? Btw, I agree
that it's a limitation we can live with..
--
Michael

On 06/18/2014 03:18 PM, Michael Paquier wrote:
> On Wed, Jun 18, 2014 at 9:03 PM, Heikki Linnakangas
> <hlinnakangas@vmware.com> wrote:
>> On 06/13/2014 03:39 AM, Michael Paquier wrote:
>>>>
>>>>> Oh. I don't think that's really necessary. Windows can handle relative
>>>>> paths
>>>>> just fine. With the \! syntax, though, it seems that Windows wants an
>>>>> extra
>>>>> pair of quotes. So this works:
>>>>>
>>>>> \! "src/<testname>-test"
>>>>>
>>>>> That also works on Unix, although the extra quotes are not necessary
>>>>> there,
>>>>> so let's just do that.
>>>
>>> OK good to know. I didn't think about that when writing this patch.
>>
>> On further testing, that only works on Windows with psql version 9.4 or
>> later. Maybe we can live with that..
> Ouch OK. Is there any reason specific to psql for that?

Older versions didn't quote the command passed to CMD properly. That was
fixed in 9.4, but not backpatched, in fear that it would break people's
scripts that currently include an extra pair of quotes in the command,
to work around that. See commit
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=a692ee5870f0f442565b4c4bff367094599e9bdf

- Heikki

On Wed, Jun 18, 2014 at 9:50 PM, Heikki Linnakangas
<hlinnakangas@vmware.com> wrote:
> Older versions didn't quote the command passed to CMD properly. That was
> fixed in 9.4, but not backpatched, in fear that it would break people's
> scripts that currently include an extra pair of quotes in the command, to
> work around that. See commit
> http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=a692ee5870f0f442565b4c4bff367094599e9bdf
Ah yes, right. I recall now. That's a fix for the report Nikhil
Deshpande sent here:
http://www.postgresql.org/message-id/CAH_aGJj4YbcL7V9c_iX_YZ4WJuxgydHNvR5JgwU0K+=abeiNeg@mail.gmail.com
--
Michael