Thread: win2k, service, pg_ctl, popen, etc
Hi! Been trying to find out why things don't work well on Windows 2000. The closest I've come to a solution is that popen() does not work on windows 2000 *when running as a service*. This isused in find_other_exec(), which is what pg_ctl uses to find postmaster.exe. It does appear to work fine on XP and 2003. Haven't tested NT4. The symptoms are popen() returning NULL, with both errno and GetLastError() set to 0. Anyone know *why* this doesn't work? If we don't know why, sign one more down as a bug in mingw (hardly surprising - just the area of pipes and consoles seems to be where it's weak). In that case, why not just recode pg_ctl to execute postmaster.exe from the same directory it found pg_ctl.exe in. This can easily be done using GetModuleFileName(). Is a pipe opened at any other place? Or does other places need to be fixed as well (I haven't gone through the whole code). Oh, and also - has *anybody* got the service running wit hpg_ctl as service wrapper on windows 2000? In that case, it's version or environment dependent... Thoughts? //Magnus
Following up to myself, I found these references: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&threadm=3475 F130.80F%40us.ibm.com&rnum=1&prev=/groups%3Fq%3Dwin32%2Bpopen%2Bservice% 26hl%3Den%26lr%3D%26ie%3DUTF-8%26safe%3Doff%26selm%3D3475F130.80F%2540us .ibm.com%26rnum%3D1 http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&threadm=5f4a rb%24q5n%40chaos.dac.neu.edu&rnum=7&prev=/groups%3Fq%3Dwin32%2Bpopen%2Bs ervice%26hl%3Den%26lr%3D%26ie%3DUTF-8%26safe%3Doff%26selm%3D5f4arb%2524q 5n%2540chaos.dac.neu.edu%26rnum%3D7 Seems we need to either write our own version of popen(), or we need to do a system("whatever > somefile") and then read the file... //Magnus >-----Original Message----- > > >Hi! > >Been trying to find out why things don't work well on Windows 2000. The >closest I've come to a solution is that popen() does not work >on windows >2000 *when running as a service*. This isused in find_other_exec(), >which is what pg_ctl uses to find postmaster.exe. It does >appear to work >fine on XP and 2003. Haven't tested NT4. >The symptoms are popen() returning NULL, with both errno and >GetLastError() set to 0. > >Anyone know *why* this doesn't work? > >If we don't know why, sign one more down as a bug in mingw (hardly >surprising - just the area of pipes and consoles seems to be where it's >weak). In that case, why not just recode pg_ctl to execute >postmaster.exe from the same directory it found pg_ctl.exe in. This can >easily be done using GetModuleFileName(). Is a pipe opened at any other >place? Or does other places need to be fixed as well (I haven't gone >through the whole code). > >Oh, and also - has *anybody* got the service running wit hpg_ctl as >service wrapper on windows 2000? In that case, it's version or >environment dependent... > >Thoughts? > >//Magnus > > >---------------------------(end of >broadcast)--------------------------- >TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly >
Magnus Hagander wrote: > Hi! > > Been trying to find out why things don't work well on Windows 2000. The > closest I've come to a solution is that popen() does not work on windows > 2000 *when running as a service*. This isused in find_other_exec(), > which is what pg_ctl uses to find postmaster.exe. It does appear to work > fine on XP and 2003. Haven't tested NT4. > The symptoms are popen() returning NULL, with both errno and > GetLastError() set to 0. > > Anyone know *why* this doesn't work? > > If we don't know why, sign one more down as a bug in mingw (hardly > surprising - just the area of pipes and consoles seems to be where it's > weak). In that case, why not just recode pg_ctl to execute > postmaster.exe from the same directory it found pg_ctl.exe in. This can > easily be done using GetModuleFileName(). Is a pipe opened at any other > place? Or does other places need to be fixed as well (I haven't gone > through the whole code). > > Oh, and also - has *anybody* got the service running wit hpg_ctl as > service wrapper on windows 2000? In that case, it's version or > environment dependent... Via IM, Magnus told me the cause is that there is no stdin/stdout/stderr when running as a service: http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&threadm=3475F130.80F%40us.ibm.com&rnum=1&prev=/groups%3Fq%3Dwin32%2Bpopen%2Bservice%26hl%3Den%26lr%3D%26ie%3DUTF-8%26safe%3Doff%26selm%3D3475F130.80F%2540us.ibm.com%26rnum%3D1 http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&threadm=5f4arb%24q5n%40chaos.dac.neu.edu&rnum=7&prev=/groups%3Fq%3Dwin32%2Bpopen%2Bservice%26hl%3Den%26lr%3D%26ie%3DUTF-8%26safe%3Doff%26selm%3D5f4arb%2524q5n%2540chaos.dac.neu.edu%26rnum%3D7 The only solution I can think of is to do: system("cmd > %tmpdir%/pgsql_###.tmp") and then read 'pgsql_###.tmp'. ### would be the port number. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> If we don't know why, sign one more down as a bug in mingw (hardly > surprising - just the area of pipes and consoles seems to be > where it's weak). In that case, why not just recode pg_ctl to execute > postmaster.exe from the same directory it found pg_ctl.exe > in. This can easily be done using GetModuleFileName(). Is a pipe opened at > any other place? Or does other places need to be fixed as well (I haven't gone > through the whole code). Moreover, all the find_*_exec calls could (and IMHO, should) make use of GetModuleFileName. Win32 has an API to find the running file name; *nix does not and so we have to jump through a few hoops. I just don't see the sense in jumping through the same hoops under Win32. Sure, the functions also make certain that the version is as expected, but to my mind this is just a safety check to make sure the hoops we just jumped through were all in order. Of course, we'd also lose the ability to protect people who mix versions in the same directory, but such people need a lot more protection from themselves than we can provide. ;-) Does there continue to be any resistance to this approach? If not, I'll gladly provide a patch. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see <a href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em ailpolicy.html</a>
> > If we don't know why, sign one more down as a bug in mingw (hardly > > surprising - just the area of pipes and consoles seems to be where > > it's weak). In that case, why not just recode pg_ctl to execute > > postmaster.exe from the same directory it found pg_ctl.exe in. This > > can easily be done using GetModuleFileName(). Is a pipe opened at > > > any other place? Or does other places need to be fixed as well (I > > haven't > gone > > through the whole code). > > Moreover, all the find_*_exec calls could (and IMHO, should) > make use of GetModuleFileName. Win32 has an API to find the > running file name; *nix does not and so we have to jump > through a few hoops. I just don't see the sense in jumping > through the same hoops under Win32. mmm, I agree. I think the talk back then was "if it works, do it the same way on Unix". In this case it clearly does not work. If we need to do a workaround, this sounds like the correct fix - rather than just patching a method that's made for a different platform. > Does there continue to be any resistance to this approach? If > not, I'll gladly provide a patch. None from me at least :-) //Magnus
Claudio Natoli wrote: >>If we don't know why, sign one more down as a bug in mingw (hardly >>surprising - just the area of pipes and consoles seems to be >>where it's weak). In that case, why not just recode pg_ctl to execute >>postmaster.exe from the same directory it found pg_ctl.exe >>in. This can easily be done using GetModuleFileName(). Is a pipe opened at >> >> > > > >>any other place? Or does other places need to be fixed as well (I haven't >> >> >gone > > >>through the whole code). >> >> > >Moreover, all the find_*_exec calls could (and IMHO, should) make use of >GetModuleFileName. Win32 has an API to find the running file name; *nix does >not and so we have to jump through a few hoops. I just don't see the sense >in jumping through the same hoops under Win32. > >Sure, the functions also make certain that the version is as expected, but >to my mind this is just a safety check to make sure the hoops we just jumped >through were all in order. Of course, we'd also lose the ability to protect >people who mix versions in the same directory, but such people need a lot >more protection from themselves than we can provide. ;-) > >Does there continue to be any resistance to this approach? If not, I'll >gladly provide a patch. > > > > As I understand it, Windows has a standard set of DLL/EXE metadata (build number, copyright, product name yadda yadda yadda) stored in some well-known segment of the file. Is there any reason we can't put the version number somewhere in there and use some standard API to extract it, rather than running the .exe to make it tell us? (Not that I have any idea how to do such a thing.) cheers andrew
> >Does there continue to be any resistance to this approach? > If not, I'll > >gladly provide a patch. > > > > > > > > > > As I understand it, Windows has a standard set of DLL/EXE > metadata (build number, copyright, product name yadda yadda > yadda) stored in some well-known segment of the file. Is > there any reason we can't put the version number somewhere in > there and use some standard API to extract it, rather than > running the .exe to make it tell us? (Not that I have any > idea how to do such a thing.) We could, see VerQueryValue() and friends on MSDN. It *requires* that the version number *always* follows the pattern a.b.c.d, where each of a-d is a 32 bit unsigned int. Meaning there is no way to determine RCs etc, unless we start tracking build numbers in some way. While I think the whole checking of the version there is somewhat of an overkill (again, if you mix binary versions, you are probably going to mess up a whole lot of other things too), it is probably not a bad idea to add version numbering to the binaries *anyway*... //Magnus
Andrew Dunstan wrote: > > As I understand it, Windows has a standard set of DLL/EXE metadata > (build number, copyright, product name yadda yadda yadda) stored in some > well-known segment of the file. Is there any reason we can't put the > version number somewhere in there and use some standard API to extract > it, rather than running the .exe to make it tell us? (Not that I have > any idea how to do such a thing.) It's the VERSION resource, look at GetFileVersionInfo in the docs. Regards, Andreas
Claudio Natoli wrote: > > > If we don't know why, sign one more down as a bug in mingw (hardly > > surprising - just the area of pipes and consoles seems to be > > where it's weak). In that case, why not just recode pg_ctl to execute > > postmaster.exe from the same directory it found pg_ctl.exe > > in. This can easily be done using GetModuleFileName(). Is a pipe opened at > > > any other place? Or does other places need to be fixed as well (I haven't > gone > > through the whole code). > > Moreover, all the find_*_exec calls could (and IMHO, should) make use of > GetModuleFileName. Win32 has an API to find the running file name; *nix does > not and so we have to jump through a few hoops. I just don't see the sense > in jumping through the same hoops under Win32. > > Sure, the functions also make certain that the version is as expected, but > to my mind this is just a safety check to make sure the hoops we just jumped > through were all in order. Of course, we'd also lose the ability to protect > people who mix versions in the same directory, but such people need a lot > more protection from themselves than we can provide. ;-) > > Does there continue to be any resistance to this approach? If not, I'll > gladly provide a patch. GetModuleFileName() doesn't solve our problem here. We already know what we think is the right path, but we have to be sure that we are calling a matching version binary. For example, initdb wants to call a postgres that is the right version, as does pg_ctl. We have had mismatches reported in the past, and with relocatable installs it is even more likely. The version check is centralized in one place, exec.c, and I don't want to have different behaviors on Win32 and Unix if we can help it. Let's go with the system() into a temp file. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> > As I understand it, Windows has a standard set of DLL/EXE metadata > > (build number, copyright, product name yadda yadda > > yadda) stored in some well-known segment of the file. Is there any > > reason we can't put the version number somewhere in there > and use some > > standard API to extract it, rather than running the .exe to make it > > tell us? (Not that I have any idea how to do such a thing.) > > We could, see VerQueryValue() and friends on MSDN. > It *requires* that the version number *always* follows the > pattern a.b.c.d, where each of a-d is a 32 bit unsigned int. > Meaning there is no way to determine RCs etc, unless we start > tracking build numbers in some way. > Actually, if we want the string, we could always create a string table resource and store it in that. And then load it with LoadString(). We just need to be sure we store the exact same thing there, but a string resource will support any null terminated string. //Magnus
Bruce Momjian wrote: > The version check is centralized in one place, exec.c, and I don't want > to have different behaviors on Win32 and Unix if we can help it. Let's > go with the system() into a temp file. You know that system() isn't executed synchronously under win32, don't you? Version resource checking seems much more suitable. Additionally, more or less every win32 executable or dll should have the version resource. IMHO a missing resource is an indicator of a win32 novice programmer. Regards, Andreas
> -----Original Message----- > From: pgsql-hackers-win32-owner@postgresql.org > [mailto:pgsql-hackers-win32-owner@postgresql.org] On Behalf > Of Magnus Hagander > Sent: 21 July 2004 15:32 > To: Andrew Dunstan; pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] win2k, service, pg_ctl, popen, etc > > Actually, if we want the string, we could always create a string table > resource and store it in that. And then load it with LoadString(). We > just need to be sure we store the exact same thing there, but a string > resource will support any null terminated string. Alternatively we can just use one of the other fields in the version resource such as ProductVersion. /D
Bruce Momjian wrote: >The version check is centralized in one place, exec.c, and I don't want >to have different behaviors on Win32 and Unix if we can help it. Let's >go with the system() into a temp file. > > > I had another thought - would the popen work for a nonconsole app (e.g. a service) if we redirect stdin from the nul device (and maybe also do stderr via 2>&1), so it doesn't try to inherit from the caller, which doesn't have these? cheers andrew
Dave Page wrote: > > > > >>-----Original Message----- >>From: pgsql-hackers-win32-owner@postgresql.org >>[mailto:pgsql-hackers-win32-owner@postgresql.org] On Behalf >>Of Magnus Hagander >>Sent: 21 July 2004 15:32 >>To: Andrew Dunstan; pgsql-hackers-win32@postgresql.org >>Subject: Re: [pgsql-hackers-win32] win2k, service, pg_ctl, popen, etc >> >>Actually, if we want the string, we could always create a string table >>resource and store it in that. And then load it with LoadString(). We >>just need to be sure we store the exact same thing there, but a string >>resource will support any null terminated string. >> >> > >Alternatively we can just use one of the other fields in the version >resource such as ProductVersion. > > > > My reading suggests you can actually put a string with an arbitrary name in this resource. cheers andrew
> -----Original Message----- > From: pgsql-hackers-win32-owner@postgresql.org > [mailto:pgsql-hackers-win32-owner@postgresql.org] On Behalf > Of Andrew Dunstan > Sent: 21 July 2004 16:25 > To: pgsql-hackers-win32@postgresql.org > Subject: Re: [pgsql-hackers-win32] win2k, service, pg_ctl, popen, etc > > > > > My reading suggests you can actually put a string with an > arbitrary name in this resource. That's right, you can have anything you like. ProductVersion is one of the standard ones though, and strikes me as the perfect place for the PostgreSQL version string. We should add it to *all* exes really - as Andreas pointed out, no exe should be without a version resource. http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winui/W inUI/WindowsUserInterface/Resources/VersionInformation/VersionInformatio nReference/VersionInformationStructures/String.asp Regards, Dave
Dave Page wrote: > >> > > Alternatively we can just use one of the other fields in the version > resource such as ProductVersion. We're not restricted to the predefined fields, we can invent a "PostgreSQL VERSION" as well. This will show up if "file properties" are selected from the windows explorer, string resources won't. Regards, Andreas
> GetModuleFileName() doesn't solve our problem here. We already know > what we think is the right path, but we have to be sure that we are > calling a matching version binary. For example, initdb wants > to call a postgres that is the right version, as does pg_ctl. But do we really care about people who put 7.5 initdb.exe in the same directory as, say, 7.6 postgres.exe? > We have had mismatches reported in the past, and with relocatable installs it is > even more likely. Exactly my point. The mismatches occur because we *think* we know the right path under *nix, and so we grab the version string to make certain. Under win32, there would be no such ambiguity, except if people mix versions in the same directory; which certainly doesn't appear worth the cost to protect against. > The version check is centralized in one place, exec.c, and I don't want > to have different behaviors on Win32 and Unix if we can help it. Let's > go with the system() into a temp file. Ok; please allow me to continue to respectfully disagree that this is justified :-) Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see <a href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em ailpolicy.html</a>
Claudio Natoli wrote: > > > > GetModuleFileName() doesn't solve our problem here. We already know > > what we think is the right path, but we have to be sure that we are > > calling a matching version binary. For example, initdb wants > > to call a postgres that is the right version, as does pg_ctl. > > But do we really care about people who put 7.5 initdb.exe in the same > directory as, say, 7.6 postgres.exe? > > > > We have had mismatches reported in the past, and with relocatable installs > it is > > even more likely. > > Exactly my point. The mismatches occur because we *think* we know the right > path under *nix, and so we grab the version string to make certain. Under > win32, there would be no such ambiguity, except if people mix versions in > the same directory; which certainly doesn't appear worth the cost to protect > against. Have you looked at find_other_exec(). We call find_my_exec(), then assuming that works, we go ahead and check versions. We don't loop back and keep poking around until we find a matching version. This code was added particularly so relocable installs don't get wacked out, and you can bet Win32 will wack it out even more than unix folks. We need the check. If we have to put some custom Win32 code in, or use system() or whatever, we need it. I am not willing to loosen that check for Win32 --- in fact I would like that check especially for Win32 where users are less experienced. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
> Have you looked at find_other_exec(). We call find_my_exec(), then Yes. IIRC, I assisted you with the recent refactoring of exec.c. > assuming that works, we go ahead and check versions. We don't loop back > and keep poking around until we find a matching version. > > This code was added particularly so relocable installs don't get wacked > out, and you can bet Win32 will wack it out even more than unix folks. GetMFN used in find_my_exec will continue to allow relocatable installs. What it does is remove any ambiguity over having found the correct location of the running binary. In this case, all that the version check then achieves (other than consistency between *nix and Win32) is to make sure that people don't mix different versions of PostgreSQL binaries in the same directory, which IMHO seems like overkill. > I am not willing to loosen that check for Win32 --- in fact I > would like that check especially for Win32 where users are less experienced. That is clear. I lack any concern for users who mix PostgreSQL binaries of different versions in the same directory, but you maintain concern. We simply disagree. Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see <a href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em ailpolicy.html</a>
Claudio Natoli wrote:
>
> > Have you looked at find_other_exec(). We call find_my_exec(), then
>
> Yes. IIRC, I assisted you with the recent refactoring of exec.c.
>
>
> > assuming that works, we go ahead and check versions. We don't loop back
> > and keep poking around until we find a matching version.
> >
> > This code was added particularly so relocable installs don't get wacked
> > out, and you can bet Win32 will wack it out even more than unix folks.
>
> GetMFN used in find_my_exec will continue to allow relocatable installs.
> What it does is remove any ambiguity over having found the correct location
> of the running binary. In this case, all that the version check then
> achieves (other than consistency between *nix and Win32) is to make sure
> that people don't mix different versions of PostgreSQL binaries in the same
> directory, which IMHO seems like overkill.
>
>
> > I am not willing to loosen that check for Win32 --- in fact I
> > would like that check especially for Win32 where users are less
> experienced.
>
> That is clear. I lack any concern for users who mix PostgreSQL binaries of
> different versions in the same directory, but you maintain concern. We
> simply disagree.
We have received several bug reports of binary mismatches so tightening
it seems logical, and we are doing it in all cases:
pg_ctl finding postmaster
postmaster finding postgres
initdb finding postgres
pg_dumpall finding pg_dump
It has _nothing_ to do with any weakness in finding our own binary path
under Unix.
It all happens in exec.c. The test is simple in Unix. If it isn't
simple in Win32, then we need to work harder to make it happen.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073
> It all happens in exec.c. The test is simple in Unix. If it isn't > simple in Win32, then we need to work harder to make it happen. Are we amenable to replacing this popen call with one to CreateProcess with i/o handles redirected (as per the STARTUPINFO argument), or do we want to concentrate on a common *nix/win32 solution? Cheers, Claudio --- Certain disclaimers and policies apply to all email sent from Memetrics. For the full text of these disclaimers and policies see <a href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em ailpolicy.html</a>
Claudio Natoli wrote: > > > It all happens in exec.c. The test is simple in Unix. If it isn't > > simple in Win32, then we need to work harder to make it happen. > > Are we amenable to replacing this popen call with one to CreateProcess with > i/o handles redirected (as per the STARTUPINFO argument), or do we want to > concentrate on a common *nix/win32 solution? I think CreateProcess() is fine. I can't think of another way to do it. At least it will be isolated in exec.c. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073