Thread: Porting to Native WindowsNT/2000
I understand that the current port of Postgres for Windows requires the cygwin package. I'd like to understand the requirement for cygwin,and possibly try to port Postgres to run natively on Windows as a NT/2K service. Anyone like to identify the challenges in such a port? Is it at all possible? Anyone else trying to do this? D
Dwayne Miller wrote: > > I understand that the current port of Postgres for Windows requires the > cygwin package. I'd like to understand the requirement for cygwin,and > possibly try to port Postgres to run natively on Windows as a NT/2K > service. Anyone like to identify the challenges in such a port? Is it > at all possible? Anyone else trying to do this? > I'm not trying to do so, but I'm not sure I would say it is possible without the the type of technology in cygwin. I have spent a lot of years writing NT drivers and programs. Unless you have a real reason why cygwin is not practical, why bother? The OS differences between NT and UNIX are huge. The main difference are processes. There is no "fork" in NT, and that is a huge gulf to cross. Is there a reason why you would not want to use cygwin? -- 5-4-3-2-1 Thunderbirds are GO! ------------------------ http://www.mohawksoft.com
Well, for one.... I have no idea what cygwin is, or what it does to your system, or what security vulnerabilities it might add to your system. It comes with alot of stuff that I may or may not need, but what components I need to run Postgres is not clear. Two.... could Postgres be made more efficient on Windows if it ran without cygwin? Three.... can you start cygwin programs on startup of the system? mlw wrote: >Dwayne Miller wrote: > >>I understand that the current port of Postgres for Windows requires the >>cygwin package. I'd like to understand the requirement for cygwin,and >>possibly try to port Postgres to run natively on Windows as a NT/2K >>service. Anyone like to identify the challenges in such a port? Is it >>at all possible? Anyone else trying to do this? >> > >I'm not trying to do so, but I'm not sure I would say it is possible without >the the type of technology in cygwin. > >I have spent a lot of years writing NT drivers and programs. Unless you have a >real reason why cygwin is not practical, why bother? > >The OS differences between NT and UNIX are huge. The main difference are >processes. There is no "fork" in NT, and that is a huge gulf to cross. Is there >a reason why you would not want to use cygwin? > >
"Dwayne Miller" <dmiller@espgroup.net> writes: > Well, for one.... I have no idea what cygwin is, or what it does to > your system, or what security vulnerabilities it might add to your > system. It comes with alot of stuff that I may or may not need, but > what components I need to run Postgres is not clear. Cygwin is a Unix environment for Windows. For information, see http://cygwin.com/ Cygwin comes with a lot of stuff which you don't need to run Postgres. Simply having that stuff on your computer will not introduce any security vulnerabilities if you don't run the programs. Cygwin is simply a DLL and a bunch of Unix programs. It has no server component. In order to build Postgres, you will need the compiler and associated tools. In order to run all the Postgres commands, you will need the shell and several of the tools. In fact, I believe that a cygwin distribution actually comes with Postgres prebuilt and ready to run. (To be honest, the idea of worrying about security vulnerabilities on Windows seems odd to me. If you are honestly worried about security on your database server, the first step is to stop running Windows.) > Two.... could Postgres be made more efficient on Windows if it ran > without cygwin? Yes. Cygwin adds measurable overhead to all I/O operations, and obviously a database does a lot of I/O. Postgres employs operations which are fast on Unix but are very slow on cygwin, such as fork. As mlw said, porting Postgres to run natively on Windows would be a significant effort. The forking mechanism it uses currently would have to be completely rearchitected. The buffer, file manager, and networking code would have to be rewritten. Off the top of my head, for a top programmer who is an expert in Unix, Windows, and Postgres, it might take a year. There would also be a heavy ongoing maintenance cost to keep up with new Postgres releases. > Three.... can you start cygwin programs on startup of the system? Sure. cygwin programs are just Windows programs which use a particular DLL. Ian
"Ian Lance Taylor" <ian@airs.com> wrote: > "Dwayne Miller" <dmiller@espgroup.net> writes: > > > Well, for one.... I have no idea what cygwin is, or what it does to > > your system, or what security vulnerabilities it might add to your > > system. It comes with alot of stuff that I may or may not need, but > > what components I need to run Postgres is not clear. > > Cygwin is a Unix environment for Windows. For information, see > http://cygwin.com/ > > Cygwin comes with a lot of stuff which you don't need to run Postgres. > Simply having that stuff on your computer will not introduce any > security vulnerabilities if you don't run the programs. Cygwin is > simply a DLL and a bunch of Unix programs. It has no server > component. > > In order to build Postgres, you will need the compiler and associated > tools. In order to run all the Postgres commands, you will need the > shell and several of the tools. > > In fact, I believe that a cygwin distribution actually comes with > Postgres prebuilt and ready to run. Yes, if you use the setup.exe at cygwin.com, it will by default include postgres. It would be nice if we had a minimal list of programs need to run Postgresql > > (To be honest, the idea of worrying about security vulnerabilities on > Windows seems odd to me. If you are honestly worried about security > on your database server, the first step is to stop running Windows.) That's just a cheap shot. I've seen no evidence that Windows NT/2000 is inherently less secure than any given Unix or Linux distribution, it is just a lot more popular and tends to have less experienced system administrators. Having an easy-to-install Windows set up would be a plus for Postgres. There are millions of Windows NT servers out there. > > > Two.... could Postgres be made more efficient on Windows if it ran > > without cygwin? > > Yes. Cygwin adds measurable overhead to all I/O operations, and > obviously a database does a lot of I/O. Postgres employs operations > which are fast on Unix but are very slow on cygwin, such as fork. > > As mlw said, porting Postgres to run natively on Windows would be a > significant effort. The forking mechanism it uses currently would > have to be completely rearchitected. This is true. However, a process-pool architecture would benefit Postgres on other platforms besides Windows. Postgresql has been ported to the HP3000 MPE/iX operating system, for example, which is POSIX-compliant, but has an awfully slow fork(). > The buffer, file manager, and > networking code would have to be rewritten. I don't think this is true. Most of the unix-style interfaces are supported out of the box by the Microsoft C compiler. > > > Three.... can you start cygwin programs on startup of the system? > > Sure. cygwin programs are just Windows programs which use a > particular DLL. It's not quite as simple as that. You can run it as a service under the SRVANY program, but that doesn't provide for a clean shut-down. Has anybody written an NT service wrapper for Postgresql?
> "Dwayne Miller" <dmiller@espgroup.net> writes: > > > Well, for one.... I have no idea what cygwin is, or what it does to > > your system, or what security vulnerabilities it might add to your > > system. It comes with alot of stuff that I may or may not need, but > > what components I need to run Postgres is not clear. > > Cygwin is a Unix environment for Windows. For information, see > http://cygwin.com/ > > Cygwin comes with a lot of stuff which you don't need to run Postgres. > Simply having that stuff on your computer will not introduce any > security vulnerabilities if you don't run the programs. Cygwin is > simply a DLL and a bunch of Unix programs. It has no server > component. > > In order to build Postgres, you will need the compiler and associated > tools. In order to run all the Postgres commands, you will need the > shell and several of the tools. > > In fact, I believe that a cygwin distribution actually comes with > Postgres prebuilt and ready to run. > > (To be honest, the idea of worrying about security vulnerabilities on > Windows seems odd to me. If you are honestly worried about security > on your database server, the first step is to stop running Windows.) > > > Two.... could Postgres be made more efficient on Windows if it ran > > without cygwin? > > Yes. Cygwin adds measurable overhead to all I/O operations, and > obviously a database does a lot of I/O. Postgres employs operations > which are fast on Unix but are very slow on cygwin, such as fork. > > As mlw said, porting Postgres to run natively on Windows would be a > significant effort. The forking mechanism it uses currently would > have to be completely rearchitected. The buffer, file manager, and > networking code would have to be rewritten. Off the top of my head, > for a top programmer who is an expert in Unix, Windows, and Postgres, > it might take a year. There would also be a heavy ongoing maintenance > cost to keep up with new Postgres releases. > > > Three.... can you start cygwin programs on startup of the system? > > Sure. cygwin programs are just Windows programs which use a > particular DLL. > > Ian > Cygrunsrv allows postgresql to be run as a service. There's a slight hiccup on shutdown meaning that the postmaster.pid file gets left. This is due to sighup being sent by windows shutdown. I think current cygwin snapshots might cure this, otherwise there is a patch some where that causes SIGHUP to be ignored. I *think* the pre-built binary already has this patch applied. - Stuart
Ian Lance Taylor wrote: > > "Dwayne Miller" <dmiller@espgroup.net> writes: > > > As mlw said, porting Postgres to run natively on Windows would be a > significant effort. The forking mechanism it uses currently would > have to be completely rearchitected. The buffer, file manager, and > networking code would have to be rewritten. Off the top of my head, > for a top programmer who is an expert in Unix, Windows, and Postgres, > it might take a year. IIRC someone had the backend working in multithreaded mode (actually he had one of the forked backends doing it) as a part of some java-driven backend. From the description of it it seemed that it did not take a full man-year to get there ;) ------------------ Hannu
"Ken Hirsch" <kenhirsch@myself.com> writes: > > (To be honest, the idea of worrying about security vulnerabilities on > > Windows seems odd to me. If you are honestly worried about security > > on your database server, the first step is to stop running Windows.) > > That's just a cheap shot. I've seen no evidence that Windows NT/2000 is > inherently less secure than any given Unix or Linux distribution, it is just > a lot more popular and tends to have less experienced system administrators. I agree that it looks like a cheap shot, but I didn't intend to make one. There are various arguments why Windows NT is probably less secure than Unix, ranging from interface design to code maturity to platform popularity to actual statistics of numbers of cracked systems and numbers of different successful cracks. I personally don't know of any arguments why Unix is less secure than Windows NT, other than guessing. Unless you are an expert in the field, which I am not, I think you should follow the preponderance of evidence, which I read as saying that where security is a significant concern, it's best to avoid Windows. (This is off-topic for the Postgres mailing list, though, so if I reply on this further I'll take it off list.) > Having an easy-to-install Windows set up would be a plus for Postgres. > There are millions of Windows NT servers out there. I agree. > > > Two.... could Postgres be made more efficient on Windows if it ran > > > without cygwin? > > > > Yes. Cygwin adds measurable overhead to all I/O operations, and > > obviously a database does a lot of I/O. Postgres employs operations > > which are fast on Unix but are very slow on cygwin, such as fork. > > > > As mlw said, porting Postgres to run natively on Windows would be a > > significant effort. The forking mechanism it uses currently would > > have to be completely rearchitected. > > This is true. However, a process-pool architecture would benefit Postgres > on other platforms besides Windows. Postgresql has been ported to the > HP3000 MPE/iX operating system, for example, which is POSIX-compliant, but > has an awfully slow fork(). On the other hand, POSIX-compliant systems generally are moving toward a faster and faster fork, as they should given the nature of POSIX programs. A process pool architecture for a system like Postgres would require very careful attention to memory usage, in order to be able to return swap space to the system or at least avoid using it. Otherwise, I believe the different processes would fragment memory over time, decreasing system performance. Process pools work best for systems with fixed memory usage. > > The buffer, file manager, and > > networking code would have to be rewritten. > > I don't think this is true. Most of the unix-style interfaces are > supported out of the box by the Microsoft C compiler. I've written code which ran natively on both Unix and Windows, and that kind of statement doesn't get you very far. Even when the interfaces are the same, there are critical differences all over the place (e.g., select() on Windows only works on sockets, not pipes). You can deal with each problem as it comes up, but they keep coming up. That's why Steve Chamberlain started the cygwin project in the first place--we were both at Cygnus at the time, and I spent several months working on cygwin myself a couple of years later. Ian
Ian Lance Taylor (& others) wrote: > > This is true. However, a process-pool architecture would benefit Postgres > > on other platforms besides Windows. Postgresql has been ported to the > > HP3000 MPE/iX operating system, for example, which is POSIX-compliant, but > > has an awfully slow fork(). > > On the other hand, POSIX-compliant systems generally are moving toward > a faster and faster fork, as they should given the nature of POSIX > programs. > > A process pool architecture for a system like Postgres would require > very careful attention to memory usage, in order to be able to return > swap space to the system or at least avoid using it. Otherwise, I > believe the different processes would fragment memory over time, > decreasing system performance. Process pools work best for systems > with fixed memory usage. What about a pre-forked model? What about using the Apache Portable Runtime? The Apache & Postgres licenses are compatible, are they not? Cheers, Colin
"Ken Hirsch" <kenhirsch@myself.com> writes:
>>> Three.... can you start cygwin programs on startup of the system?
> It's not quite as simple as that. You can run it as a service under the
> SRVANY program, but that doesn't provide for a clean shut-down. Has anybody
> written an NT service wrapper for Postgresql?
IIRC, Jason Tishler was working on one awhile back. Check the mailing
list archives.
As far as the general topic goes: this has come up several times before,
and the conclusion has always been that a native Windows port would
require effort (both initial, and ongoing maintenance) vastly out of
proportion to the reward.
But it occurs to me that it might be useful to provide a downloadable
package that includes both the Postgres server and as much of Cygwin
as you need to run it, all wrapped up in a nice friendly installer.
regards, tom lane
> -----Original Message----- > From: Tom Lane [mailto:tgl@sss.pgh.pa.us] > Sent: 04 September 2001 06:43 > To: dave Page > Subject: Re: [HACKERS] Porting to Native WindowsNT/2000 > > > I thought this might interest you. > ************************************** Thanks Tom, > "Ken Hirsch" <kenhirsch@myself.com> writes: > >>> Three.... can you start cygwin programs on startup of the system? > > > It's not quite as simple as that. You can run it as a > service under the > SRVANY program, but that doesn't provide > for a clean shut-down. Has anybody > written an NT service > wrapper for Postgresql? > > IIRC, Jason Tishler was working on one awhile back. Check > the mailing list archives. Jason and others have indeed have indeed got it running as a service using Cygwins cygrunsrv program. I'm now using this configuration for pgAdmin hacking on my laptop and it works well. > As far as the general topic goes: this has come up several > times before, and the conclusion has always been that a > native Windows port would require effort (both initial, and > ongoing maintenance) vastly out of proportion to the reward. > > But it occurs to me that it might be useful to provide a > downloadable package that includes both the Postgres server > and as much of Cygwin as you need to run it, all wrapped up > in a nice friendly installer. Jean-Michel Poure and I were discussing this yesterday and were looking into writing a plugin for pgAdmin II that will guide the users through installing minimal Cygwin with PostgreSQL & the IPC-Daemon on their system. The idea is that they download and install pgAdmin which is a simple procedure for the Windows user (== non *nix user) then run a wizard which downloads and sets up the rest for them so they end up with a working PostgreSQL, running as a service, with pgAdmin as the admin front end. We're also looking into a pg_hba.conf editor to make it easier to write and test pg_hba.conf files. Regards, Dave.