Thread: Re: Roadmap for a Win32 port

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> Sent: Tuesday, June 04, 2002 9:34 PM
> To: PostgreSQL-development
> Subject: [HACKERS] Roadmap for a Win32 port
>
>
> OK, I think I am now caught up on the Win32/cygwin
> discussion, and would
> like to make some remarks.
>
> First, are we doing enough to support the Win32 platform?  I think the
> answer is clearly "no".  There are 3-5 groups/companies
> working on Win32
> ports of PostgreSQL.  We always said there would not be
> PostgreSQL forks
> if we were doing our job to meet user needs.  Well,
> obviously, a number
> of groups see a need for a better Win32 port and we aren't
> meeting that
> need, so they are.  I believe this is one of the few cases
> where groups
> are going out on their own because we are falling behind.
>
> So, there is no question in my mind we need to do more to encourage
> Win32 ports.  Now, on to the details.
>
> INSTALLER
> ---------
>
> We clearly need an installer that is zero-hassle for users.
> We need to
> decide on a direction for this.
>
> GUI
> ---
>
> We need a slick GUI.  pgadmin2 seems to be everyone's favorite, with
> pgaccess on Win32 also an option.  What else do we need here?

Nothing else.  It is better than any commercial tools in current use.
An excellent piece of work.
> BINARY
> ------
>
> This is the big daddy.   It is broken down into several sections:
>
> FORK()
>
> How do we handle fork()?  Do we use the cygwin method that copies the
> whole data segment, or put the global data in shared memory and copy
> that small part manually after we create a new process?

Do not try to do a fork() on Win32.  The one at PW32 is better, but
still awful.  Win32 just does not have fascilities for fork().

If you use Cygwin, it will kill the project for commercial use (at least
for many institutions).  That's fine, but it will become an academic
exercise instead of a viable commercial tool.  If they are comfortable
in that [Cygwin] environment, it makes no sense to use Cygwin instead of
Redhat.  The Redhat version will fork() 100 times faster.  After all, if
they are going to use unix tools in a unix interface with Unix scripts
you might as well use UNIX.  And Cygwin requires a license for
commercial use.
http://cygwin.com/licensing.html
> THREADING
>
> Related to fork(), do we implement an optionally threaded postmaster,
> which eliminates CreateProcess() entirely?  I don't think we will have
> superior performance on Win32 without it.  (This would greatly help
> Solaris as well.)

CreateProcess() works well for Win32.  That is the approach that we used
and also the approach used by the Japanese team.
It is very simple.  Simply do a create process call and then perform the
same operations that were done up to that point.  It isn't difficult.
Threading is another possibility.  I think create process is better,
because you can clone the rights of the one who attaches for the spawned
server (if you want to do that).

>
> IPC
>
> We can use Cygwin, MinGW, Apache, or our own code for this. Are there
> other options?

We wrote our own from scratch.  Cygwin will kill it.  If there is a
MinGW version it might be OK, but if MinGW is GPL, that will kill it.
Have a look at ACE:
http://www.cs.wustl.edu/~schmidt/ACE.html
Their license is on the same level as a BSD license.  Now, they use C++,
but you can always write:
extern "C" {
}
wrappers for stuff and keep PostgreSQL itself in pure, vanilla C.  GCC
does come with a C++ compiler, so it isn't going to cut anyone off.
> ENVIRONMENT
>
> Lots of our code requires a Unix shell and utilities.  Will
> we continue
> using cygwin for this?

We wrote our own utilities from scratch (e.g. initdb).  The Japanese
group that did the port did the same thing.
> --------------------------------------------------------------
> -------------
>
> As a roadmap, it would be good to get consensus on as many of these
> items as possible so people can start working in these areas.  We can
> keep a web page of decisions we have made to help rally developers to
> the project.

If you want a roadmap, the Japanese group laid it out for you.   They
did the exact same steps as we did.  Now, I don't know if we will be
able to contribute or not (it is very much up in the air).  And we had
to do a lot of hacking of the source, so you might not want it if we
volunteered.

Suggestion:
Ask the Japanese group if they would like to post their changes back or
expose them so that the programming team can get ideas form it.

I actually like what they did better than what we did (A giant DLL and
all the binaries are microscopic -- it was how I suggested to do it here
but it was vetoed).

Anyway, here is a roadmap laid out for you exactly.  Just do what it
says and you will be fine:
http://hp.vector.co.jp/authors/VA023283/PostgreSQLe.html

Look at where it says "Gists for patch" and do that.

Dan,

The following is to help keep the archives accurate and should not be
construed as an argument against the native Win32 port.

On Tue, Jun 04, 2002 at 10:02:14PM -0700, Dann Corbit wrote:
> And Cygwin requires a license for commercial use.
> http://cygwin.com/licensing.html

The above is not necessarily true:
   Red Hat sells a special Cygwin License for customers who are unable   to provide their application in open source
codeform.
 

Note that the above only comes into play if your application links
with the Cygwin DLL.  This is easily avoidable by using JDBC, ODBC,
Win32 libpq, etc.  Hence, most people will not be required to purchase
this license from Red Hat.

Jason

Jason Tishler wrote:
> Dan,
> 
> The following is to help keep the archives accurate and should not be
> construed as an argument against the native Win32 port.
> 
> On Tue, Jun 04, 2002 at 10:02:14PM -0700, Dann Corbit wrote:
> > And Cygwin requires a license for commercial use.
> > http://cygwin.com/licensing.html
> 
> The above is not necessarily true:
> 
>     Red Hat sells a special Cygwin License for customers who are unable
>     to provide their application in open source code form.
> 
> Note that the above only comes into play if your application links
> with the Cygwin DLL.  This is easily avoidable by using JDBC, ODBC,
> Win32 libpq, etc.  Hence, most people will not be required to purchase
> this license from Red Hat.

So apps written using client libraries are BSD, while server-side
changes would have to release source.  Makes sense, though we have never
had this distinction before.  I assume plpgsql stored procedures would
have be open source, but of course those are stored in plaintext on the
server so that isn't a problem.  If companies created custom C stored
procedures, those would have to be open source if using cygwin.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

> -----Original Message-----
> From: Thomas Lockhart [mailto:thomas@fourpalms.org]
> Sent: Wednesday, June 05, 2002 3:03 PM
> To: Bruce Momjian
> Cc: Igor Kovalenko; PostgreSQL-development
> Subject: Re: [HACKERS] Roadmap for a Win32 port
>
>
> ...
> > Good summary.  I think we would support both threaded and fork()
> > operation, and users can control which they prefer.  For a
> web backend
> > where many sessions are a single query, people may want to
> give up the
> > stability of fork() and go with threads, even on Unix.
>
> I would think that we would build on our strengths of having
> a fork/exec
> model for separate clients. A threaded model *could* benefit
> individual
> clients who are doing queries on multiprocessor servers, and
> I would be
> supportive of efforts to enable that.
>
> But the requirements for that may be less severe than for managing
> multiple clients within the same process, and imho there is not strong
> requirement to enable the latter for our current crop of well
> supported
> targets. If it came for free then great, but if it came with
> a high cost
> then the choice is not as obvious. It is also not a
> *requirement* if we
> were instead able to do the multiple threads for a single client
> scenerio first.

Notion:
Have one version do both.  Your server can fork(), and your sever can
thread.  It can fork() and thread, it can fork() or thread.

That gives the best of all worlds.  One client who has his attachments
to a database all setup might want to do a bunch of similar queries.
Hence a threaded model is nice.

A server may be set up to clone the rights of the attaching process for
security reasons.  Then you launch a new server with fork().

...
> Notion:
> Have one version do both.  Your server can fork(), and your sever can
> thread.  It can fork() and thread, it can fork() or thread.
> That gives the best of all worlds.  One client who has his attachments
> to a database all setup might want to do a bunch of similar queries.
> Hence a threaded model is nice.
> A server may be set up to clone the rights of the attaching process for
> security reasons.  Then you launch a new server with fork().

Right. If/when that is possible then let's do it, as long as the cost is
not too high. But the intermediate steps are a possibility also, and are
not precluded from discussion.

This will all work out as a *convergence* of interests imho. And there
is no great identifiable benefit for our current crop of platforms for
going to a threaded model *unless* that enables queries for a single
client to execute in parallel (all imho of course ;). 

So our convergence of interests for all platforms is in enabling
threading for these two purposes, and focusing on enabling the
multithreaded single client *first* means that the current crop of
clients don't have to accept all negatives while we start on the road to
better support of Win32 machines.
                 - Thomas

> -----Original Message-----
> From: Steve Howe [mailto:howe@carcass.dhs.org]
> Sent: 06 June 2002 02:37
> To: Bruce Momjian
> Cc: PostgreSQL-development
> Subject: Re: Roadmap for a Win32 port
>
>
> Hello Bruce,
>
> Wednesday, June 5, 2002, 1:33:44 AM, you wrote:
>
> BM> INSTALLER
> BM> ---------
>
> BM> We clearly need an installer that is zero-hassle for
> users.  We need
> BM> to decide on a direction for this.
> I suggest Nullsoft install system
> (http://www.nullsoft.com/free/nsis/). It's > real good and very
> simple to use. I can help on this if you want.

I think that a Windows Installer compatible package would be better as
it would allow us to build the package as a merge module which others
could use in their installers for their PostgreSQL based apps, allowing
one installation to install everything they require easily and (more
importantly) correctly. An example of this can be found in the psqlODBC
installer.

I can handle this if required.

> BM> ENVIRONMENT
>
> I also would like to empathize that probably a small GUI for
> controlling the PostgreSQL service/application would be nice.

I'm happy to add such code to pgAdmin - seems like the natural thing to
do (to me at least!).

Regards, Dave.

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> Sent: 08 June 2002 22:48
> To: Peter Eisentraut
> Cc: PostgreSQL-development
> Subject: Re: Roadmap for a Win32 port
>
>
> >
> > > Also, it seems Win32 doesn't need these scripts, except initdb.
> >
> > The utility of these programs is independent of the
> platform.  If we
> > think pg_dumpall is not useful, then let's remove it.
>
> I think the first two targets for C-ification would be
> pg_dumpall and initdb.  The others have SQL equivalents.
> Maybe pg_ctl too.

I looked at this issue some time ago & came to the conclusion that the
only scripts that Win32 really needed were pg_dumpall, initdb &
initlocation.

The others have SQL equivalents as you say, apart from pg_ctl which
under Windows should probably (and generally is) be replaced by the SCM
(Service Control Manager). The only thing that comes to mind that the
SCM can't do is a reload.

Regards, Dave.

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> Sent: Monday, June 17, 2002 6:01 PM
> To: Jan Wieck
> Cc: Peter Eisentraut; PostgreSQL-development
> Subject: Re: [HACKERS] Roadmap for a Win32 port
>
>
> Jan Wieck wrote:
> > > And pg_ctl will be run with a symlink to postmaster like postgres,
> > > right?  Makes sense.
> >
> > No symlink. Windows doesn't have symlinks, the "link" stuff you
> > see is just some file with a special meaning for the Windows
> > explorer. There is absolutely no support built into the OS. They
> > really haven't learned alot since the DOS times, when they added
> > "." and ".." entries to directories to "look" similar to UNIX.
> > Actually, they never really understood what a hardlink is in the
> > first place, so why do you expect them to know how to implement
> > symbolic ones?
> >
> > It will be at least another copy of the postmaster (dot exe).
>
> Yea, I just liked the idea of the postmaster binary somehow reporting
> the postmaster status.  Seems it is in a better position to
> do that than
> a shell script.

Architectural notion:
The Postmaster is about 100x bigger than it needs to be.

The Postmaster needs to set up shared memory and launch servers.  It
does not need to know anything about SQL grammar or any of that
rigamarole.

It could be a 15K executable.

Why not have an itty-bitty Postmaster that does nothing but a spawn or a
create process to create threaded Postgres instances?

Just a notion.

Dann Corbit wrote:
> > > It will be at least another copy of the postmaster (dot exe).
> > 
> > Yea, I just liked the idea of the postmaster binary somehow reporting
> > the postmaster status.  Seems it is in a better position to 
> > do that than
> > a shell script.
> 
> Architectural notion:
> The Postmaster is about 100x bigger than it needs to be.
> 
> The Postmaster needs to set up shared memory and launch servers.  It
> does not need to know anything about SQL grammar or any of that
> rigamarole.
> 
> It could be a 15K executable.
> 
> Why not have an itty-bitty Postmaster that does nothing but a spawn or a
> create process to create threaded Postgres instances?

Can't.  postmaster/postgres are symlinks to the same file, and we fork()
from postmaster to create backends.  All the code has to be in the
postmaster so the fork works.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> Sent: Monday, June 17, 2002 6:20 PM
> To: Dann Corbit
> Cc: Jan Wieck; Peter Eisentraut; PostgreSQL-development
> Subject: Re: [HACKERS] Roadmap for a Win32 port
>
>
> Dann Corbit wrote:
> > > > It will be at least another copy of the postmaster (dot exe).
> > >
> > > Yea, I just liked the idea of the postmaster binary
> somehow reporting
> > > the postmaster status.  Seems it is in a better position to
> > > do that than
> > > a shell script.
> >
> > Architectural notion:
> > The Postmaster is about 100x bigger than it needs to be.
> >
> > The Postmaster needs to set up shared memory and launch servers.  It
> > does not need to know anything about SQL grammar or any of that
> > rigamarole.
> >
> > It could be a 15K executable.
> >
> > Why not have an itty-bitty Postmaster that does nothing but
> a spawn or a
> > create process to create threaded Postgres instances?
>
> Can't.  postmaster/postgres are symlinks to the same file,
> and we fork()
> from postmaster to create backends.  All the code has to be in the
> postmaster so the fork works.

Is fork() faster than creation of a new process via exec()?  After the
creation of the shared memory, the information needed to use it could be
passed to the Postgres servers on the command line.

The startup stuff for PostgreSQL is just a few files.  It does not seem
insurmountable to change it.  But it is none of my business.  If it is a
major hassle (for reasons which I am not aware) then I see no driving
reason to change it.

Dann Corbit wrote:
> > Can't.  postmaster/postgres are symlinks to the same file, 
> > and we fork()
> > from postmaster to create backends.  All the code has to be in the
> > postmaster so the fork works.
> 
> Is fork() faster than creation of a new process via exec()?  After the
> creation of the shared memory, the information needed to use it could be
> passed to the Postgres servers on the command line.
> 
> The startup stuff for PostgreSQL is just a few files.  It does not seem
> insurmountable to change it.  But it is none of my business.  If it is a
> major hassle (for reasons which I am not aware) then I see no driving
> reason to change it.

We used to fork() and exec(), but that was slow.  Now we preload stuff
in the postmaster for each backend.  It is faster.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

Dann Corbit wrote:
> 
> > -----Original Message-----
> > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> > Sent: Monday, June 17, 2002 6:20 PM
> > To: Dann Corbit
> > Cc: Jan Wieck; Peter Eisentraut; PostgreSQL-development
> > Subject: Re: [HACKERS] Roadmap for a Win32 port
> >
> >
> > Dann Corbit wrote:
> > > > > It will be at least another copy of the postmaster (dot exe).
> > > >
> > > > Yea, I just liked the idea of the postmaster binary
> > somehow reporting
> > > > the postmaster status.  Seems it is in a better position to
> > > > do that than
> > > > a shell script.
> > >
> > > Architectural notion:
> > > The Postmaster is about 100x bigger than it needs to be.
> > >
> > > The Postmaster needs to set up shared memory and launch servers.  It
> > > does not need to know anything about SQL grammar or any of that
> > > rigamarole.
> > >
> > > It could be a 15K executable.
> > >
> > > Why not have an itty-bitty Postmaster that does nothing but
> > a spawn or a
> > > create process to create threaded Postgres instances?
> >
> > Can't.  postmaster/postgres are symlinks to the same file,
> > and we fork()
> > from postmaster to create backends.  All the code has to be in the
> > postmaster so the fork works.
> 
> Is fork() faster than creation of a new process via exec()?  After the
> creation of the shared memory, the information needed to use it could be
> passed to the Postgres servers on the command line.

exec() does NOT create new processes. It loads another executable file
into the existing, calling process. 

fork() duplicates the calling process. In modern unix variants, this is
done in a very efficient way, so that the text segment (program code) is
shared readonly and everything else (data and stack segments) are shared
copy on write. Thus, fork() itself doesn't even cause memory copying.
That happens later when one of the now two processes writes to a memory
page the first time.

Windows does not have these two separate steps. It wants the full blown
expensive "create process and load executable", or the "let's all muck
around with the same handles" modell, called threading. 

> 
> The startup stuff for PostgreSQL is just a few files.  It does not seem
> insurmountable to change it.  But it is none of my business.  If it is a
> major hassle (for reasons which I am not aware) then I see no driving
> reason to change it.

It has to be changed for Windows, it is a major hassle for reasons I
wasn't aware of, and I am half way through ;-)


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

On Tue, 2002-06-18 at 09:07, Jan Wieck wrote:
> Dann Corbit wrote:
> >
> > The startup stuff for PostgreSQL is just a few files.  It does not seem
> > insurmountable to change it.  But it is none of my business.  If it is a
> > major hassle (for reasons which I am not aware) then I see no driving
> > reason to change it.
>
> It has to be changed for Windows, it is a major hassle for reasons I
> wasn't aware of, and I am half way through ;-)
>

Well, if you're going to go through the trouble of rewriting postmaster
to be win32 specific, you might as well make it hook into the services
control panel.

If I recall, shared memory is "owned" by a process in Win32 (please
correct as needed...as I've slept since I last looked).  That means that
the postmaster process not only owns the shared memory but needs to make
sure that it's persists as the rest of postgres is expecting.

Please provide more details as to the nature of your Win32 changes.  I'm
certainly curious.  If you've already covered them, just say so...I have
no problem going back to the archives! :)

Greg

Hello,

I am new to PostgreSQL, but I am interested in the Win32 port.
I have studied the architecture of other databases like Oracle.

They have had to turn their multi-process model used on Unix into a fully
multi-threaded one on Win32. I have the feeling that they have had the same
debate that the one you have.

The CreateProcess() syscall is very costly on Windows. Some improvements
have been done in Windows XP but it is still far more costly than a Unix
fork().

I have been programming with threads on NT for a long time now.
They are quiet robust and efficient. I fear that it is the only successful
way to port PostgreSQL.

Sorry for this interruption,
Serge

-----Original Message-----
From: Jan Wieck [mailto:JanWieck@Yahoo.com] 
Sent: Tuesday, June 18, 2002 16:07
To: Dann Corbit
Cc: Bruce Momjian; Peter Eisentraut; PostgreSQL-development
Subject: Re: [HACKERS] Roadmap for a Win32 port

Dann Corbit wrote:
> 
> > -----Original Message-----
> > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> > Sent: Monday, June 17, 2002 6:20 PM
> > To: Dann Corbit
> > Cc: Jan Wieck; Peter Eisentraut; PostgreSQL-development
> > Subject: Re: [HACKERS] Roadmap for a Win32 port
> >
> >
> > Dann Corbit wrote:
> > > > > It will be at least another copy of the postmaster (dot exe).
> > > >
> > > > Yea, I just liked the idea of the postmaster binary
> > somehow reporting
> > > > the postmaster status.  Seems it is in a better position to
> > > > do that than
> > > > a shell script.
> > >
> > > Architectural notion:
> > > The Postmaster is about 100x bigger than it needs to be.
> > >
> > > The Postmaster needs to set up shared memory and launch servers.  It
> > > does not need to know anything about SQL grammar or any of that
> > > rigamarole.
> > >
> > > It could be a 15K executable.
> > >
> > > Why not have an itty-bitty Postmaster that does nothing but
> > a spawn or a
> > > create process to create threaded Postgres instances?
> >
> > Can't.  postmaster/postgres are symlinks to the same file,
> > and we fork()
> > from postmaster to create backends.  All the code has to be in the
> > postmaster so the fork works.
> 
> Is fork() faster than creation of a new process via exec()?  After the
> creation of the shared memory, the information needed to use it could be
> passed to the Postgres servers on the command line.

exec() does NOT create new processes. It loads another executable file
into the existing, calling process. 

fork() duplicates the calling process. In modern unix variants, this is
done in a very efficient way, so that the text segment (program code) is
shared readonly and everything else (data and stack segments) are shared
copy on write. Thus, fork() itself doesn't even cause memory copying.
That happens later when one of the now two processes writes to a memory
page the first time.

Windows does not have these two separate steps. It wants the full blown
expensive "create process and load executable", or the "let's all muck
around with the same handles" modell, called threading. 

> 
> The startup stuff for PostgreSQL is just a few files.  It does not seem
> insurmountable to change it.  But it is none of my business.  If it is a
> major hassle (for reasons which I am not aware) then I see no driving
> reason to change it.

It has to be changed for Windows, it is a major hassle for reasons I
wasn't aware of, and I am half way through ;-)


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

I know that Apache Group created special library to handle difference
between different platforms (including win32). They had similar problems
porting Apache to Windows. They build very portable threads api (win32,
POSIX, native Linux thread and more) There is also all IPC stuff (mutex,
signals mmap etc.) and many more. This functions work both on unix and
windows and use most effective implementation (e.g. POSIX functions on
Winodws are slow compared to native).

http://apr.apache.org/



-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Serge Adda
Sent: Tuesday, June 18, 2002 6:43 PM
To: 'Jan Wieck'; 'Dann Corbit'
Cc: 'Bruce Momjian'; 'Peter Eisentraut'; 'PostgreSQL-development'
Subject: Re: [HACKERS] Roadmap for a Win32 port

Hello,

I am new to PostgreSQL, but I am interested in the Win32 port.
I have studied the architecture of other databases like Oracle.

They have had to turn their multi-process model used on Unix into a
fully
multi-threaded one on Win32. I have the feeling that they have had the
same
debate that the one you have.

The CreateProcess() syscall is very costly on Windows. Some improvements
have been done in Windows XP but it is still far more costly than a Unix
fork().

I have been programming with threads on NT for a long time now.
They are quiet robust and efficient. I fear that it is the only
successful
way to port PostgreSQL.

Sorry for this interruption,
Serge

-----Original Message-----
From: Jan Wieck [mailto:JanWieck@Yahoo.com] 
Sent: Tuesday, June 18, 2002 16:07
To: Dann Corbit
Cc: Bruce Momjian; Peter Eisentraut; PostgreSQL-development
Subject: Re: [HACKERS] Roadmap for a Win32 port

Dann Corbit wrote:
> 
> > -----Original Message-----
> > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
> > Sent: Monday, June 17, 2002 6:20 PM
> > To: Dann Corbit
> > Cc: Jan Wieck; Peter Eisentraut; PostgreSQL-development
> > Subject: Re: [HACKERS] Roadmap for a Win32 port
> >
> >
> > Dann Corbit wrote:
> > > > > It will be at least another copy of the postmaster (dot exe).
> > > >
> > > > Yea, I just liked the idea of the postmaster binary
> > somehow reporting
> > > > the postmaster status.  Seems it is in a better position to
> > > > do that than
> > > > a shell script.
> > >
> > > Architectural notion:
> > > The Postmaster is about 100x bigger than it needs to be.
> > >
> > > The Postmaster needs to set up shared memory and launch servers.
It
> > > does not need to know anything about SQL grammar or any of that
> > > rigamarole.
> > >
> > > It could be a 15K executable.
> > >
> > > Why not have an itty-bitty Postmaster that does nothing but
> > a spawn or a
> > > create process to create threaded Postgres instances?
> >
> > Can't.  postmaster/postgres are symlinks to the same file,
> > and we fork()
> > from postmaster to create backends.  All the code has to be in the
> > postmaster so the fork works.
> 
> Is fork() faster than creation of a new process via exec()?  After the
> creation of the shared memory, the information needed to use it could
be
> passed to the Postgres servers on the command line.

exec() does NOT create new processes. It loads another executable file
into the existing, calling process. 

fork() duplicates the calling process. In modern unix variants, this is
done in a very efficient way, so that the text segment (program code) is
shared readonly and everything else (data and stack segments) are shared
copy on write. Thus, fork() itself doesn't even cause memory copying.
That happens later when one of the now two processes writes to a memory
page the first time.

Windows does not have these two separate steps. It wants the full blown
expensive "create process and load executable", or the "let's all muck
around with the same handles" modell, called threading. 

> 
> The startup stuff for PostgreSQL is just a few files.  It does not
seem
> insurmountable to change it.  But it is none of my business.  If it is
a
> major hassle (for reasons which I am not aware) then I see no driving
> reason to change it.

It has to be changed for Windows, it is a major hassle for reasons I
wasn't aware of, and I am half way through ;-)


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/users-lounge/docs/faq.html

Le Mardi 18 Juin 2002 18:42, Serge Adda a écrit :
> I am new to PostgreSQL, but I am interested in the Win32 port.
> I have studied the architecture of other databases like Oracle.

Hello,

It seems clear that several teams are working without central point management
and contact:

- W32 port: at least a Japase team, a PostgreSQL hacker team and a company are
working separately. This makes three separate efforts for the most important
project this year.

- Replication: development is slow although a lot of people would be
interested in helping. But there is no central organization apart from the
hackers-list.

- Web site: a www list is working on several issues. Is there a central design
for all PostgreSQL site, like PHP does?

- Marketing: MySQL sucks and has a team of marketing sending junk technical
emails and writing false benchmarks. Who is in charge of marketing at
PostgreSQL? Where can I find a list of PostgreSQL features?

Personnaly, I agree with someone who wrote "Remember Betamax. It was the best
technical standard, but it died for commercial reasons". I don't want
PostgreSQL to be the next Betamax.

Some projects, like Debian, have a democratic organisation. The team leader is
elected for a year. Why not settle a similar organization? This would help
take decisions ... and not loose time on important issues.

PostgreSQL is a software but it is also a community. If we believe in
democracy, I suggest we should organize in a democratic way and elect a
leader for a year.

Just my 2 cents.
Cheers.
Jean-Michel POURE

On Thu, Jun 20, 2002 at 12:01:53PM +0200, Jean-Michel POURE wrote:

> Some projects, like Debian, have a democratic organisation. The team leader is 
> elected for a year. Why not settle a similar organization? This would help 
IMHO there is not problem with organization -- I don't know what doyou want to organize on actual number of developers
/contributors:-)The PostgreSQL is not project like Debian -- if you want to work on Debian you must know package system
andDebian standards only. But if you want to work on PostgreSQL you must be at the least average programmer (means
severalyears experience) and you must know a great many of current PostgreSQL code.
 
> PostgreSQL is a software but it is also a community. If we believe in 
> democracy, I suggest we should organize in a democratic way and elect a 
> leader for a year.
What is non-democratic now?
   Karel

-- Karel Zak  <zakkr@zf.jcu.cz>http://home.zf.jcu.cz/~zakkr/C, PostgreSQL, PHP, WWW, http://docs.linux.cz,
http://mape.jcu.cz

Le Jeudi 20 Juin 2002 13:39, Karel Zak a écrit :
>  IMHO there is not problem with organization -- I don't know what do
>  you want to organize on actual number of developers / contributors

Dear Karel,

My previous e-mail points out several projects where, IMHO, a leadership would
benefit the community at large :
- replication,
- W32 port,
- marketing (read the post "Read this and puke").

>  What is non-democratic now?

The current processes are based on discussion, and therefore are democratic.
My proposal does not intend to change discussion processes between
pgsql-hackers.

But, in order to face companies like MySQL AB, Oracle or Micro$oft, the
community needs to take important decisions that will help team work. A
clarified organization would help.

Please note I am not a PostgreSQL hacker myself, as I do not contribute code
to PostgreSQL main sources. But, as an outside spectator, I would only like
to point out that some efforts need coordination.

Debian is a very interesting example of Open-Source organization, as for all
aspects linked to "decision making". Usually, at Debian, when a discussion is
driven, a clear choice arizes after a limited time. Projects are sometimes
slow, but always reach their goals.

As for current PostgreSQL organization, can someone explain me which W32 port
will make its way to PostgreSQL main source code? Can someone publish a
schedule for replication availability? Who is in charge of explaining newbees
that MySQL InnoDB is just a marketing lie? What is the current PostgreSQL
market share?

In other words, we should ask ourselves the question of PostgreSQL future
organization. We come to point where PostgreSQL has equal chances to become
the #1 database or die like Betamax.

Best regards to you all,
Jean-Michel

On Thu, 2002-06-20 at 14:33, Jean-Michel POURE wrote:
> Le Jeudi 20 Juin 2002 13:39, Karel Zak a écrit :
> >  IMHO there is not problem with organization -- I don't know what do
> >  you want to organize on actual number of developers / contributors
>
> Dear Karel,
>
> My previous e-mail points out several projects where, IMHO, a leadership would
> benefit the community at large :
> - replication,
> - W32 port,
> - marketing (read the post "Read this and puke").
>
> >  What is non-democratic now?
>
> The current processes are based on discussion, and therefore are democratic.
> My proposal does not intend to change discussion processes between
> pgsql-hackers.
>
> But, in order to face companies like MySQL AB, Oracle or Micro$oft, the
> community needs to take important decisions that will help team work. A
> clarified organization would help.

In what way ?

Do you really think that if we would elect Tom or Bruce or someone else
"The President of the PostgreSQL Community" then their word would weight
more in mainstream press ?

> Please note I am not a PostgreSQL hacker myself, as I do not contribute code
> to PostgreSQL main sources. But, as an outside spectator, I would only like
> to point out that some efforts need coordination.
>
> Debian is a very interesting example of Open-Source organization, as for all
> aspects linked to "decision making". Usually, at Debian, when a discussion is
> driven, a clear choice arizes after a limited time. Projects are sometimes
> slow, but always reach their goals.

From an "outside spactator" perspective Debian seems to be really,
really slow.
> As for current PostgreSQL organization, can someone explain me which W32 port
> will make its way to PostgreSQL main source code?

There is already one W32 port in main source (the one that uses cygwin).

IMHO the first native port that a) worksb) does not make *nix version slower/harder to maintain
andc) is submitted for inclusion will make it into main source.

> Can someone publish a schedule for replication availability?

I guess any of the teams working on different ways to replicate could do
it. Another question is if they can stick to it.

> Who is in charge of explaining newbees that MySQL InnoDB is just a
> marketing lie?

Nobody is "in charge", but everybody is welcome to do it, even without
being "elected" or "nominated ";)

Still, having a "success stories" or "advocacy" section on
www.postgresq.org seems like a good idea.

> What is the current PostgreSQL  market share?
>
> In other words, we should ask ourselves the question of PostgreSQL future
> organization.

The current organization is "a loosely knit team" which seems to work
quite well.

> We come to point where PostgreSQL has equal chances to become
> the #1 database or die like Betamax.

Open-source will probably work differently, i.e postgres will probably
not die even if it will not be #1.

----------------
Hannu

On Thu, Jun 20, 2002 at 02:33:04PM +0200, Jean-Michel POURE wrote:
> Le Jeudi 20 Juin 2002 13:39, Karel Zak a écrit :
> >  IMHO there is not problem with organization -- I don't know what do
> >  you want to organize on actual number of developers / contributors
> 
> My previous e-mail points out several projects where, IMHO, a leadership would 
> benefit the community at large :
> - replication,
> - W32 port,
> - marketing (read the post "Read this and puke").
I understend you. I saw a lot of project and ideas, but it _always_ depend on people and their time. You can organize,
youcanprepare cool planns, but don't forget -- in finale must somebodyimplement it. Who? I don't know method how clone
TomLane or getmoney for others developers who can't full time work on PostgreSQLnow. 
 
> As for current PostgreSQL organization, can someone explain me which W32 port 
> will make its way to PostgreSQL main source code? Can someone publish a 
If nobody -- you can test it, make it better and write somethingabout it. If nobody works on some theme it means this
themeis notimportant for now. BUT everybody can change it and everybody can start work on arbitrary TODO item. It seems
hard,but it's right :-)   Karel
 

-- Karel Zak  <zakkr@zf.jcu.cz>http://home.zf.jcu.cz/~zakkr/C, PostgreSQL, PHP, WWW, http://docs.linux.cz,
http://mape.jcu.cz

Jean-Michel POURE wrote:
> [...]
> As for current PostgreSQL organization, can someone explain me which W32 port
> will make its way to PostgreSQL main source code? Can someone publish a
> schedule for replication availability? Who is in charge of explaining newbees
> that MySQL InnoDB is just a marketing lie? What is the current PostgreSQL
> market share?

I think the first native Win32 port that gets contributet will make it.
So far I have seen a couple of people claiming the "have" a native Win32
port, not using CygWIN. But none of them was willing to contribute their
work.

There is no official "schedule" for any of the features. PostgreSQL is a
100% volunteer project, so people work on it if and when they feel the
need to AND have the time. Even if some of us do work on PostgreSQL at
their job, a few even fulltime, you will not see such commitments.

And why do you think InnoDB is a marketing lie? It bought MySQL a good
number of features, including row level locking, transactions and
limited foreign key support. Actually it stopped MySQL from yelling out
the FUD they told people far too long, like "transactions are useless
overhead" and "referential integrity is bad".


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

On Thu, 20 Jun 2002, Jean-Michel POURE wrote:

> Le Jeudi 20 Juin 2002 13:39, Karel Zak a écrit :
> >  IMHO there is not problem with organization -- I don't know what do
> >  you want to organize on actual number of developers / contributors
>
> Dear Karel,
>
> My previous e-mail points out several projects where, IMHO, a leadership would
> benefit the community at large :
> - replication,
> - W32 port,
> - marketing (read the post "Read this and puke").
>
> >  What is non-democratic now?
>
> The current processes are based on discussion, and therefore are democratic.
> My proposal does not intend to change discussion processes between
> pgsql-hackers.
>
> But, in order to face companies like MySQL AB, Oracle or Micro$oft, the
> community needs to take important decisions that will help team work. A
> clarified organization would help.

Jean,

Why on earth does this matter? Postgres will continue to be a good
database as long as developers and users cut code. It is not a
sufficiently complicated project to warrant too much concern about things
like this. Besides, a significant amount of the code committed to
Postgres is inspired by personal interest not obligation.

Gavin

Jean-Michel POURE <jm.poure@freesurf.fr> writes:
> As for current PostgreSQL organization, can someone explain me which
> W32 port will make its way to PostgreSQL main source code? Can someone
> publish a schedule for replication availability? Who is in charge of
> explaining newbees that MySQL InnoDB is just a marketing lie? What is
> the current PostgreSQL market share?

And an "elected leader" will make all this stuff clear exactly how?

If someone would like to step up and actually *do* that marketing work,
that'd be great.  Complaining that it's not being done is a waste of
list bandwidth.  Electing someone isn't going to magically make it
happen, either.

It does concern me that there seem to be several W32 porting efforts
going on without contact with each other or with the wider pghackers
community; but if they want to work that way, I don't think there's
much we can do to force them to come out in the open.

BTW, we do already have a recognized leadership group: the core
committee.  The committee members mostly prefer to lead by example
and by consensus, rather than trying to impose their will on others,
which is maybe why you hadn't noticed.
        regards, tom lane

Le Jeudi 20 Juin 2002 15:22, Tom Lane a écrit :
> BTW, we do already have a recognized leadership group: the core
> committee.  The committee members mostly prefer to lead by example
> and by consensus, rather than trying to impose their will on others,
> which is maybe why you hadn't noticed.

You are right. Leading by example and consensus is the way things work. If the
committee can publish clear goals, there is no need to elect a leader.

Now I realize my concern is about project management. Do you think the
committee could publish a project web page showing who works on what?

Someone willing to help on a projet would view the page and contact
developpers more easilly. Presently, there is no clear knowledge who is
working on the W32 port or on replication. Using a project page, there would
be less chances to see new forks of particular projects (W32, replication).

This page would be accessible from the to-do-list. Conversly, the to-do-list
would lead to the projects page.

Just my 2 cents.
Thanks to all of you who replied my mail.

Cheers,
Jean-Michel

Jean-Michel POURE wrote:
> Le Mardi 18 Juin 2002 18:42, Serge Adda a ?crit :
> > I am new to PostgreSQL, but I am interested in the Win32 port.
> > I have studied the architecture of other databases like Oracle.
> 
> Hello,
> 
> It seems clear that several teams are working without central point management 
> and contact:
> 
> - W32 port: at least a Japanese team, a PostgreSQL hacker team and a company are 
> working separately. This makes three separate efforts for the most important 
> project this year.

Funny you should mention that. I talked with three people on the phone
just yesterday in an attempt to get them together on the Win32 port.  I
have been hesitant to publicly try and join them together because they
haven't publicly stated if they are going to contribute their code
back to the project, or haven't been clear on _when_ they would
contribute it back.

My goal is to take the roadmap I posted on June 5, make a web page, and
get some agreement from the community on how to address each item:
http://candle.pha.pa.us/cgi-bin/pgtodo?win32
. 
Then, the groups can know how we prefer to have things done, and if they
contribute back quickly, the other projects can benefit from their work,
and we _don't_ get conflicting Win32 implementations contributed back to
the project.

> - Replication: development is slow although a lot of people would be 
> interested in helping. But there is no central organization apart from the 
> hackers-list.

I am on the replication mailing list:
http://gborg.postgresql.org/project/pgreplication/projdisplay.php

That project is moving along, and hopefully they can merge their code
into the main tree so others can assist them.  Not sure what else we can
do.

> - Web site: a www list is working on several issues. Is there a central design 
> for all PostgreSQL site, like PHP does?
> 
> - Marketing: MySQL sucks and has a team of marketing sending junk technical 
> emails and writing false benchmarks. Who is in charge of marketing at 
> PostgreSQL? Where can I find a list of PostgreSQL features?

I skipped commenting on the marketing discussion because I was working
on a patch.  Probably our biggest problem compared to MySQL is that
MySQL is a single company behind a single database, while we have >4
companies behind PostgreSQL, and their thrust is promoting their company
rather than PostgreSQL itself.

While we have an excellent development team, we don't have a funded team
to go around to trade shows and write articles promoting PostgreSQL.  We
only have a very few people employed full-time with PostgreSQL, and it
usually takes a few full-time people to do just marketing.  Great Bridge
was really the only one to push PostgreSQL, and they are gone.

MySQL has such a team, and so does Oracle, and it helps.  Linux was in a
similar boat, with multiple companies behind Linux, and every one
promoting its own company rather than Linux itself.  We need large
PostgreSQL companies that promote themselves, and PostgreSQL along with
it.  Linux is in the same boat, with distributors promoting themselves
and Linux along with it.


> Personnaly, I agree with someone who wrote "Remember Betamax. It was the best 
> technical standard, but it died for commercial reasons". I don't want 
> PostgreSQL to be the next Betamax.

I thought the problem with Betamax was that Sony controlled the
standard, and other companies didn't like that.


I am always looking for more ideas in these areas.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

...
> MySQL has such a team, and so does Oracle, and it helps.  Linux was in a
> similar boat, with multiple companies behind Linux, and every one
> promoting its own company rather than Linux itself.  We need large
> PostgreSQL companies that promote themselves, and PostgreSQL along with
> it.  Linux is in the same boat, with distributors promoting themselves
> and Linux along with it.

Right. That may have slowed down the trade press recognition of Linux,
but in the end Linux has risen into view by popular demand and acclaim.

PostgreSQL has a similar opportunity and can succeed on a similar path.
We do have a superior technical solution, we have a superior open-source
support system, and we have (imho :) a superior development model and
development team open to new contributions and innovation.

The basis is there for success into the future, and at some point we
will get a lot for a little effort on marketing. Other products
bootstrap themselves by marketing early and often. And to the detriment
of the technical side.
                  - Thomas

Jan Wieck wrote:
> Jean-Michel POURE wrote:
> > [...]
> > As for current PostgreSQL organization, can someone explain me which W32 port
> > will make its way to PostgreSQL main source code? Can someone publish a
> > schedule for replication availability? Who is in charge of explaining newbees
> > that MySQL InnoDB is just a marketing lie? What is the current PostgreSQL
> > market share?
> 
> I think the first native Win32 port that gets contributet will make it.

This line bothers me.  With multiple people working on Win32, I would
like us to decide how we would _like_ such a port to be implemented. I
think this will assist those working on the project to _know_ that their
work will be accepted if submitted.

Also, I encourage people working on Win32 ports to contribute their work
_as_ the complete each module, rather than as one huge patch.  That way,
other Win32 porters can benefit from their work and maybe we can get
this done in 1/2 the time.

What I don't want to happen is two Win32 projects contributing duplicate
code at the same time.  It is a waste when they could have combined
their efforts.

So, my message to Win32 porters is to communicate what you are doing,
including implementation details, and contribute back as soon as you
can.  If you don't, someone else may beat you to it, and your patch may
be rejected, leaving you to either scrap your work or continually port
your changes to every future PostgreSQL release.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

On Thu, 20 Jun 2002, Jean-Michel POURE wrote:

> As for current PostgreSQL organization, can someone explain me which W32
> port will make its way to PostgreSQL main source code?

Whichever one actually submits patches for review first that is deemed
acceptable for inclusion ... as its always been ...

> Can someone publish a schedule for replication availability?

Its available now, and has been for several months *shrug*  There is a
project called PgReplication that is Open Source (hopefully someone from
that camp will pop up) ... and PgSQL, Inc has several deployments of their
commercial replication out there now, with several more pending ...

In fact, all of our work has been based on the rserv code that is in
contrib, and that we released over (almost?) a year ago ... but I've seen
nobody actually try to build on it, altho its what we've extended and are
using sucessfully in production environments ...

> Who is in charge of explaining newbees that MySQL InnoDB is just a
> marketing lie?

You are ... and anyone else that asks about it / mentions it ...

> What is the current PostgreSQL market share?

If you can think of a method of calculating this, as there is no
'commercial licensing' involved, please let us know ... would love to find
out ... there have been several surveys about it, but, quite frankly,
without having some sort of 'licensing' required to use, there is zero way
of getting any *real* numbers on this ...

> In other words, we should ask ourselves the question of PostgreSQL
> future organization. We come to point where PostgreSQL has equal chances
> to become the #1 database or die like Betamax.

At this point in time, the current organization == future organization ...
you are putting a 'marketing effort' as being the responsibility of the
open source project, and using MySQL AB as a comparison ... MySQL AB is a
*commercial company*, as is PostgreSQL, Inc -and- SRA and several other
newcomers, all of whom are doing marketing in their own way, based on
budget and requirements for growth ...

The "developmental organization", which we are, has been successful for
the past 7 years now ... flame wars are minimal, as are disagreements ...
there are some patches that get rejected that those generating them are
disappointed about, but most of *them* just bounce back with improvements
based on what has been told to them as being unacceptable ... about the
only thing that *has* changed over the past 7 years is that our standards
are tightened up as we move from mainly fixing bugs/stability to improving
the server itself ...

On Thu, 20 Jun 2002, Bruce Momjian wrote:

> Jan Wieck wrote:
> > Jean-Michel POURE wrote:
> > > [...]
> > > As for current PostgreSQL organization, can someone explain me which W32 port
> > > will make its way to PostgreSQL main source code? Can someone publish a
> > > schedule for replication availability? Who is in charge of explaining newbees
> > > that MySQL InnoDB is just a marketing lie? What is the current PostgreSQL
> > > market share?
> >
> > I think the first native Win32 port that gets contributet will make it.
>
> This line bothers me.  With multiple people working on Win32, I would
> like us to decide how we would _like_ such a port to be implemented. I
> think this will assist those working on the project to _know_ that their
> work will be accepted if submitted.

1. that is not accurate, as it won't necessarily be accepted if submitted
... its not like years ago when are standards were lax ... Jan is
perfectly accurate in his assessment of which will make it in, except for
omitting one point ... it has to meet our standards ... so, its more "the
first to contribute *and* meet our standards" ...

> Also, I encourage people working on Win32 ports to contribute their work
> _as_ the complete each module, rather than as one huge patch.  That way,
> other Win32 porters can benefit from their work and maybe we can get
> this done in 1/2 the time.

This is what should be done anyway ... I wouldn't even be so much worried
abou t"other Win32 porters" as much as trying to integrate that 'hugh
patch' at a later date, consiedering all the changes that go into our code
:)

> What I don't want to happen is two Win32 projects contributing duplicate
> code at the same time.  It is a waste when they could have combined
> their efforts.

IMHO, that is actually their problem ... without meaning to sound crass
about it, but its not like we haven't discussed it extensively here, and
openly ... hell, we've even tried to break down the whole project into
smaller components to make the whole easier to merge in :)

> So, my message to Win32 porters is to communicate what you are doing,
> including implementation details, and contribute back as soon as you
> can.  If you don't, someone else may beat you to it, and your patch may
> be rejected, leaving you to either scrap your work or continually port
> your changes to every future PostgreSQL release.

Agreed here ... and this doesn't just apply to Win32 porters ... there
have been *alot* of "big projects" worked on over the years that have been
implemented in pieces so that code-base doesn't diverge too much, making
patching difficult ... the last thing anyone should want is to work for a
few months to create a nice big patch to find out that what they have
"fixed" got yanked out of the code already :)

On Thu, 20 Jun 2002, Jean-Michel POURE wrote:

> Le Jeudi 20 Juin 2002 15:22, Tom Lane a �crit :
> > BTW, we do already have a recognized leadership group: the core
> > committee.  The committee members mostly prefer to lead by example
> > and by consensus, rather than trying to impose their will on others,
> > which is maybe why you hadn't noticed.
>
> You are right. Leading by example and consensus is the way things work. If the
> committee can publish clear goals, there is no need to elect a leader.
>
> Now I realize my concern is about project management. Do you think the
> committee could publish a project web page showing who works on what?

No ... cause nobody works on anything specific ... they work on what
appeals to them, as it appeals to them ...

> Someone willing to help on a projet would view the page and contact
> developpers more easilly. Presently, there is no clear knowledge who is
> working on the W32 port or on replication. Using a project page, there
> would be less chances to see new forks of particular projects (W32,
> replication).

If someone workign on the w32 port isn't willing to announce such on
-hackers, how do you think we'd get that information for a web page?  The
easiest way to 'contact developers' is to post to this list, and then you
don't restrict yourself to anyone one person, but everyone involved in teh
project ...

I think you are confusing a project on the scale of an operating system
with PgSQL ... an OS has to have someone at the top to make sure each
sub-package integrates well into the overall OS ... we *are* the
sub-package, and our "leaders" are essentially "our developers" ... we
have a core committee that consists of several of us that brought PgSQL
out of Berkeley and into the real world, but if you saw the archives for
that list, you woudln't see much, since we tend to try and keep all
discussions *on* -hackers ...

On 20 Jun 2002, Hannu Krosing wrote:

> Nobody is "in charge", but everybody is welcome to do it, even without
> being "elected" or "nominated ";)
>
> Still, having a "success stories" or "advocacy" section on
> www.postgresq.org seems like a good idea.

Being worked on ... we are actually working on totally revamping and
cleaning up the web site(s) ...

On Thu, 20 Jun 2002 12:09:35 -0400 (EDT)
"Bruce Momjian" <pgman@candle.pha.pa.us> wrote:
> Jean-Michel POURE wrote:
> > - Replication: development is slow although a lot of people would be 
> > interested in helping. But there is no central organization apart from the 
> > hackers-list.

Replication development is co-ordinated on the pgreplication-general
list which Bruce mentions below, not on -hackers. Any interested
developers should subscribe to it, read the relevant research papers
on Postgres-R, and contribute code.

As for the speed of development, I've started to contribute code recently,
and Darren Johnson (the main replication developer) says he should have some
free time soon -- there are also some other new developers interested in
the project. So while there was a period of inactivity, I think that progress
is now being made.

> I am on the replication mailing list:
> 
>     http://gborg.postgresql.org/project/pgreplication/projdisplay.php
> 
> That project is moving along, and hopefully they can merge their code
> into the main tree so others can assist them.  Not sure what else we can
> do.

I can't make any claims about a schedule -- since everyone working on the
project is a volunteer, the only release goals I'd like to set are "when it's
ready". The code right now is pretty unstable, so I don't think it's appropriate
for the main CVS tree until we work it into better shape.

Cheers,

Neil

-- 
Neil Conway <neilconway@rogers.com>
PGP Key ID: DB3C29FC

"Marc G. Fournier" wrote:
> 
> On Thu, 20 Jun 2002, Bruce Momjian wrote:
> 
> > This line bothers me.  With multiple people working on Win32, I would
> > like us to decide how we would _like_ such a port to be implemented. I
> > think this will assist those working on the project to _know_ that their
> > work will be accepted if submitted.
> 
> 1. that is not accurate, as it won't necessarily be accepted if submitted
> ... its not like years ago when are standards were lax ... Jan is
> perfectly accurate in his assessment of which will make it in, except for
> omitting one point ... it has to meet our standards ... so, its more "the
> first to contribute *and* meet our standards" ...

Omitted point taken :-)

> > What I don't want to happen is two Win32 projects contributing duplicate
> > code at the same time.  It is a waste when they could have combined
> > their efforts.
> 
> IMHO, that is actually their problem ... without meaning to sound crass
> about it, but its not like we haven't discussed it extensively here, and
> openly ... hell, we've even tried to break down the whole project into
> smaller components to make the whole easier to merge in :)

The problem with this kind of project is that you have a big stumbling
block at the beginning, which has to be done before you can rollout and
integrate the help of developers scattered around the globe. This was
the case with the foreign key project, where the trigger queue and one
set of triggers where working, and then Stephan did all the others and I
forgot who else helped to do the utility commands and CREATE TABLE
syntax and tried to decrypt the SQL definitions? In the Windows port
case it is to get it as far that you at least can fire up a postmaster,
get past the startup process, connect to the database and do a few
queries before the thing blows up. Before this everybody has exactly the
same problem, "It doesn't startup", so the likelyhood of everyone
stomping over each others work every single night is about 99.9%!


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #

Jan Wieck wrote:
> > > What I don't want to happen is two Win32 projects contributing duplicate
> > > code at the same time.  It is a waste when they could have combined
> > > their efforts.
> > 
> > IMHO, that is actually their problem ... without meaning to sound crass
> > about it, but its not like we haven't discussed it extensively here, and
> > openly ... hell, we've even tried to break down the whole project into
> > smaller components to make the whole easier to merge in :)
> 
> The problem with this kind of project is that you have a big stumbling
> block at the beginning, which has to be done before you can rollout and
> integrate the help of developers scattered around the globe. This was
> the case with the foreign key project, where the trigger queue and one
> set of triggers where working, and then Stephan did all the others and I
> forgot who else helped to do the utility commands and CREATE TABLE
> syntax and tried to decrypt the SQL definitions? In the Windows port
> case it is to get it as far that you at least can fire up a postmaster,
> get past the startup process, connect to the database and do a few
> queries before the thing blows up. Before this everybody has exactly the
> same problem, "It doesn't startup", so the likelyhood of everyone
> stomping over each others work every single night is about 99.9%!

Yes, but it doesn't prevent discussion.  I think open implementation
discussion will help.  I am suggesting this to everyone, not just Jan. 
I have been in private discussion with others too.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

On Thursday 20 June 2002 02:57 pm, Jan Wieck wrote:
> set of triggers where working, and then Stephan did all the others and I
> forgot who else helped to do the utility commands and CREATE TABLE
> syntax and tried to decrypt the SQL definitions?

Don Baccus?
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Jan Wieck wrote:
> 
> "Marc G. Fournier" wrote:
> >

...

> > IMHO, that is actually their problem ... without meaning to sound crass
> > about it, but its not like we haven't discussed it extensively here, and
> > openly ... hell, we've even tried to break down the whole project into
> > smaller components to make the whole easier to merge in :)
> 
> The problem with this kind of project is that you have a big stumbling
> block at the beginning, which has to be done before you can rollout and
> integrate the help of developers scattered around the globe. This was
> the case with the foreign key project, where the trigger queue and one
> set of triggers where working, and then Stephan did all the others and I
> forgot who else helped to do the utility commands and CREATE TABLE
> syntax and tried to decrypt the SQL definitions? In the Windows port
> case it is to get it as far that you at least can fire up a postmaster,
> get past the startup process, connect to the database and do a few
> queries before the thing blows up. Before this everybody has exactly the
> same problem, "It doesn't startup", so the likelyhood of everyone
> stomping over each others work every single night is about 99.9%!

It would be nice to also have it "fire up" under Windows CE as well ;-)

Mike Mascari
mascarm@mascari.com

On Thu, 20 Jun 2002, Jan Wieck wrote:

> "Marc G. Fournier" wrote:
> >
> > On Thu, 20 Jun 2002, Bruce Momjian wrote:
> >
> > > This line bothers me.  With multiple people working on Win32, I would
> > > like us to decide how we would _like_ such a port to be implemented. I
> > > think this will assist those working on the project to _know_ that their
> > > work will be accepted if submitted.
> >
> > 1. that is not accurate, as it won't necessarily be accepted if submitted
> > ... its not like years ago when are standards were lax ... Jan is
> > perfectly accurate in his assessment of which will make it in, except for
> > omitting one point ... it has to meet our standards ... so, its more "the
> > first to contribute *and* meet our standards" ...
>
> Omitted point taken :-)
>
> > > What I don't want to happen is two Win32 projects contributing duplicate
> > > code at the same time.  It is a waste when they could have combined
> > > their efforts.
> >
> > IMHO, that is actually their problem ... without meaning to sound crass
> > about it, but its not like we haven't discussed it extensively here, and
> > openly ... hell, we've even tried to break down the whole project into
> > smaller components to make the whole easier to merge in :)
>
> The problem with this kind of project is that you have a big stumbling
> block at the beginning, which has to be done before you can rollout and
> integrate the help of developers scattered around the globe. This was
> the case with the foreign key project, where the trigger queue and one
> set of triggers where working, and then Stephan did all the others and I
> forgot who else helped to do the utility commands and CREATE TABLE
> syntax and tried to decrypt the SQL definitions? In the Windows port

Actually, IIRC Don did the triggers, and I did the utility commands/create
stuff, but the point is still the same.  (Made in the point of
historical accuracy since I don't want someone else's work to end up
getting attributed to me since that's unfair to them. :) )

Stephan Szabo wrote:

> Actually, IIRC Don did the triggers, and I did the utility commands/create
> stuff, but the point is still the same.  (Made in the point of
> historical accuracy since I don't want someone else's work to end up
> getting attributed to me since that's unfair to them. :) )

This is why I love this PostgreSQL Project. The honesty and
humbleness everyone is practicing. Who ever did what, together we
did a good job! Which other open source database has the full set
of referential actions and deferrability?


Jan

-- 

#======================================================================#
# It's easier to get forgiveness for being wrong than for being
right. #
# Let's break this rule - forgive
me.                                  #
#==================================================
JanWieck@Yahoo.com #

It could be helpful to create a mailing list just for this project,
since not all members of pg-hackers will/shall participate, and we
would probably flood this list quite a bit trying to figure out what
is the best way to implement a win32 port.  Just like the
pg-replication list, this new list would be project specific.

However, as an aside, I think the 'first best fit shall be commited'
approach is a _bad_ idea.  Everyone (whos interested in the port)
agrees with the basic goals, and we will get a working system much
faster if we all work on a single solution: And not try to race each
other.  
If the main pg developers do not want to bless a specific method/project
for the port, then the people interested should hash it out, before
hundreds of man-hours are wasted developing something that ends up not
being used.  Debuging-into existence is a bad idea, as the single-night
example hints at (wether intentionaly or not) - with a proper plan we
should be able to create unit tests that can prove whether the methods
choosen are functioning well before we ever get a fully working
postmaster.

~Jon Franz

Bruce Momjian wrote:

>Jan Wieck wrote:
>
>>>>What I don't want to happen is two Win32 projects contributing duplicate
>>>>code at the same time.  It is a waste when they could have combined
>>>>their efforts.
>>>>
>>>IMHO, that is actually their problem ... without meaning to sound crass
>>>about it, but its not like we haven't discussed it extensively here, and
>>>openly ... hell, we've even tried to break down the whole project into
>>>smaller components to make the whole easier to merge in :)
>>>
>>The problem with this kind of project is that you have a big stumbling
>>block at the beginning, which has to be done before you can rollout and
>>integrate the help of developers scattered around the globe. This was
>>the case with the foreign key project, where the trigger queue and one
>>set of triggers where working, and then Stephan did all the others and I
>>forgot who else helped to do the utility commands and CREATE TABLE
>>syntax and tried to decrypt the SQL definitions? In the Windows port
>>case it is to get it as far that you at least can fire up a postmaster,
>>get past the startup process, connect to the database and do a few
>>queries before the thing blows up. Before this everybody has exactly the
>>same problem, "It doesn't startup", so the likelyhood of everyone
>>stomping over each others work every single night is about 99.9%!
>>
>
>Yes, but it doesn't prevent discussion.  I think open implementation
>discussion will help.  I am suggesting this to everyone, not just Jan. 
>I have been in private discussion with others too.
>

Jon Franz wrote:
> It could be helpful to create a mailing list just for this project,
> since not all members of pg-hackers will/shall participate, and we
> would probably flood this list quite a bit trying to figure out what
> is the best way to implement a win32 port.  Just like the
> pg-replication list, this new list would be project specific.
> 
> However, as an aside, I think the 'first best fit shall be commited'
> approach is a _bad_ idea.  Everyone (whos interested in the port)
> agrees with the basic goals, and we will get a working system much
> faster if we all work on a single solution: And not try to race each
> other.  

I think we have to be involved to prevent chaos when those patches
arrive.

> If the main pg developers do not want to bless a specific method/project
> for the port, then the people interested should hash it out, before
> hundreds of man-hours are wasted developing something that ends up not
> being used.  Debuging-into existence is a bad idea, as the single-night
> example hints at (wether intentionaly or not) - with a proper plan we
> should be able to create unit tests that can prove whether the methods
> choosen are functioning well before we ever get a fully working
> postmaster.

Actually, don't we have a cygwin mailing list?  Seems that would be a
great location, except for the name.  Maybe Marc can close the list and
migrate them all to a new 'win32' list.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 

On Thu, 20 Jun 2002, Bruce Momjian wrote:

> Jon Franz wrote:
> > It could be helpful to create a mailing list just for this project,
> > since not all members of pg-hackers will/shall participate, and we
> > would probably flood this list quite a bit trying to figure out what
> > is the best way to implement a win32 port.  Just like the
> > pg-replication list, this new list would be project specific.
> >
> > However, as an aside, I think the 'first best fit shall be commited'
> > approach is a _bad_ idea.  Everyone (whos interested in the port)
> > agrees with the basic goals, and we will get a working system much
> > faster if we all work on a single solution: And not try to race each
> > other.
>
> I think we have to be involved to prevent chaos when those patches
> arrive.
>
> > If the main pg developers do not want to bless a specific method/project
> > for the port, then the people interested should hash it out, before
> > hundreds of man-hours are wasted developing something that ends up not
> > being used.  Debuging-into existence is a bad idea, as the single-night
> > example hints at (wether intentionaly or not) - with a proper plan we
> > should be able to create unit tests that can prove whether the methods
> > choosen are functioning well before we ever get a fully working
> > postmaster.
>
> Actually, don't we have a cygwin mailing list?  Seems that would be a
> great location, except for the name.  Maybe Marc can close the list and
> migrate them all to a new 'win32' list.

Two different issues here ... a win32 port is different then the cygwin
issues ... also, I had suggested ages back creatin ga seperate list, but
was told that it would frustrate the effort ... so unless those that
disagreed with it in the first place have changed their mind, the
discussion of any porting effort to win32 should remain here ...