Re: Roadmap for a Win32 port - Mailing list pgsql-hackers

Jan Wieck wrote:
> > One solution is to return to that for Win32 only, so instead of doing:
> >
> >    initialization()
> >    want for connection()
> >    fork backend()
> >
> > we do for Win32:
> >
> >    want for connection()
> >    exec backend()
> >    initialization()
> 
>     Summarizes pretty much what we discussed Monday on the phone.
>     Except that the postmaster still has to initialize the shared
>     memory  and  other  stuff.  It's  just  that the backends and
>     helper processes need to reinitialize themself (attach).

Yes, obviously I simplified, and I do believe our optimizations are
helping on Unix.  It is just that I think for Win32 the fork is more
harmful than removing those optimizations.

One thing that may not have been clear is that we don't need to play
with globals at all.  We just pass whatever info we want to the child
via command-line arguments, rather than shared memory.

> > It wouldn't be hard to do.  We would still do CreateProcess rather than
> > CreateThread, but it eliminates the fork/threading issues.  We don't
> > know the database before the connection arrives, so we don't do a whole
> > lot of initialization.
> 
>     All I see so far is the reading of the  postgresql.conf,  the
>     pg_hba.conf  and  the  password  files. Nothing fancy and the
>     postmaster could easily write out a binary content only  file
>     that   the   backends  then  read,  eliminating  the  parsing
>     overhead.

Yes, that is clearly possible.  Another option is to just write out a
no-comments, no-whitespace version of each file and just have the
backends read those.  The advantage is that we can use the same code to
read them, and I don't think it would be any slower than a binary file.

>     The bad news is that Tom is right. We did a terrible  job  in
>     using  the new side effect, that the shared memory segment is
>     at the same address in all forked processes,  after  removing
>     the need to reattach.
> 
>     In  detail  the  XLog  code,  the  FreeSpaceMap  code and the
>     "shared memory" hashtable code now use pointers,  located  in
>     shared  memory.  For  the  XLog  and  FreeSpace  code this is
>     understandable, because they where developed under the fork()
>     only  model.  But  the  dynahash code used offsets only until
>     v7.1!
> 
>     All three (no claim that that's all) make  it  impossible  to
>     ever  have  someone  attaching  to the shared memory from the
>     outside. So with these moves we  made  the  shared  memory  a
>     "Postmaster  and  children" only thing.  Raises the question,
>     why we need an IPC key at all any more.

Well, we could force shmat() to bind to the same address, but I suspect
that might fail in some cases.

>     Anyhow, looks as if I can get that fork()  vs.  fork()+exec()
>     feature  done  pretty  soon.  It'll  be controlled by another
>     Postmaster commandline switch. After cleaning up the  mess  I
>     did  to  get  it  working  quick,  I'll  provide  a patch for
>     discussion.

Yes, very little impact.  We then need someone to do some Win32 timings
to see if things have improved.  As Tom mentioned, we need some hard
numbers for these things.  In fact, I would like a Win32 test that takes
our code and compares fork(), then exit(), with CreateProcess(), exit().
It doesn't have create a db session, but I would like to see some
timings to know what we are gaining. Heck, time CreateThread too and
let's see what that shows.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026