Thread: single task postgresql
Having frustrated with performance on Windows box I'm wondering if it's possible to get postgresql optimized for working without shared memory, say in single-task mode. It looks like it's shared memory emulation on disk (by cygipc daemon) is responsible for performance degradation. In our project we have to use Windows for desktop application and it's single task, so we don't need shared memory. In principle, it's possible to hack cygipc, so it wouldn't emulate shared memory and address calls to normal memory, but I'm wondering if it's possible from postgres side. Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Oleg Bartunov wrote: > > Having frustrated with performance on Windows box I'm wondering if it's > possible to get postgresql optimized for working without shared memory, > say in single-task mode. It looks like it's shared memory emulation on disk > (by cygipc daemon) is responsible for performance degradation. > In our project we have to use Windows for desktop application and it's > single task, so we don't need shared memory. In principle, it's possible > to hack cygipc, so it wouldn't emulate shared memory and address calls > to normal memory, but I'm wondering if it's possible from postgres side. > > Regards, How does cygwin do shared memory on Windows? Windows support real shared memory, surely the cygwin guys are using "real" shared memory. Are you sure that this is the problem? If so, might we not be able to use a few macros/#ifdefs to do it right? I'll be up for that if you guys don't mind some "ifdef/endif" stuff here and there.
Oleg Bartunov <oleg@sai.msu.su> writes:
> Having frustrated with performance on Windows box I'm wondering if it's
> possible to get postgresql optimized for working without shared memory,
> say in single-task mode. It looks like it's shared memory emulation on disk
> (by cygipc daemon) is responsible for performance degradation.
> In our project we have to use Windows for desktop application and it's
> single task, so we don't need shared memory. In principle, it's possible
> to hack cygipc, so it wouldn't emulate shared memory and address calls
> to normal memory, but I'm wondering if it's possible from postgres side.
As mlw comments, that is probably not really the source of the
performance issue. However, you should be able to hack it if you
want to check. A standalone backend just malloc()s what would otherwise
be the shared memory area. As a first approximation you could just fire
up a standalone backend and see if it seems any faster.
regards, tom lane
Oleg Bartunov wrote: > > we have to support earlier windows which have no shared memory support. I'm pretty sure it could a configuration option then. Using a file is just plain stupid. "Every" version of Windows has supported shared memory in one form or another. What version are you wishing to support which the cygwin guys claim does not have shared memory? I'm prety sure I can write a shared memory interface for Windows which should work across all "supported" versions. (BTW I have been a Windows developer since version 1.03.) > > On Tue, 26 Feb 2002, mlw wrote: > > > Oleg Bartunov wrote: > > > > > > Having frustrated with performance on Windows box I'm wondering if it's > > > possible to get postgresql optimized for working without shared memory, > > > say in single-task mode. It looks like it's shared memory emulation on disk > > > (by cygipc daemon) is responsible for performance degradation. > > > In our project we have to use Windows for desktop application and it's > > > single task, so we don't need shared memory. In principle, it's possible > > > to hack cygipc, so it wouldn't emulate shared memory and address calls > > > to normal memory, but I'm wondering if it's possible from postgres side. > > > > > > Regards, > > > > How does cygwin do shared memory on Windows? Windows support real shared > > memory, surely the cygwin guys are using "real" shared memory. Are you sure > > that this is the problem? If so, might we not be able to use a few > > macros/#ifdefs to do it right? > > > > I'll be up for that if you guys don't mind some "ifdef/endif" stuff here and > > there. > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83
Windows does not really have shared memory support. This has been a beef with the Win32 API for a long time now. Because it has been a long time complaint, it was finally added in Win2000 and later. Likewise, I'd like to point out that thinks like sims, shared memory, pipes, etc, and other entities commonly used for concurrent programming strategies are slower in XP. So, because shared memory really isn't well supported, they elected to have what is, in essense, memory mapped files. Multiple processes then map the same file and read/write to it as needed, more or less as you would shared memory. Unless you plan on only targetting on Win 2000 and XP, it sounds like a waste of time. As for your shared memory interface, I suspect you'll find up with pretty much what the cygwin guys have. Your implementation may prove to be slightly lighter as they may of had some other issues to address but I doubt yours would prove to be much of a difference from a performance perspective as it's still going to be pretty much the same thing under the covers for most of the Win32 platforms. Also, expect to be taking a performance hit with XP as is so I can't say targetting that platform makes much sense, IMOHO. Here's from MSDN: CreateSharedMemory The CreateSharedMemory function creates a section of memory that is shared by client processes and the security package. PVOID NTAPI CreateSharedMemory ( ULONG MaxSize, ULONG InitialSize ); Parameters [in] MaxSize Specifies the maximum size of the shared memory. [in] InitialSize Specifies the initial size of the shared memory. Return Values The function returns a pointer to the block of shared memory, or NULL if the block was not reserved. Remarks Creating a shared section for each client is not advisable because it is a very expensive operation and may exhaust system resources. The package's clients can write to shared memory which makes it susceptible to attack. Data in the shared segment should not be trusted. The pointer returned by the CreateSharedMemory function is required by the AllocateSharedMemory, DeleteSharedMemory, and FreeSharedMemory functions. Use the DeleteSharedMemory function to release memory reserved by the CreateSharedMemory function. Pointers to these functions are available in the LSA_SECPKG_FUNCTION_TABLE structure received by the SpInitialize function. Requirements Windows NT/2000 or later: Requires Windows 2000 or later. Header: Declared in Ntsecpkg.h. Here's an except from Python's Win32 API: Memory Mapped Files Access to Win32 Memory Mapped files. Memory mapped files allow efficient sharing of data between separate processes. It allows a block of (possibly shared) memory to behave like a file. This was just thrown in to show that memory mapped files are the standard for shared memory on Win32 systems. Likewise, I'd like to point out that I've not seen anything published that would suggest that shared memory versus memory mapped files via the Win32 API should yield a higher performance. Greg mlw wrote: > Oleg Bartunov wrote: > >>we have to support earlier windows which have no shared memory support. >> > > I'm pretty sure it could a configuration option then. Using a file is just > plain stupid. "Every" version of Windows has supported shared memory in one > form or another. > > What version are you wishing to support which the cygwin guys claim does not > have shared memory? > > I'm prety sure I can write a shared memory interface for Windows which should > work across all "supported" versions. > > (BTW I have been a Windows developer since version 1.03.) > > > > >>On Tue, 26 Feb 2002, mlw wrote: >> >> >>>Oleg Bartunov wrote: >>> >>>>Having frustrated with performance on Windows box I'm wondering if it's >>>>possible to get postgresql optimized for working without shared memory, >>>>say in single-task mode. It looks like it's shared memory emulation on disk >>>>(by cygipc daemon) is responsible for performance degradation. >>>>In our project we have to use Windows for desktop application and it's >>>>single task, so we don't need shared memory. In principle, it's possible >>>>to hack cygipc, so it wouldn't emulate shared memory and address calls >>>>to normal memory, but I'm wondering if it's possible from postgres side. >>>> >>>> Regards, >>>> >>>How does cygwin do shared memory on Windows? Windows support real shared >>>memory, surely the cygwin guys are using "real" shared memory. Are you sure >>>that this is the problem? If so, might we not be able to use a few >>>macros/#ifdefs to do it right? >>> >>>I'll be up for that if you guys don't mind some "ifdef/endif" stuff here and >>>there. >>> >>> >> Regards, >> Oleg >>_____________________________________________________________ >>Oleg Bartunov, sci.researcher, hostmaster of AstroNet, >>Sternberg Astronomical Institute, Moscow University (Russia) >>Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ >>phone: +007(095)939-16-83, +007(095)939-23-83 >> > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Greg Copeland, Principal Consultant Copeland Computer Consulting -------------------------------------------------- PGP/GPG Key at http://www.keyserver.net 5A66 1470 38F5 5E1B CABD 19AF E25A F56E 96DC 2FA9 --------------------------------------------------
Oh ya, sorry for not including these in my previous posting, however, here's the links that may prove to be of value for you: Memory mapped files as shared memory: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/filemap_64jd.asp Shared memory interface, added August 2001: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/security/customsecfunctions_13eh.asp Enjoy,Greg mlw wrote: > Oleg Bartunov wrote: > >>we have to support earlier windows which have no shared memory support. >> > > I'm pretty sure it could a configuration option then. Using a file is just > plain stupid. "Every" version of Windows has supported shared memory in one > form or another. > > What version are you wishing to support which the cygwin guys claim does not > have shared memory? > > I'm prety sure I can write a shared memory interface for Windows which should > work across all "supported" versions. > > (BTW I have been a Windows developer since version 1.03.) > > > > >>On Tue, 26 Feb 2002, mlw wrote: >> >> >>>Oleg Bartunov wrote: >>> >>>>Having frustrated with performance on Windows box I'm wondering if it's >>>>possible to get postgresql optimized for working without shared memory, >>>>say in single-task mode. It looks like it's shared memory emulation on disk >>>>(by cygipc daemon) is responsible for performance degradation. >>>>In our project we have to use Windows for desktop application and it's >>>>single task, so we don't need shared memory. In principle, it's possible >>>>to hack cygipc, so it wouldn't emulate shared memory and address calls >>>>to normal memory, but I'm wondering if it's possible from postgres side. >>>> >>>> Regards, >>>> >>>How does cygwin do shared memory on Windows? Windows support real shared >>>memory, surely the cygwin guys are using "real" shared memory. Are you sure >>>that this is the problem? If so, might we not be able to use a few >>>macros/#ifdefs to do it right? >>> >>>I'll be up for that if you guys don't mind some "ifdef/endif" stuff here and >>>there. >>> >>> >> Regards, >> Oleg >>_____________________________________________________________ >>Oleg Bartunov, sci.researcher, hostmaster of AstroNet, >>Sternberg Astronomical Institute, Moscow University (Russia) >>Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ >>phone: +007(095)939-16-83, +007(095)939-23-83 >> > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > -- Greg Copeland, Principal Consultant Copeland Computer Consulting -------------------------------------------------- PGP/GPG Key at http://www.keyserver.net 5A66 1470 38F5 5E1B CABD 19AF E25A F56E 96DC 2FA9 --------------------------------------------------
Greg Copeland wrote: > > Windows does not really have shared memory support. This has been a > beef with the Win32 API for a long time now. Because it has been a long > time complaint, it was finally added in Win2000 and later. Likewise, > I'd like to point out that thinks like sims, shared memory, pipes, etc, > and other entities commonly used for concurrent programming strategies > are slower in XP. So, because shared memory really isn't well > supported, they elected to have what is, in essense, memory mapped > files. Multiple processes then map the same file and read/write to it > as needed, more or less as you would shared memory. Unless you plan on > only targetting on Win 2000 and XP, it sounds like a waste of time. This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can be done in 16 bit land with a touch of assembly and a DLL. Allocate, with globalalloc, a shared memory segment. The base selector is a valid 32 bit selector, and the memory is mapped in the above 2G space shared and mapped to all 32bit processes. Under NT through 2K, yes using a memory mapped files is the way to do it, but you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, which is the NT equivilent of the system memory file. The handle returned is a system global object which can be shared across processes. >
If you can improve the ipc implementation under cygwin that would be great. Perhaps you could look at the source for the current implementation (http://www.neuro.gatech.edu/users/cwilson/cygutils/V1.1/cygipc/), and come up with a patch that would improve the performance/implmentation. --Barry mlw wrote: > Oleg Bartunov wrote: > >>we have to support earlier windows which have no shared memory support. >> > > I'm pretty sure it could a configuration option then. Using a file is just > plain stupid. "Every" version of Windows has supported shared memory in one > form or another. > > What version are you wishing to support which the cygwin guys claim does not > have shared memory? > > I'm prety sure I can write a shared memory interface for Windows which should > work across all "supported" versions. > > (BTW I have been a Windows developer since version 1.03.) > > > > >>On Tue, 26 Feb 2002, mlw wrote: >> >> >>>Oleg Bartunov wrote: >>> >>>>Having frustrated with performance on Windows box I'm wondering if it's >>>>possible to get postgresql optimized for working without shared memory, >>>>say in single-task mode. It looks like it's shared memory emulation on disk >>>>(by cygipc daemon) is responsible for performance degradation. >>>>In our project we have to use Windows for desktop application and it's >>>>single task, so we don't need shared memory. In principle, it's possible >>>>to hack cygipc, so it wouldn't emulate shared memory and address calls >>>>to normal memory, but I'm wondering if it's possible from postgres side. >>>> >>>> Regards, >>>> >>>How does cygwin do shared memory on Windows? Windows support real shared >>>memory, surely the cygwin guys are using "real" shared memory. Are you sure >>>that this is the problem? If so, might we not be able to use a few >>>macros/#ifdefs to do it right? >>> >>>I'll be up for that if you guys don't mind some "ifdef/endif" stuff here and >>>there. >>> >>> >> Regards, >> Oleg >>_____________________________________________________________ >>Oleg Bartunov, sci.researcher, hostmaster of AstroNet, >>Sternberg Astronomical Institute, Moscow University (Russia) >>Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ >>phone: +007(095)939-16-83, +007(095)939-23-83 >> > > ---------------------------(end of broadcast)--------------------------- > TIP 6: Have you searched our list archives? > > http://archives.postgresql.org > >
mlw wrote: > Greg Copeland wrote: > >>Windows does not really have shared memory support. This has been a >>beef with the Win32 API for a long time now. Because it has been a long >>time complaint, it was finally added in Win2000 and later. Likewise, >>I'd like to point out that thinks like sims, shared memory, pipes, etc, >>and other entities commonly used for concurrent programming strategies >>are slower in XP. So, because shared memory really isn't well >>supported, they elected to have what is, in essense, memory mapped >>files. Multiple processes then map the same file and read/write to it >>as needed, more or less as you would shared memory. Unless you plan on >>only targetting on Win 2000 and XP, it sounds like a waste of time. >> > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > globalalloc, a shared memory segment. The base selector is a valid 32 bit > selector, and the memory is mapped in the above 2G space shared and mapped to > all 32bit processes. That's fine ad all, however, we're talking about the Win32 API. None of that qualifies. If you wish to implement back door magic for your own use, I guess that's fine but I'd wonder how well it would be received by the rest of the developers at large. I'm betting it wouldn't give me the warm fuzzies. > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > which is the NT equivilent of the system memory file. The handle returned is a > system global object which can be shared across processes. > Yes, that's fine, however, this pretty much puts us back to a memory mapped file/shared memory implementation. I guess you could have a feature set which is perhaps optimial (again, I've not seen anything which indicates you'll see perf improvement) for Win2000/XP platforms. Perhaps you'd be willing to perform some benchmarks and provide source of random read/writes to these segments and present to us for comparsion? I'm personally very curious as to see if there is any addition speed to be gleaned even if I am doubtful. On a side note, if you did create such a benchmark, I'm fairly sure there are a couple of guys at IBM that may find them of interest and might be willing to contrast these results with other platforms (various win32 and unix/linux) and publish the results. After all, if this path truely provides an optimized solution I'm sure many Win32 coders would like to hear about it. Greg
Oleg Bartunov wrote: > > On Wed, 27 Feb 2002, mlw wrote: > > > Greg Copeland wrote: > > > > > > Windows does not really have shared memory support. This has been a > > > beef with the Win32 API for a long time now. Because it has been a long > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > and other entities commonly used for concurrent programming strategies > > > are slower in XP. So, because shared memory really isn't well > > > supported, they elected to have what is, in essense, memory mapped > > > files. Multiple processes then map the same file and read/write to it > > > as needed, more or less as you would shared memory. Unless you plan on > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > selector, and the memory is mapped in the above 2G space shared and mapped to > > all 32bit processes. > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > which is the NT equivilent of the system memory file. The handle returned is a > > system global object which can be shared across processes. > > > > Mark, > > do you consider to work on this issue ? Yea, let me think about it. What is your time frame? When I offered to work on it, I thought it could be a leasurely thing. I have to get a machine running some form of Windows on which to develop and test. I want to say yes, and if no one else does it, I will, but I'm not sure what your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If you need something quickly, I can help, but I don't think I could shoulder the whole thing. I have a couple things I have promised people. Let me get those done. I will try to write an equivilent set of functions for shget, shmat, etc. as soon as I can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. How does that sound?
Greg Copeland wrote: > > mlw wrote: > > Greg Copeland wrote: > > > >>Windows does not really have shared memory support. This has been a > >>beef with the Win32 API for a long time now. Because it has been a long > >>time complaint, it was finally added in Win2000 and later. Likewise, > >>I'd like to point out that thinks like sims, shared memory, pipes, etc, > >>and other entities commonly used for concurrent programming strategies > >>are slower in XP. So, because shared memory really isn't well > >>supported, they elected to have what is, in essense, memory mapped > >>files. Multiple processes then map the same file and read/write to it > >>as needed, more or less as you would shared memory. Unless you plan on > >>only targetting on Win 2000 and XP, it sounds like a waste of time. > >> > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > selector, and the memory is mapped in the above 2G space shared and mapped to > > all 32bit processes. > > That's fine ad all, however, we're talking about the Win32 API. None of > that qualifies. If you wish to implement back door magic for your own > use, I guess that's fine but I'd wonder how well it would be received by > the rest of the developers at large. I'm betting it wouldn't give me > the warm fuzzies. It cerainly wont give UNIX developers the warm and fuzzies, thats for sure, but I have developed Windows software since there has been a Windows, and believe me, there is no "non-trivial" Windows application which is not forced, at some point, to do these sorts of things. Windows, as a platform, is pure and simple, a disaster. > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > which is the NT equivilent of the system memory file. The handle returned is a > > system global object which can be shared across processes. > > > > Yes, that's fine, however, this pretty much puts us back to a memory > mapped file/shared memory implementation. Not nessisarily true. The system memory map should not have the same criteria for disk writes. A memory mapped file has some notion hard disk "persistence." NT will maintain the disk image. When a view of the system memory is mapped, there is no requirement that the disk image is written. It will follow the OS virtual memory strategy for generic memory. (It may even make it more persistent than generic RAM, I will have to double check.) > > I guess you could have a feature set which is perhaps optimial (again, > I've not seen anything which indicates you'll see perf improvement) for > Win2000/XP platforms. Perhaps you'd be willing to perform some > benchmarks and provide source of random read/writes to these segments > and present to us for comparsion? I'm personally very curious as to see > if there is any addition speed to be gleaned even if I am doubtful. I am not sure if there will be much benefit, I too am skeptical, with one caveat, if NT or Windows is using a shared file handle, there may be contention at the disk level for the disk, and that could impact performance. On NT, puting the shared memory file on one disk, and the database on another, one should be able to see how often the disk which contains the shared image is written. Another posibility is to create a RAM disk and put the shared memory file on it. > > On a side note, if you did create such a benchmark, I'm fairly sure > there are a couple of guys at IBM that may find them of interest and > might be willing to contrast these results with other platforms (various > win32 and unix/linux) and publish the results. > > After all, if this path truely provides an optimized solution I'm sure > many Win32 coders would like to hear about it. Sigh, it looks like I have to reinstall NT and startup my Windows development stuff again. I was having so much fun the last few years developing for Linux. Again, sigh. :-|
mlw wrote:
> Oleg Bartunov wrote:
>
>>On Wed, 27 Feb 2002, mlw wrote:
>>
>>
[Greg's ramblings removed]
>>
>>do you consider to work on this issue ?
>>
>
> Yea, let me think about it. What is your time frame? When I offered to work on
> it, I thought it could be a leasurely thing. I have to get a machine running
> some form of Windows on which to develop and test.
>
> I want to say yes, and if no one else does it, I will, but I'm not sure what
> your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If
> you need something quickly, I can help, but I don't think I could shoulder the
> whole thing.
It appears that cygipc's shared memory implementation boils down to
pretty much this section of code. As extracted from shm_connect in shm.c:
GFdShm = open(CYGWIN_IPCNT_FILESHM, O_RDWR, 00666 ) ; shareadrshm = (CYGWIN_IPCNT_SHMSTR *) mmap(0,
sizeof(CYGWIN_IPCNT_SHMSTR),PROT_WRITE|PROT_READ, MAP_SHARED, GFdShm, 0) ; if( shareadrshm ==
(CYGWIN_IPCNT_SHMSTR*) -1 ) { close (GFdShm) ; return (0) ; }
So, I guess the question of the day is, how is Cygwin handling mmap()
calls??
Perhaps looking to see if Cygwin can support page file access via
INVALID_HANDLE_VALUE to the *assumed* CreateFileMapping call?? I get
the impression that you already understand this, however, I thought I'd
go ahead and toss this out there. By using the INVALID_HANDLE_VALUE
(0xFFFFFFFF) value during's initial mapped file's creation, it is
actually being backed by the VM's page file rather than a distinct file
layered on top of the FS.
As you said earlier, this may provide for less disk contension, however,
I'm doubtful you're going to find a significant speed improvement. Just
the same, I guess a couple percent here and there can add up.
In case you can't tell, I'm somewhat interested in this thread. When
you get ready to do the implementation, I wouldn't mind kicking the code
around a little. Of course, being that pesky voice that says, "what
about this..." can just be plain fun too. ;)
Greg
On Tue, 26 Feb 2002, Tom Lane wrote: > Oleg Bartunov <oleg@sai.msu.su> writes: > > Having frustrated with performance on Windows box I'm wondering if it's > > possible to get postgresql optimized for working without shared memory, > > say in single-task mode. It looks like it's shared memory emulation on disk > > (by cygipc daemon) is responsible for performance degradation. > > In our project we have to use Windows for desktop application and it's > > single task, so we don't need shared memory. In principle, it's possible > > to hack cygipc, so it wouldn't emulate shared memory and address calls > > to normal memory, but I'm wondering if it's possible from postgres side. > > As mlw comments, that is probably not really the source of the > performance issue. However, you should be able to hack it if you earlier versions of windows doesn't have shared memory and I did see a file on disk when run postgresql on w98. That's why I suggest it's a source of performance drop. > want to check. A standalone backend just malloc()s what would otherwise > be the shared memory area. As a first approximation you could just fire > up a standalone backend and see if it seems any faster. > thanks, it works and doesn't need shared memory, only 2 semaphors. will investigate further. > regards, tom lane > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Tue, 2002-02-26 at 17:52, Oleg Bartunov wrote: > Having frustrated with performance on Windows box I'm wondering if it's > possible to get postgresql optimized for working without shared memory, > say in single-task mode. It looks like it's shared memory emulation on disk > (by cygipc daemon) is responsible for performance degradation. > In our project we have to use Windows for desktop application and it's > single task, so we don't need shared memory. In principle, it's possible > to hack cygipc, so it wouldn't emulate shared memory and address calls > to normal memory, but I'm wondering if it's possible from postgres side. You might also be interested in hacking the threaded verion to run on native windows without cygwin. On Sun, 2002-01-27 at 02:19, mkscott@sacadia.com wrote: > For anyone interested, I have posted a new version of multi-threaded > Postgres 7.0.2 here: > > http://sourceforge.net/projects/mtpgsql > --------------- Hannu
On 27 Feb 2002, Hannu Krosing wrote: > On Tue, 2002-02-26 at 17:52, Oleg Bartunov wrote: > > Having frustrated with performance on Windows box I'm wondering if it's > > possible to get postgresql optimized for working without shared memory, > > say in single-task mode. It looks like it's shared memory emulation on disk > > (by cygipc daemon) is responsible for performance degradation. > > In our project we have to use Windows for desktop application and it's > > single task, so we don't need shared memory. In principle, it's possible > > to hack cygipc, so it wouldn't emulate shared memory and address calls > > to normal memory, but I'm wondering if it's possible from postgres side. > > You might also be interested in hacking the threaded verion to run on > native windows without cygwin. unfortunately it's based on 7.0.2 and doesn't supports dynamic function call. > > On Sun, 2002-01-27 at 02:19, mkscott@sacadia.com wrote: > > For anyone interested, I have posted a new version of multi-threaded > > Postgres 7.0.2 here: > > > > http://sourceforge.net/projects/mtpgsql > > > > --------------- > Hannu > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
-----Original Message----- From: Hannu Krosing [mailto:hannu@krosing.net] Sent: Tuesday, February 26, 2002 9:54 PM To: Oleg Bartunov Cc: Pgsql Hackers Subject: Re: [HACKERS] single task postgresql On Tue, 2002-02-26 at 17:52, Oleg Bartunov wrote: > Having frustrated with performance on Windows box I'm wondering if it's > possible to get postgresql optimized for working without shared memory, > say in single-task mode. It looks like it's shared memory emulation on disk > (by cygipc daemon) is responsible for performance degradation. > In our project we have to use Windows for desktop application and it's > single task, so we don't need shared memory. In principle, it's possible > to hack cygipc, so it wouldn't emulate shared memory and address calls > to normal memory, but I'm wondering if it's possible from postgres side. You might also be interested in hacking the threaded verion to run on native windows without cygwin. On Sun, 2002-01-27 at 02:19, mkscott@sacadia.com wrote: > For anyone interested, I have posted a new version of multi-threaded > Postgres 7.0.2 here: > > http://sourceforge.net/projects/mtpgsql > >> I did a modification of a port of POSIX threads to NT so that I can run UNIX threaded chess programs on NT. I don't know if it will work with the threaded PostgreSQL server, but you are welcome to try. Here is the stuff I did, you are free to use it however you like: ftp://cap.connx.com/pub/amy/pthreads.ZIP <<
On Wed, 27 Feb 2002, mlw wrote: > Oleg Bartunov wrote: > > > > we have to support earlier windows which have no shared memory support. > > I'm pretty sure it could a configuration option then. Using a file is just > plain stupid. "Every" version of Windows has supported shared memory in one > form or another. > > What version are you wishing to support which the cygwin guys claim does not > have shared memory? We've tried windows 98 and would like to have support for ALL versions beginning from W95. > > I'm prety sure I can write a shared memory interface for Windows which should > work across all "supported" versions. > > (BTW I have been a Windows developer since version 1.03.) > great ! We have had success to run our OpenFTS under Windows but need postgresql runs a bit faster. It'd be great if you write such interface, so postgresql could run under older windows without shared memory. btw, BeoS seems is supported in similar way. I think people would appreciate this. There are still many old hardware with Windows95 installed which could be used for desktop application based on postgresql. > > > > > > On Tue, 26 Feb 2002, mlw wrote: > > > > > Oleg Bartunov wrote: > > > > > > > > Having frustrated with performance on Windows box I'm wondering if it's > > > > possible to get postgresql optimized for working without shared memory, > > > > say in single-task mode. It looks like it's shared memory emulation on disk > > > > (by cygipc daemon) is responsible for performance degradation. > > > > In our project we have to use Windows for desktop application and it's > > > > single task, so we don't need shared memory. In principle, it's possible > > > > to hack cygipc, so it wouldn't emulate shared memory and address calls > > > > to normal memory, but I'm wondering if it's possible from postgres side. > > > > > > > > Regards, > > > > > > How does cygwin do shared memory on Windows? Windows support real shared > > > memory, surely the cygwin guys are using "real" shared memory. Are you sure > > > that this is the problem? If so, might we not be able to use a few > > > macros/#ifdefs to do it right? > > > > > > I'll be up for that if you guys don't mind some "ifdef/endif" stuff here and > > > there. > > > > > > > Regards, > > Oleg > > _____________________________________________________________ > > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > > Sternberg Astronomical Institute, Moscow University (Russia) > > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > > phone: +007(095)939-16-83, +007(095)939-23-83 > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Wed, 27 Feb 2002, mlw wrote: > Oleg Bartunov wrote: > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > Greg Copeland wrote: > > > > > > > > Windows does not really have shared memory support. This has been a > > > > beef with the Win32 API for a long time now. Because it has been a long > > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > > and other entities commonly used for concurrent programming strategies > > > > are slower in XP. So, because shared memory really isn't well > > > > supported, they elected to have what is, in essense, memory mapped > > > > files. Multiple processes then map the same file and read/write to it > > > > as needed, more or less as you would shared memory. Unless you plan on > > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > > selector, and the memory is mapped in the above 2G space shared and mapped to > > > all 32bit processes. > > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > > which is the NT equivilent of the system memory file. The handle returned is a > > > system global object which can be shared across processes. > > > > > > > Mark, > > > > do you consider to work on this issue ? > > Yea, let me think about it. What is your time frame? When I offered to work on yesterday :-) I think, at first we need to be sure this way could provide performance win and compatibility. > it, I thought it could be a leasurely thing. I have to get a machine running > some form of Windows on which to develop and test. > > I want to say yes, and if no one else does it, I will, but I'm not sure what > your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If > you need something quickly, I can help, but I don't think I could shoulder the > whole thing. > looks like people will appreciate your work. Currently we're investigating another possibility Tom suggested - standalone backend. But things are still dim, so we'll also track your development. > I have a couple things I have promised people. Let me get those done. I will > try to write an equivilent set of functions for shget, shmat, etc. as soon as I > can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. > > How does that sound? > Loosk great, we could run and test. I'm not sure if we have some funding for this work but I think I could talk with project manager once I'd be sure this way is promising. Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On Wed, 27 Feb 2002, mlw wrote: > Greg Copeland wrote: > > > > Windows does not really have shared memory support. This has been a > > beef with the Win32 API for a long time now. Because it has been a long > > time complaint, it was finally added in Win2000 and later. Likewise, > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > and other entities commonly used for concurrent programming strategies > > are slower in XP. So, because shared memory really isn't well > > supported, they elected to have what is, in essense, memory mapped > > files. Multiple processes then map the same file and read/write to it > > as needed, more or less as you would shared memory. Unless you plan on > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > globalalloc, a shared memory segment. The base selector is a valid 32 bit > selector, and the memory is mapped in the above 2G space shared and mapped to > all 32bit processes. > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > which is the NT equivilent of the system memory file. The handle returned is a > system global object which can be shared across processes. > Mark, do you consider to work on this issue ? > > > > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
Sorry for taking so long to get back to everyone. I wanted to post a follow up to the profiling data that has been submitted as well as comment on the provided link (thank you btw). The profiling data provided had some minor issues with it. It seems that everything was able to run in exactly zero % of overall time. While this doesn't have to mean the results are invalid, it does raise that question. It is certainly possible that the overall % duration for the reported functions was so negligible that it should show up as 0%, however, somewhere in the back of my head I seem to recall that cygwin's profiler is broken in this regard. So, while there are some minor differences, there are no timings to indicate which path in which profiling is consuming the greatest amount of time. Is it possible to run a longer benchmark to see if we can get anything to register with a percent value higher than 0%? At any rate, I'm still wading through the two files to determine if there is yet any value to found within the profiling results. Greg On Wed, 2002-03-06 at 09:36, Oleg Bartunov wrote: > Mark, > > I've found > "Fast synchronized access to shared memory for Windows and for i86 Unix-es" > http://www.ispras.ru/~knizhnik/shmem/Readme.htm > Would't be useful ? > > > Regards, > > Oleg > > > On Wed, 27 Feb 2002, mlw wrote: > > > Oleg Bartunov wrote: > > > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > > > Greg Copeland wrote: > > > > > > > > > > Windows does not really have shared memory support. This has been a > > > > > beef with the Win32 API for a long time now. Because it has been a long > > > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > > > and other entities commonly used for concurrent programming strategies > > > > > are slower in XP. So, because shared memory really isn't well > > > > > supported, they elected to have what is, in essense, memory mapped > > > > > files. Multiple processes then map the same file and read/write to it > > > > > as needed, more or less as you would shared memory. Unless you plan on > > > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > > > selector, and the memory is mapped in the above 2G space shared and mapped to > > > > all 32bit processes. > > > > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > > > which is the NT equivilent of the system memory file. The handle returned is a > > > > system global object which can be shared across processes. > > > > > > > > > > Mark, > > > > > > do you consider to work on this issue ? > > > > Yea, let me think about it. What is your time frame? When I offered to work on > > it, I thought it could be a leasurely thing. I have to get a machine running > > some form of Windows on which to develop and test. > > > > I want to say yes, and if no one else does it, I will, but I'm not sure what > > your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If > > you need something quickly, I can help, but I don't think I could shoulder the > > whole thing. > > > > I have a couple things I have promised people. Let me get those done. I will > > try to write an equivilent set of functions for shget, shmat, etc. as soon as I > > can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. > > > > How does that sound? > > > > Regards, > Oleg >
Hehe. Thank, I just submitted another email about this very issue. Greg On Wed, 2002-03-06 at 12:55, Oleg Bartunov wrote: > On 6 Mar 2002, Greg Copeland wrote: > > > Sorry for taking so long to get back to everyone. I wanted to post a > > follow up to the profiling data that has been submitted as well as > > comment on the provided link (thank you btw). > > > > The profiling data provided had some minor issues with it. It seems > > that everything was able to run in exactly zero % of overall time. > > While this doesn't have to mean the results are invalid, it does raise > > that question. It is certainly possible that the overall % duration for > > the reported functions was so negligible that it should show up as 0%, > > however, somewhere in the back of my head I seem to recall that cygwin's > > profiler is broken in this regard. So, while there are some minor > > differences, there are no timings to indicate which path in which > > profiling is consuming the greatest amount of time. Is it possible to > > run a longer benchmark to see if we can get anything to register with a > > percent value higher than 0%? > > timings doesnt' correct ! > gettimeofday doesn't works properly under Cygwin. It's claimed that > 1.3.10 (we have used 1.3.9) > "- Use QueryPerformance* functions for gettimeofday calls. (cgf)". > We'll rerun tests and submit results. > > btw, Konstatntin Knizhnik suggested that poor performance of postgres > under cygwin+cygipc could be caused by locking. We need to look at > task manager to see how cpu is occupied by postgres. > > > > At any rate, I'm still wading through the two files to determine if > > there is yet any value to found within the profiling results. > > > > Greg > > > > > > On Wed, 2002-03-06 at 09:36, Oleg Bartunov wrote: > > > Mark, > > > > > > I've found > > > "Fast synchronized access to shared memory for Windows and for i86 Unix-es" > > > http://www.ispras.ru/~knizhnik/shmem/Readme.htm > > > Would't be useful ? > > > > > > > > > Regards, > > > > > > Oleg > > > > > > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > > > Oleg Bartunov wrote: > > > > > > > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > > > > > > > Greg Copeland wrote: > > > > > > > > > > > > > > Windows does not really have shared memory support. This has been a > > > > > > > beef with the Win32 API for a long time now. Because it has been a long > > > > > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > > > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > > > > > and other entities commonly used for concurrent programming strategies > > > > > > > are slower in XP. So, because shared memory really isn't well > > > > > > > supported, they elected to have what is, in essense, memory mapped > > > > > > > files. Multiple processes then map the same file and read/write to it > > > > > > > as needed, more or less as you would shared memory. Unless you plan on > > > > > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > > > > > > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > > > > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > > > > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > > > > > selector, and the memory is mapped in the above 2G space shared and mapped to > > > > > > all 32bit processes. > > > > > > > > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > > > > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > > > > > which is the NT equivilent of the system memory file. The handle returned is a > > > > > > system global object which can be shared across processes. > > > > > > > > > > > > > > > > Mark, > > > > > > > > > > do you consider to work on this issue ? > > > > > > > > Yea, let me think about it. What is your time frame? When I offered to work on > > > > it, I thought it could be a leasurely thing. I have to get a machine running > > > > some form of Windows on which to develop and test. > > > > > > > > I want to say yes, and if no one else does it, I will, but I'm not sure what > > > > your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If > > > > you need something quickly, I can help, but I don't think I could shoulder the > > > > whole thing. > > > > > > > > I have a couple things I have promised people. Let me get those done. I will > > > > try to write an equivilent set of functions for shget, shmat, etc. as soon as I > > > > can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. > > > > > > > > How does that sound? > > > > > > > > > > Regards, > > > Oleg > > > > > > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 >
Seems I'm replying rather quickly but I thought I'd point out that I went back and started looking at the cygipc code again. I'm now starting to suspect that that the majority of the performance impact we are seeing has to do with the semaphore implementation versus the shared memory implementation. Basically, the version which is using local memory is not having to contend with a significant amount of semaphore negotiations while the shared memory version must contend with this issue each and every time memory is accessed. It's worth noting that based on what I've seen so far (pointing out that lots still need to be reviewed), the semaphore implementation on via the cygipc library is going to yield an absolute worse case semaphore performance on Win32 platforms. That is, of all the natve synchronization mechanisms available for Win32, the use of a generic Semaphore is going to deliver the worst performance whereby one would expect it to be perhaps an order of magnitude or slower than an ideal Win32 semaphore implementation. So, if you have the time, perhaps you can write some quick benchmarks on Win32. One which simply allocates a shared memory region and randomly reads and writes to it. Same thing for the shared/memory mapped file implementation. Next, create a multi-process implementation based on each of your previous two tests. In these test, try using the cygwin library, native Win32 Mutex implementation (warning, this has some issues because on Win32 Mutexs are optimized for threading and not multi-process implementations), a Win32 Critical section (warning SMP scaling issues -- absolute fastest for uni-processor systems on Win32) and a native Win32 Semaphore implementation (horrible performance). Of course, timing each. If my *guess* is correct, this will tell you that the significant performance issues are directly associated with the use of Semaphores. As I am starting to understand things, it looks like performance suffers because the Win32 platforms are significantly biased toward threaded IPC implementations and significantly suffer when forced into a multi-process architecture (very Unix like). Greg On Wed, 2002-03-06 at 09:36, Oleg Bartunov wrote: > Mark, > > I've found > "Fast synchronized access to shared memory for Windows and for i86 Unix-es" > http://www.ispras.ru/~knizhnik/shmem/Readme.htm > Would't be useful ? > > > Regards, > > Oleg > > > On Wed, 27 Feb 2002, mlw wrote: > > > Oleg Bartunov wrote: > > > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > > > Greg Copeland wrote: > > > > > > > > > > Windows does not really have shared memory support. This has been a > > > > > beef with the Win32 API for a long time now. Because it has been a long > > > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > > > and other entities commonly used for concurrent programming strategies > > > > > are slower in XP. So, because shared memory really isn't well > > > > > supported, they elected to have what is, in essense, memory mapped > > > > > files. Multiple processes then map the same file and read/write to it > > > > > as needed, more or less as you would shared memory. Unless you plan on > > > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > > > selector, and the memory is mapped in the above 2G space shared and mapped to > > > > all 32bit processes. > > > > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > > > which is the NT equivilent of the system memory file. The handle returned is a > > > > system global object which can be shared across processes. > > > > > > > > > > Mark, > > > > > > do you consider to work on this issue ? > > > > Yea, let me think about it. What is your time frame? When I offered to work on > > it, I thought it could be a leasurely thing. I have to get a machine running > > some form of Windows on which to develop and test. > > > > I want to say yes, and if no one else does it, I will, but I'm not sure what > > your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If > > you need something quickly, I can help, but I don't think I could shoulder the > > whole thing. > > > > I have a couple things I have promised people. Let me get those done. I will > > try to write an equivilent set of functions for shget, shmat, etc. as soon as I > > can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. > > > > How does that sound? > > > > Regards, > Oleg > _____________________________________________________________ > Oleg Bartunov, sci.researcher, hostmaster of AstroNet, > Sternberg Astronomical Institute, Moscow University (Russia) > Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ > phone: +007(095)939-16-83, +007(095)939-23-83 >
Mark, I've found "Fast synchronized access to shared memory for Windows and for i86 Unix-es" http://www.ispras.ru/~knizhnik/shmem/Readme.htm Would't be useful ? Regards, Oleg On Wed, 27 Feb 2002, mlw wrote: > Oleg Bartunov wrote: > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > Greg Copeland wrote: > > > > > > > > Windows does not really have shared memory support. This has been a > > > > beef with the Win32 API for a long time now. Because it has been a long > > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > > and other entities commonly used for concurrent programming strategies > > > > are slower in XP. So, because shared memory really isn't well > > > > supported, they elected to have what is, in essense, memory mapped > > > > files. Multiple processes then map the same file and read/write to it > > > > as needed, more or less as you would shared memory. Unless you plan on > > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > > selector, and the memory is mapped in the above 2G space shared and mapped to > > > all 32bit processes. > > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > > which is the NT equivilent of the system memory file. The handle returned is a > > > system global object which can be shared across processes. > > > > > > > Mark, > > > > do you consider to work on this issue ? > > Yea, let me think about it. What is your time frame? When I offered to work on > it, I thought it could be a leasurely thing. I have to get a machine running > some form of Windows on which to develop and test. > > I want to say yes, and if no one else does it, I will, but I'm not sure what > your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If > you need something quickly, I can help, but I don't think I could shoulder the > whole thing. > > I have a couple things I have promised people. Let me get those done. I will > try to write an equivilent set of functions for shget, shmat, etc. as soon as I > can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. > > How does that sound? > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83
On 6 Mar 2002, Greg Copeland wrote: > Sorry for taking so long to get back to everyone. I wanted to post a > follow up to the profiling data that has been submitted as well as > comment on the provided link (thank you btw). > > The profiling data provided had some minor issues with it. It seems > that everything was able to run in exactly zero % of overall time. > While this doesn't have to mean the results are invalid, it does raise > that question. It is certainly possible that the overall % duration for > the reported functions was so negligible that it should show up as 0%, > however, somewhere in the back of my head I seem to recall that cygwin's > profiler is broken in this regard. So, while there are some minor > differences, there are no timings to indicate which path in which > profiling is consuming the greatest amount of time. Is it possible to > run a longer benchmark to see if we can get anything to register with a > percent value higher than 0%? timings doesnt' correct ! gettimeofday doesn't works properly under Cygwin. It's claimed that 1.3.10 (we have used 1.3.9) "- Use QueryPerformance* functions for gettimeofday calls. (cgf)". We'll rerun tests and submit results. btw, Konstatntin Knizhnik suggested that poor performance of postgres under cygwin+cygipc could be caused by locking. We need to look at task manager to see how cpu is occupied by postgres. > > At any rate, I'm still wading through the two files to determine if > there is yet any value to found within the profiling results. > > Greg > > > On Wed, 2002-03-06 at 09:36, Oleg Bartunov wrote: > > Mark, > > > > I've found > > "Fast synchronized access to shared memory for Windows and for i86 Unix-es" > > http://www.ispras.ru/~knizhnik/shmem/Readme.htm > > Would't be useful ? > > > > > > Regards, > > > > Oleg > > > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > Oleg Bartunov wrote: > > > > > > > > On Wed, 27 Feb 2002, mlw wrote: > > > > > > > > > Greg Copeland wrote: > > > > > > > > > > > > Windows does not really have shared memory support. This has been a > > > > > > beef with the Win32 API for a long time now. Because it has been a long > > > > > > time complaint, it was finally added in Win2000 and later. Likewise, > > > > > > I'd like to point out that thinks like sims, shared memory, pipes, etc, > > > > > > and other entities commonly used for concurrent programming strategies > > > > > > are slower in XP. So, because shared memory really isn't well > > > > > > supported, they elected to have what is, in essense, memory mapped > > > > > > files. Multiple processes then map the same file and read/write to it > > > > > > as needed, more or less as you would shared memory. Unless you plan on > > > > > > only targetting on Win 2000 and XP, it sounds like a waste of time. > > > > > > > > > > This is not really true. Under DOS windows, i.e. 95,98, etc. Shared memory can > > > > > be done in 16 bit land with a touch of assembly and a DLL. Allocate, with > > > > > globalalloc, a shared memory segment. The base selector is a valid 32 bit > > > > > selector, and the memory is mapped in the above 2G space shared and mapped to > > > > > all 32bit processes. > > > > > > > > > > Under NT through 2K, yes using a memory mapped files is the way to do it, but > > > > > you do not actually need to create a file, you can use (HANDLE)0xFFFFFFFF, > > > > > which is the NT equivilent of the system memory file. The handle returned is a > > > > > system global object which can be shared across processes. > > > > > > > > > > > > > Mark, > > > > > > > > do you consider to work on this issue ? > > > > > > Yea, let me think about it. What is your time frame? When I offered to work on > > > it, I thought it could be a leasurely thing. I have to get a machine running > > > some form of Windows on which to develop and test. > > > > > > I want to say yes, and if no one else does it, I will, but I'm not sure what > > > your timeframe is. If it is the mystical 7.3, then sure I can do it easily. If > > > you need something quickly, I can help, but I don't think I could shoulder the > > > whole thing. > > > > > > I have a couple things I have promised people. Let me get those done. I will > > > try to write an equivilent set of functions for shget, shmat, etc. as soon as I > > > can. Anyone wanting to run with them can hack and test PostgreSQL on Windows. > > > > > > How does that sound? > > > > > > > Regards, > > Oleg > > > > Regards, Oleg _____________________________________________________________ Oleg Bartunov, sci.researcher, hostmaster of AstroNet, Sternberg Astronomical Institute, Moscow University (Russia) Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/ phone: +007(095)939-16-83, +007(095)939-23-83