Thread: [PATCH] PostgreSQL fails to build with 32bit MinGW-w64
Hi PostgreSQL hackers,
support for Mingw-w64 compiler was added to postgres with commit 91812df. Unfortunately only the 64 bit output is working right now. This issue was already highlighted with initial patch in
http://archives.postgresql.org/pgsql-bugs/2011-07/msg00059.php
Mingw-w64 uses the same header files for 32 and 64 bit compiles. So the same conditions apply to mingw-w32 bit as for the WIN64 case. In WIN64 "WSAAPI" is defined to nothing, but in 32 bit to stdcall, so it needs to be used in the accept-parameter check, too. Maybe you prefer PASCAL instead of WSAAPI in configure.
I tested successful compilation for the following platforms:
- i686-w64-mingw32 - gcc v4.6.1
- x86_64-w64-mingw32 - gcc v4.6.1
- i586-mingw32msvc - gcc v4.4.4
- x86_64-linux-gnu - gcc v4.6.1
--
Kind regards,
Lars Kanis
Attachment
Hi, Isn't it better to check the value of macros itsef rather than checking for system dependent macros that does not directly relate to the issue? specifically for getaddrinfo.c case I think #if EAI_NODATA != EAI_NONAME is a better check than checking for #if !defined(__MINGW64_VERSION_MAJOR) && !defined(WIN32_ONLY_COMPILER) /* MSVC/WIN64 duplicate */ For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before <winsock2.h> some google suggests that defining _WINSOCKAPI_ before <windows.h> prevents inclusion of winsock.h but that does not have relation to inclusion of <winsock2.h> and if <winsock2.h> is included first, it should be ok. If this guess is right, perhaps it could be better to remove the three lines. #if !defined(WIN64) || defined(WIN32_ONLY_COMPILER) #define _WINSOCKAPI_ #endif +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by + * mingw-w64, however it gots defined only after Why not use __MINGW32__, which is defined without including any headers? On 2011/11/24, at 17:24, Lars Kanis wrote: > Hi PostgreSQL hackers, > > support for Mingw-w64 compiler was added to postgres with commit 91812df. Unfortunately only the 64 bit output is workingright now. This issue was already highlighted with initial patch in > http://archives.postgresql.org/pgsql-bugs/2011-07/msg00059.php > > Mingw-w64 uses the same header files for 32 and 64 bit compiles. So the same conditions apply to mingw-w32 bit as for theWIN64 case. In WIN64 "WSAAPI" is defined to nothing, but in 32 bit to stdcall, so it needs to be used in the accept-parametercheck, too. Maybe you prefer PASCAL instead of WSAAPI in configure. > > I tested successful compilation for the following platforms: > - i686-w64-mingw32 - gcc v4.6.1 > - x86_64-w64-mingw32 - gcc v4.6.1 > - i586-mingw32msvc - gcc v4.4.4 > - x86_64-linux-gnu - gcc v4.6.1 > > -- > Kind regards, > Lars Kanis > > <fix-mingw-w64-32bit_v1.patch> > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers
Hello, Lars. You wrote: LK> Hi PostgreSQL hackers, LK> LK> support for Mingw-w64 compiler was added to postgres with commit LK> 91812df. Unfortunately only the 64 bit output is working right LK> now. This issue was already highlighted with initial patch in LK> http://archives.postgresql.org/pgsql-bugs/2011-07/msg00059.php LK> LK> Mingw-w64 uses the same header files for 32 and 64 bit compiles. LK> So the same conditions apply to mingw-w32 bit as for the WIN64 LK> case. In WIN64 "WSAAPI" is defined to nothing, but in 32 bit to LK> stdcall, so it needs to be used in the accept-parameter check, LK> too. Maybe you prefer PASCAL instead of WSAAPI in configure. LK> LK> I tested successful compilation for the following platforms: LK> - i686-w64-mingw32 - gcc v4.6.1 LK> - x86_64-w64-mingw32 - gcc v4.6.1 LK> - i586-mingw32msvc - gcc v4.4.4 LK> - x86_64-linux-gnu - gcc v4.6.1 LK> Can you please provide me with some howto on building PG sources with mingw-w64? -- With best wishes,Pavel mailto:pavel@gf.microolap.com
Hi, 2011-11-24 09:55 keltezéssel, Pavel Golub írta: > > Can you please provide me with some howto on building PG sources with > mingw-w64? Install Fedora 15 or 16, add this repo file into /etc/yum.repos.d : http://build1.openftd.org/fedora-cross/fedora-cross.repo Then "yum install mingw*". This will install mingw64 both 32 and 64-bit. Replaces a lot of official mingw32 packages from Fedora. Then you can do for 32-bit: cd postgresql-9.1.1 mingw32-configure mingw32-make sudo mingw32-make install or for 64-bit: mingw64-configure mingw64-make sudo mingw64-make install Best regards, Zoltán Böszörményi -- ---------------------------------- Zoltán Böszörményi Cybertec Schönig & Schönig GmbH Gröhrmühlgasse 26 A-2700 Wiener Neustadt, Austria Web: http://www.postgresql-support.de http://www.postgresql.at/
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">>Can you please provide me with some howto on building PG sources with<p style=" margin-top:0px; margin-bottom:0px;margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">> mingw-w64?<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0;text-indent:0px; "> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0;text-indent:0px; -qt-user-state:0;">For 32/64 bit mingw-v4.6.1 on ubuntu 11.10:<p style=" margin-top:0px;margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">apt-getinstall flex gcc-mingw-w64<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px;-qt-block-indent:0; text-indent:0px; -qt-user-state:0;">./configure --host=<span style=" color:#1f1c1b;background-color:#ffffff;">i686-w64-mingw32</span> --build=x86_64-linux --without-zlib && make<p style="margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">and<pstyle=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0;text-indent:0px; -qt-user-state:0;">./configure --host=<span style=" color:#1f1c1b; background-color:#ffffff;">x86_64-w64-mingw32</span>--build=x86_64-linux --without-zlib && make<p style="-qt-paragraph-type:empty;margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0;text-indent:0px; "> <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0;text-indent:0px; -qt-user-state:0;">For 32 bit mingw-v4.4.4 on ubuntu 11.10:<p style=" margin-top:0px;margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">apt-getinstall flex gcc-mingw32<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px;-qt-block-indent:0; text-indent:0px; -qt-user-state:0;">./configure --host=i586-mingw32msvc --build=x86_64-linux--without-zlib && make<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px;margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> <p style=" margin-top:0px; margin-bottom:0px;margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Regards,<p style="margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">LarsKanis<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px;-qt-block-indent:0; text-indent:0px; ">
> Isn't it better to check the value of macros itsef rather than checking for > system dependent macros that does not directly relate to the issue? > specifically for getaddrinfo.c case I think > #if EAI_NODATA != EAI_NONAME > is a better check than checking for > #if !defined(__MINGW64_VERSION_MAJOR) && !defined(WIN32_ONLY_COMPILER) /* MSVC/WIN64 duplicate */ Yes it's better and it works for all described test environments. > For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before > <winsock2.h> > some google suggests that defining _WINSOCKAPI_ before <windows.h> prevents > inclusion of winsock.h but that does not have relation to inclusion of > <winsock2.h> and if <winsock2.h> is included first, it should be ok. > > If this guess is right, perhaps it could be better to remove the three lines. > #if !defined(WIN64) || defined(WIN32_ONLY_COMPILER) > #define _WINSOCKAPI_ > #endif I only changed this for consistency. For me, it works without that define in all test environments, too. > +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by > + * mingw-w64, however it gots defined only after > Why not use __MINGW32__, which is defined without including any headers? At least in mingw32 v4.4.4 there is no crtdefs.h. I couldn't find a proper define that relates directly to that issue, so attached is a somewhat cumbersome MINGW version check. -- Regards, Lars Kanis
Attachment
On 11/24/2011 04:39 AM, Lars Kanis wrote: > > > Can you please provide me with some howto on building PG sources with > > > mingw-w64? > > For 32/64 bit mingw-v4.6.1 on ubuntu 11.10: > > apt-get install flex gcc-mingw-w64 > > ./configure --host=i686-w64-mingw32 --build=x86_64-linux > --without-zlib && make > > and > > ./configure --host=x86_64-w64-mingw32 --build=x86_64-linux > --without-zlib && make > > For 32 bit mingw-v4.4.4 on ubuntu 11.10: > > apt-get install flex gcc-mingw32 > > ./configure --host=i586-mingw32msvc --build=x86_64-linux > --without-zlib && make > > The only thing I at least am going to be interested in supporting is a native compiler, not a cross-compiler. It's impossible to automate cross-compiler testing. The way to build natively with a mingw-w64 compiler is doumented fairly simply at <http://www.postgresql.org/docs/current/static/installation-platform-notes.html#INSTALLATION-NOTES-MINGW>: To build 64 bit binaries using MinGW, install the 64 bit tool set from http://mingw-w64.sourceforge.net/, put its bindirectory in the PATH, and run configure with the --host=x86_64-w64-mingw option. This is exactly how the buildfarm member pitta was set up. The only issue here is how to add support for using the 32bit mingw-w64 compiler in pretty much the same way. cheers andrew
Hi, > The way to build natively with a mingw-w64 compiler is doumented fairly simply at <http://www.postgresql.org/docs/current/static/installation-platform-notes.html#INSTALLATION-NOTES-MINGW>: > > To build 64 bit binaries using MinGW, install the 64 bit tool set > from http://mingw-w64.sourceforge.net/, put its bin directory in the > PATH, and run configure with the --host=x86_64-w64-mingw option. > > This is exactly how the buildfarm member pitta was set up. > > The only issue here is how to add support for using the 32bit mingw-w64 compiler in pretty much the same way. I hope that configure --host=i686-w64-mingw use 32 bit tool sets and construct a 32 bit binary. It is mostly the same procedure and should have sufficient information to distinguish them. Both 64 bit and 32 bit binaries should be runnable on the same 64 bit system. Best regards,
On 11/24/2011 06:29 AM, Lars Kanis wrote: >> Isn't it better to check the value of macros itsef rather than checking for >> system dependent macros that does not directly relate to the issue? >> specifically for getaddrinfo.c case I think >> #if EAI_NODATA != EAI_NONAME >> is a better check than checking for >> #if !defined(__MINGW64_VERSION_MAJOR)&& !defined(WIN32_ONLY_COMPILER) /* MSVC/WIN64 duplicate */ > Yes it's better and it works for all described test environments. > >> For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before >> <winsock2.h> >> some google suggests that defining _WINSOCKAPI_ before<windows.h> prevents >> inclusion of winsock.h but that does not have relation to inclusion of >> <winsock2.h> and if<winsock2.h> is included first, it should be ok. >> >> If this guess is right, perhaps it could be better to remove the three lines. >> #if !defined(WIN64) || defined(WIN32_ONLY_COMPILER) >> #define _WINSOCKAPI_ >> #endif No, this broke some compilers, IIRC (probably the native mingw compiler, which is in use by several buildfarm members). Getting this right was very tricky and time-consuming when I was adding support for the 64 bit mingw-w64 compiler, and there were a couple of rounds of breakage. I'm therefore much more inclined to go the way of your earlier patch, which seems much less risky. > I only changed this for consistency. For me, it works without that define in all test > environments, too. > >> +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by >> + * mingw-w64, however it gots defined only after >> Why not use __MINGW32__, which is defined without including any headers? Because it's defined by other than mingw-w64 compilers. We have a bunch of compilers to support here. There are LOTS of compiler scenarios on Windows (several versions of MSVC, 32bit and 64bit mingw-w64, native mingw gcc, and a couple of Cygwin based compilers), and keeping track of them all and making sure they don't break can be a pain. cheers andrew
Hi, >>> For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before >>> <winsock2.h> >>> some google suggests that defining _WINSOCKAPI_ before<windows.h> prevents >>> inclusion of winsock.h but that does not have relation to inclusion of >>> <winsock2.h> and if<winsock2.h> is included first, it should be ok. >>> >>> If this guess is right, perhaps it could be better to remove the three lines. >>> #if !defined(WIN64) || defined(WIN32_ONLY_COMPILER) >>> #define _WINSOCKAPI_ >>> #endif > > No, this broke some compilers, IIRC (probably the native mingw compiler, which is in use by several buildfarm members).Getting this right was very tricky and time-consuming when I was adding support for the 64 bit mingw-w64 compiler,and there were a couple of rounds of breakage. > > I'm therefore much more inclined to go the way of your earlier patch, which seems much less risky. I agree that original patch could be less risky. However, it doesn't match what Microsoft says: http://msdn.microsoft.com/en-us/library/windows/desktop/ms737629%28v=vs.85%29.aspx So, I think the standard way is not defining _WINSOCKAPI_, and if any compiler requires that, which I think unlikely, then it should be defined as a exceptional case. At least, native mingw GCC 4.5.2 (20110802 catalogue) and 4.6.1 (latest catalogue) does compile happily without the three lines. >> I only changed this for consistency. For me, it works without that define in all test >> environments, too. >> >>> +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by >>> + * mingw-w64, however it gots defined only after >>> Why not use __MINGW32__, which is defined without including any headers? > > Because it's defined by other than mingw-w64 compilers. I see. That's because mingw (not -w64). Should it be ok if mingw is ok with that condition? > We have a bunch of compilers to support here. There are LOTS of compiler scenarios on Windows (several versions of MSVC,32bit and 64bit mingw-w64, native mingw gcc, and a couple of Cygwin based compilers), and keeping track of them alland making sure they don't break can be a pain. Yes, that really is a pain. The code block #if _MSC_VER >= 1400 || defined(WIN64) #define errcode __msvc_errcode #include <crtdefs.h> #undef errcode #endif looks as if there is no real need to crtdefs.h but, they wanted to prevent definition of errcode and therefor put to the first place. So, I was afraid moving the include downwards might break by by including a header that internally includes crtdefs.h. If this is not problematic for MSVC (perhaps you know better on that), I have no objection in moving the order.
Hi, >>>> +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by >>>> + * mingw-w64, however it gots defined only after >>>> Why not use __MINGW32__, which is defined without including any headers? >> >> Because it's defined by other than mingw-w64 compilers. > > I see. That's because mingw (not -w64). > Should it be ok if mingw is ok with that condition? This really breaks mingw gcc 4.6.1 :( it does not have crtdefs.h) If moving downward do not break MSVC, perhaps its the good way. Otherwise, we might check for the presence of crtdefs.h with configure?
On 11/27/2011 09:18 AM, NISHIYAMA Tomoaki wrote: > Hi, > >>>>> +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by >>>>> + * mingw-w64, however it gots defined only after >>>>> Why not use __MINGW32__, which is defined without including any headers? >>> Because it's defined by other than mingw-w64 compilers. >> I see. That's because mingw (not -w64). >> Should it be ok if mingw is ok with that condition? > > This really breaks mingw gcc 4.6.1 :( it does not have crtdefs.h) > If moving downward do not break MSVC, perhaps its the good way. > Otherwise, we might check for the presence of crtdefs.h with configure? > I have looked at this a bit. It's fairly ugly, and the only moderately clean way I see forward is a configure test to check for a mingw-w64 compiler, e.g. by running gcc -E -dM over a file which just includes stdio.h and checking for the definedness of __MINGW64__VERSION_MAJOR, or something similar. cheers andrew
Hi, A new patch: check for the presence of crtdefs.h in configure -#if _MSC_VER >= 1400 || defined(WIN64) +#if _MSC_VER >= 1400 || HAVE_CRTDEFS_H #define errcode __msvc_errcode #include <crtdefs.h> #undef errcode Perhaps there is no guarantee that mingw (not -w64) may not have crtdefs.h in the future versions. the 3 lines #define errcode __msvc_errcode #include <crtdefs.h> #undef errcode should be valid as far as crtdefs.h exists and errcode is not defined previously. Because this is the first system include file, we can be sure errcode is not defined by this point. So, I believe its better to just test for the presence of crtdefs.h rather than try to figure out the name of compiler. check for fseeko and ftello macro definition before defining to avoid warning in mingw-w64 4.7.0 20110827 This patch was tested to build successfully on mingw gcc version 4.6.1 mingw-w64 i686-w64-mingw32 gcc version 4.5.4 20110812 mingw-w64 x86_64-w64-mingw32 gcc version 4.7.0 20110827 On 2011/12/02, at 1:29, Andrew Dunstan wrote: > > > On 11/27/2011 09:18 AM, NISHIYAMA Tomoaki wrote: >> Hi, >> >>>>>> +/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by >>>>>> + * mingw-w64, however it gots defined only after >>>>>> Why not use __MINGW32__, which is defined without including any headers? >>>> Because it's defined by other than mingw-w64 compilers. >>> I see. That's because mingw (not -w64). >>> Should it be ok if mingw is ok with that condition? >> >> This really breaks mingw gcc 4.6.1 :( it does not have crtdefs.h) >> If moving downward do not break MSVC, perhaps its the good way. >> Otherwise, we might check for the presence of crtdefs.h with configure? >> > > > I have looked at this a bit. It's fairly ugly, and the only moderately clean way I see forward is a configure test to checkfor a mingw-w64 compiler, e.g. by running gcc -E -dM over a file which just includes stdio.h and checking for the definednessof __MINGW64__VERSION_MAJOR, or something similar. > > > cheers > > andrew > > > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Attachment
On Sat, Dec 3, 2011 at 09:24, NISHIYAMA Tomoaki <tomoakin@staff.kanazawa-u.ac.jp> wrote: > Hi, > > A new patch: > check for the presence of crtdefs.h in configure > > -#if _MSC_VER >= 1400 || defined(WIN64) > +#if _MSC_VER >= 1400 || HAVE_CRTDEFS_H > #define errcode __msvc_errcode > #include <crtdefs.h> > #undef errcode Have you verified if tihs affects _MSC_VER < 1400? Suddently that branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's something we need to worry about. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Hi, > Have you verified if tihs affects _MSC_VER < 1400? Suddently that > branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's > something we need to worry about. I have no MSVC. In that sense it is not verified in fact, and I hope those who knows well would kindly comment on it. However, it appears that pg_config.h is not created through configure, but just copied from pg_config.h.win32 in those compilers and thus HAVE_CRTDEFS_H will not be defined. So, I think this code fragment will not be enabled in _MSC_VER < 1400 In addition, the code fragment should have no harm as far as they have crtdefs.h. (Again, this is I think it should, and not tested with real tools.) On 2011/12/03, at 23:20, Magnus Hagander wrote: > On Sat, Dec 3, 2011 at 09:24, NISHIYAMA Tomoaki > <tomoakin@staff.kanazawa-u.ac.jp> wrote: >> Hi, >> >> A new patch: >> check for the presence of crtdefs.h in configure >> >> -#if _MSC_VER >= 1400 || defined(WIN64) >> +#if _MSC_VER >= 1400 || HAVE_CRTDEFS_H >> #define errcode __msvc_errcode >> #include <crtdefs.h> >> #undef errcode > > Have you verified if tihs affects _MSC_VER < 1400? Suddently that > branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's > something we need to worry about. > > -- > Magnus Hagander > Me: http://www.hagander.net/ > Work: http://www.redpill-linpro.com/ > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki <tomoakin@staff.kanazawa-u.ac.jp> wrote: > Hi, > >> Have you verified if tihs affects _MSC_VER < 1400? Suddently that >> branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's >> something we need to worry about. > > > I have no MSVC. In that sense it is not verified in fact, and I hope > those who knows well would kindly comment on it. > > However, it appears that pg_config.h is not created through > configure, but just copied from pg_config.h.win32 in those > compilers and thus HAVE_CRTDEFS_H will not be defined. > So, I think this code fragment will not be enabled in > _MSC_VER < 1400 Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it likely isn't - I was confusing it with the kind of defines that MSVC tends to stick in their own headerfiles, and thought that's what you were testing for. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On 12/03/2011 09:59 AM, Magnus Hagander wrote: > On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki > <tomoakin@staff.kanazawa-u.ac.jp> wrote: >> Hi, >> >>> Have you verified if tihs affects _MSC_VER< 1400? Suddently that >>> branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's >>> something we need to worry about. >> >> I have no MSVC. In that sense it is not verified in fact, and I hope >> those who knows well would kindly comment on it. >> >> However, it appears that pg_config.h is not created through >> configure, but just copied from pg_config.h.win32 in those >> compilers and thus HAVE_CRTDEFS_H will not be defined. >> So, I think this code fragment will not be enabled in >> _MSC_VER< 1400 > Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it > likely isn't - I was confusing it with the kind of defines that MSVC > tends to stick in their own headerfiles, and thought that's what you > were testing for. > Yes, but there's a deal more work to do here. This whole thing is falling over in my build environment (64 bit Windows 7, MinGW/MSys, the machine that runs pitta on the buildfarm.) This is a long way from a done deal. cheers andrew
On 12/03/2011 06:12 PM, Andrew Dunstan wrote: > > > On 12/03/2011 09:59 AM, Magnus Hagander wrote: >> On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki >> <tomoakin@staff.kanazawa-u.ac.jp> wrote: >>> Hi, >>> >>>> Have you verified if tihs affects _MSC_VER< 1400? Suddently that >>>> branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's >>>> something we need to worry about. >>> >>> I have no MSVC. In that sense it is not verified in fact, and I hope >>> those who knows well would kindly comment on it. >>> >>> However, it appears that pg_config.h is not created through >>> configure, but just copied from pg_config.h.win32 in those >>> compilers and thus HAVE_CRTDEFS_H will not be defined. >>> So, I think this code fragment will not be enabled in >>> _MSC_VER< 1400 >> Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it >> likely isn't - I was confusing it with the kind of defines that MSVC >> tends to stick in their own headerfiles, and thought that's what you >> were testing for. >> > > > Yes, but there's a deal more work to do here. This whole thing is > falling over in my build environment (64 bit Windows 7, MinGW/MSys, > the machine that runs pitta on the buildfarm.) > > This is a long way from a done deal. In particular, it's a major mess because it does this (or at least the version I'm using does): #define stat _stat64 which plays merry hell with pgwin32_safestat(). Working around that looks very unpleasant indeed. cheers andrew
Hi, On 2011/12/04, at 9:45, Andrew Dunstan wrote: >> Yes, but there's a deal more work to do here. This whole thing is falling over in my build environment (64 bit Windows7, MinGW/MSys, the machine that runs pitta on the buildfarm.) >> >> This is a long way from a done deal. > > In particular, it's a major mess because it does this (or at least the version I'm using does): > > #define stat _stat64 > > which plays merry hell with pgwin32_safestat(). Working around that looks very unpleasant indeed. Thank you for testing and reporting. Would you mind giving me a bit more information on the environment? --especially for the MinGW/MSYS versions and any other component that is required. I tested on a virtual machine running 64 bit Windows 7 SP1, installing MinGW/MSYS in the clean state and then compile. The versions coming with pre-packaged repository catalogues of 20110802 (gcc 4.5.2) and latest catalogue (gcc-4.6.1-2) compiles successfully. On 2011/12/04, at 9:45, Andrew Dunstan wrote: > > > On 12/03/2011 06:12 PM, Andrew Dunstan wrote: >> >> >> On 12/03/2011 09:59 AM, Magnus Hagander wrote: >>> On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki >>> <tomoakin@staff.kanazawa-u.ac.jp> wrote: >>>> Hi, >>>> >>>>> Have you verified if tihs affects _MSC_VER< 1400? Suddently that >>>>> branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's >>>>> something we need to worry about. >>>> >>>> I have no MSVC. In that sense it is not verified in fact, and I hope >>>> those who knows well would kindly comment on it. >>>> >>>> However, it appears that pg_config.h is not created through >>>> configure, but just copied from pg_config.h.win32 in those >>>> compilers and thus HAVE_CRTDEFS_H will not be defined. >>>> So, I think this code fragment will not be enabled in >>>> _MSC_VER< 1400 >>> Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it >>> likely isn't - I was confusing it with the kind of defines that MSVC >>> tends to stick in their own headerfiles, and thought that's what you >>> were testing for. >>> >> >> >> Yes, but there's a deal more work to do here. This whole thing is falling over in my build environment (64 bit Windows7, MinGW/MSys, the machine that runs pitta on the buildfarm.) >> >> This is a long way from a done deal. > > > In particular, it's a major mess because it does this (or at least the version I'm using does): > > #define stat _stat64 > > > which plays merry hell with pgwin32_safestat(). Working around that looks very unpleasant indeed. > > > cheers > > andrew > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Hi, I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32) gcc version 4.7.0 20111203 (experimental) (GCC) The code can be compiled with diff --git a/src/include/port.h b/src/include/port.h index eceb4bf..78d5c92 100644 --- a/src/include/port.h +++ b/src/include/port.h @@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir); * Some frontends don't need the size from stat,so if UNSAFE_STAT_OK * is defined we don't bother with this. */ -#if defined(WIN32) && !defined(__CYGWIN__) && !defined(UNSAFE_STAT_OK) +#if defined(WIN32_ONLY_COMPILER) && !defined(UNSAFE_STAT_OK)#include <sys/stat.h>extern int pgwin32_safestat(const char*path, struct stat * buf); but, surely we need to know if it is ok or not as the comment before says:* stat() is not guaranteed to set the st_size field on win32, so we* redefine it to our own implementationthat is. Is there any simple test program that determines if the pgwin32_safestat is required or the library stat is sufficient? I presume the stat is a library function and therefore it depends on the compiler rather than the WIN32 platform as a whole. On 2011/12/04, at 12:55, NISHIYAMA Tomoaki wrote: > Hi, > > On 2011/12/04, at 9:45, Andrew Dunstan wrote: >>> Yes, but there's a deal more work to do here. This whole thing is falling over in my build environment (64 bit Windows7, MinGW/MSys, the machine that runs pitta on the buildfarm.) >>> >>> This is a long way from a done deal. >> >> In particular, it's a major mess because it does this (or at least the version I'm using does): >> >> #define stat _stat64 >> >> which plays merry hell with pgwin32_safestat(). Working around that looks very unpleasant indeed. > > > Thank you for testing and reporting. > Would you mind giving me a bit more information on the environment? > --especially for the MinGW/MSYS versions and any other component that is required. > > I tested on a virtual machine running 64 bit Windows 7 SP1, > installing MinGW/MSYS in the clean state and then compile. > The versions coming with pre-packaged repository catalogues of 20110802 > (gcc 4.5.2) and latest catalogue (gcc-4.6.1-2) compiles successfully. > > > On 2011/12/04, at 9:45, Andrew Dunstan wrote: > >> >> >> On 12/03/2011 06:12 PM, Andrew Dunstan wrote: >>> >>> >>> On 12/03/2011 09:59 AM, Magnus Hagander wrote: >>>> On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki >>>> <tomoakin@staff.kanazawa-u.ac.jp> wrote: >>>>> Hi, >>>>> >>>>>> Have you verified if tihs affects _MSC_VER< 1400? Suddently that >>>>>> branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's >>>>>> something we need to worry about. >>>>> >>>>> I have no MSVC. In that sense it is not verified in fact, and I hope >>>>> those who knows well would kindly comment on it. >>>>> >>>>> However, it appears that pg_config.h is not created through >>>>> configure, but just copied from pg_config.h.win32 in those >>>>> compilers and thus HAVE_CRTDEFS_H will not be defined. >>>>> So, I think this code fragment will not be enabled in >>>>> _MSC_VER< 1400 >>>> Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it >>>> likely isn't - I was confusing it with the kind of defines that MSVC >>>> tends to stick in their own headerfiles, and thought that's what you >>>> were testing for. >>>> >>> >>> >>> Yes, but there's a deal more work to do here. This whole thing is falling over in my build environment (64 bit Windows7, MinGW/MSys, the machine that runs pitta on the buildfarm.) >>> >>> This is a long way from a done deal. >> >> >> In particular, it's a major mess because it does this (or at least the version I'm using does): >> >> #define stat _stat64 >> >> >> which plays merry hell with pgwin32_safestat(). Working around that looks very unpleasant indeed. >> >> >> cheers >> >> andrew >> >> -- >> Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgsql-hackers >> > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
On Sun, Dec 4, 2011 at 09:14, NISHIYAMA Tomoaki <tomoakin@staff.kanazawa-u.ac.jp> wrote: > Hi, > > I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32) > gcc version 4.7.0 20111203 (experimental) (GCC) > > The code can be compiled with > > diff --git a/src/include/port.h b/src/include/port.h > index eceb4bf..78d5c92 100644 > --- a/src/include/port.h > +++ b/src/include/port.h > @@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir); > * Some frontends don't need the size from stat, so if UNSAFE_STAT_OK > * is defined we don't bother with this. > */ > -#if defined(WIN32) && !defined(__CYGWIN__) && !defined(UNSAFE_STAT_OK) > +#if defined(WIN32_ONLY_COMPILER) && !defined(UNSAFE_STAT_OK) > #include <sys/stat.h> > extern int pgwin32_safestat(const char *path, struct stat * buf); > > but, surely we need to know if it is ok or not > as the comment before says: > * stat() is not guaranteed to set the st_size field on win32, so we > * redefine it to our own implementation that is. > > Is there any simple test program that determines if the pgwin32_safestat > is required or the library stat is sufficient? > I presume the stat is a library function and therefore it depends on the > compiler rather than the WIN32 platform as a whole. No, stat() is unreliable because it is implemented on top of FindNextFile(), and *that's* where the actual problem is at. And that's an OS API function, not a library function. See the discussion at http://archives.postgresql.org/pgsql-hackers/2008-03/msg01181.php In theory, if mingw implemented their stat() without using FindNextFile(), it might work - but I don't see how they'd do it in that case. And I can't see us going into details to remove such a simple workaround even if they do - it's better to ensure we work the same way with different compilers. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On 12/04/2011 06:31 AM, Magnus Hagander wrote: > On Sun, Dec 4, 2011 at 09:14, NISHIYAMA Tomoaki > <tomoakin@staff.kanazawa-u.ac.jp> wrote: >> Hi, >> >> I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32) >> gcc version 4.7.0 20111203 (experimental) (GCC) >> >> The code can be compiled with >> >> diff --git a/src/include/port.h b/src/include/port.h >> index eceb4bf..78d5c92 100644 >> --- a/src/include/port.h >> +++ b/src/include/port.h >> @@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir); >> * Some frontends don't need the size from stat, so if UNSAFE_STAT_OK >> * is defined we don't bother with this. >> */ >> -#if defined(WIN32)&& !defined(__CYGWIN__)&& !defined(UNSAFE_STAT_OK) >> +#if defined(WIN32_ONLY_COMPILER)&& !defined(UNSAFE_STAT_OK) >> #include<sys/stat.h> >> extern int pgwin32_safestat(const char *path, struct stat * buf); >> >> but, surely we need to know if it is ok or not >> as the comment before says: >> * stat() is not guaranteed to set the st_size field on win32, so we >> * redefine it to our own implementation that is. >> >> Is there any simple test program that determines if the pgwin32_safestat >> is required or the library stat is sufficient? >> I presume the stat is a library function and therefore it depends on the >> compiler rather than the WIN32 platform as a whole. > No, stat() is unreliable because it is implemented on top of > FindNextFile(), and *that's* where the actual problem is at. And > that's an OS API function, not a library function. See the discussion > at http://archives.postgresql.org/pgsql-hackers/2008-03/msg01181.php > > In theory, if mingw implemented their stat() without using > FindNextFile(), it might work - but I don't see how they'd do it in > that case. And I can't see us going into details to remove such a > simple workaround even if they do - it's better to ensure we work the > same way with different compilers. > Yeah. This is only a problem because, with this compiler, configure finds this: checking for _FILE_OFFSET_BITS value needed for large files... 64 checking size of off_t... 8 whereas on the mingw-w64 compiler pitta is using it finds this: checking for _FILE_OFFSET_BITS value needed for large files... unknown checking for _LARGE_FILES value needed for largefiles... unknown checking size of off_t... 4 It's the setting of _FILE_OFFSET_BITS that causes the offending macro definition. Can we just turn off largefile support for this compiler, or maybe for all mingw builds, possibly by just disabling the checks in configure.in? I note it's turned off for MSVC in all flavors apparently. pgwin32_safestat() isn't safe for large files anyway, so there would be good grounds for doing so quite apart from this, ISTM. (Of course, we should work out how to handle large files properly on Windows, but that's a task for another day.) (BTW, someone asked me the other day why anyone would want to do 32 bit builds. One answer is that often the libraries you want to link with are only available in 32 bit versions.) cheers andrew
Hi, If we are not to use 64 bit file size (and time), #undef stat may be sufficient. The #undef should be before the prototype of pgwin32_safestat because the #define stat _stat64 affect both the function and struct stat. The #undef stat necessitate #undef fstat as the parameter struct stat * is changed. Additional change are for the macro redefinition warnings. (Suppress warnings, but perhaps not very different) The patch is tested to compile on x86_64-w64-mingw32-gcc 4.7.0 20111203 (experimental) and gcc version 4.6.1 on MingW/MSYS --- a/src/include/port.h +++ b/src/include/port.h @@ -334,6 +334,12 @@ extern bool rmtree(const char *path, bool rmtopdir); */ #if defined(WIN32) && !defined(__CYGWIN__) && !defined(UNSAFE_STAT_OK) #include <sys/stat.h> +#ifdef stat +#undef stat +#endif +#ifdef fstat +#undef fstat +#endif extern int pgwin32_safestat(const char *path, struct stat * buf); #define stat(a,b) pgwin32_safestat(a,b) If this is not sufficient, we might need to change all call of stat, lstat, and fstat to some wrapper functions? : It's theoretically doable, but could be quite difficult for a huge software. On 2011/12/05, at 1:10, Andrew Dunstan wrote: > > > On 12/04/2011 06:31 AM, Magnus Hagander wrote: >> On Sun, Dec 4, 2011 at 09:14, NISHIYAMA Tomoaki >> <tomoakin@staff.kanazawa-u.ac.jp> wrote: >>> Hi, >>> >>> I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32) >>> gcc version 4.7.0 20111203 (experimental) (GCC) >>> >>> The code can be compiled with >>> >>> diff --git a/src/include/port.h b/src/include/port.h >>> index eceb4bf..78d5c92 100644 >>> --- a/src/include/port.h >>> +++ b/src/include/port.h >>> @@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir); >>> * Some frontends don't need the size from stat, so if UNSAFE_STAT_OK >>> * is defined we don't bother with this. >>> */ >>> -#if defined(WIN32)&& !defined(__CYGWIN__)&& !defined(UNSAFE_STAT_OK) >>> +#if defined(WIN32_ONLY_COMPILER)&& !defined(UNSAFE_STAT_OK) >>> #include<sys/stat.h> >>> extern int pgwin32_safestat(const char *path, struct stat * buf); >>> >>> but, surely we need to know if it is ok or not >>> as the comment before says: >>> * stat() is not guaranteed to set the st_size field on win32, so we >>> * redefine it to our own implementation that is. >>> >>> Is there any simple test program that determines if the pgwin32_safestat >>> is required or the library stat is sufficient? >>> I presume the stat is a library function and therefore it depends on the >>> compiler rather than the WIN32 platform as a whole. >> No, stat() is unreliable because it is implemented on top of >> FindNextFile(), and *that's* where the actual problem is at. And >> that's an OS API function, not a library function. See the discussion >> at http://archives.postgresql.org/pgsql-hackers/2008-03/msg01181.php >> >> In theory, if mingw implemented their stat() without using >> FindNextFile(), it might work - but I don't see how they'd do it in >> that case. And I can't see us going into details to remove such a >> simple workaround even if they do - it's better to ensure we work the >> same way with different compilers. >> > > > Yeah. > > > This is only a problem because, with this compiler, configure finds this: > > checking for _FILE_OFFSET_BITS value needed for large files... 64 > checking size of off_t... 8 > > whereas on the mingw-w64 compiler pitta is using it finds this: > > checking for _FILE_OFFSET_BITS value needed for large files... unknown > checking for _LARGE_FILES value needed for large files... unknown > checking size of off_t... 4 > > > It's the setting of _FILE_OFFSET_BITS that causes the offending macro definition. > > Can we just turn off largefile support for this compiler, or maybe for all mingw builds, possibly by just disabling thechecks in configure.in? I note it's turned off for MSVC in all flavors apparently. pgwin32_safestat() isn't safe for largefiles anyway, so there would be good grounds for doing so quite apart from this, ISTM. (Of course, we should work outhow to handle large files properly on Windows, but that's a task for another day.) > > (BTW, someone asked me the other day why anyone would want to do 32 bit builds. One answer is that often the librariesyou want to link with are only available in 32 bit versions.) > > > cheers > > andrew > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers >
Attachment
On 12/05/2011 09:31 AM, NISHIYAMA Tomoaki wrote:
> Hi,
>
> If we are not to use 64 bit file size (and time),
> #undef stat may be sufficient. The #undef should be
> before the prototype of pgwin32_safestat because the
> #define stat _stat64 affect both the function and struct stat.
> The #undef stat necessitate #undef fstat as the parameter
> struct stat * is changed.
I don't think I'm going to do it that way, but leave this with me, I can
take it from here. Right now I'm down to the following interesting
regression failure:
$ cat regression.diffs ***
C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/expected/float8-exp-three-digits-win32.out
FriNov 25 14:24:49 2011 ---
C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/results/float8.out Mon Dec 5 18:17:36 2011
*************** *** 382,388 **** SET f1 = FLOAT8_TBL.f1 * '-1' WHERE FLOAT8_TBL.f1 > '0.0';
SELECT'' AS bad, f.f1 * '1e200' from FLOAT8_TBL f; ! ERROR: value out of range: overflow SELECT '' AS bad, f.f1
^'1e200' from FLOAT8_TBL f; ERROR: value out of range: overflow SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5;
---382,396 ---- SET f1 = FLOAT8_TBL.f1 * '-1' WHERE FLOAT8_TBL.f1 > '0.0'; SELECT '' AS bad, f.f1
*'1e200' from FLOAT8_TBL f; ! bad | ?column? ! -----+------------------ ! | 0 ! |
-3.484e+201 ! | -1.0043e+203 ! | -Infinity ! | -1.2345678901234 ! (5 rows) !
SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f; ERROR: value out of range: overflow SELECT 0 ^ 0 + 0 ^
1+ 0 ^ 0.0 + 0 ^ 0.5;
======================================================================
cheers
andrew
On 12/05/2011 06:27 PM, Andrew Dunstan wrote: > > > $ cat regression.diffs > *** > > C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/expected/float8-exp-three-digits-win32.out > Fri Nov 25 14:24:49 2011 > --- > > C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/results/float8.out > Mon Dec 5 18:17:36 2011 > *************** > *** 382,388 **** > SET f1 = FLOAT8_TBL.f1 * '-1' > WHERE FLOAT8_TBL.f1 > '0.0'; > SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f; > ! ERROR: value out of range: overflow > SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f; > ERROR: value out of range: overflow > SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5; > --- 382,396 ---- > SET f1 = FLOAT8_TBL.f1 * '-1' > WHERE FLOAT8_TBL.f1 > '0.0'; > SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f; > ! bad | ?column? > ! -----+------------------ > ! | 0 > ! | -3.484e+201 > ! | -1.0043e+203 > ! | -Infinity > ! | -1.2345678901234 > ! (5 rows) > ! > SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f; > ERROR: value out of range: overflow > SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5; > > ====================================================================== > > > This is apparently an optimization bug in the compiler. If I turn optimization off (CFLAGS=-O0) it goes away. Ick. So at the moment I'm a bit blocked. I can't really file a bug because the compiler can't currently be used to build postgres, I don't have time to construct a self-contained test case, and I don't want to commit changes to enable the compiler until the issue is solved. FYI I've been testing with the attached patch. We'll need to construct a configure test for HAVE_CRTDEFS_H. cheers andrew
Attachment
Hi, > FYI I've been testing with the attached patch. > We'll need to construct a configure test for HAVE_CRTDEFS_H. Isn't it enough to add the name in configure.in and run autoconf to update configure and autoheaders to update pg_config.h.in? The check of win32 before large file perhaps should also go to configure.in, otherwise they would be wiped with next autoconf. The patch recreated with removing the #undef but adding the conditional to skip AC_SYS_LARGEFILE in configure.in and update configure by autoconf. On 2011/12/09, at 2:46, Andrew Dunstan wrote: > > > On 12/05/2011 06:27 PM, Andrew Dunstan wrote: >> >> >> $ cat regression.diffs >> *** >> C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/expected/float8-exp-three-digits-win32.out Fri Nov 25 14:24:49 2011 >> --- >> C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/results/float8.out Mon Dec 5 18:17:36 2011 >> *************** >> *** 382,388 **** >> SET f1 = FLOAT8_TBL.f1 * '-1' >> WHERE FLOAT8_TBL.f1 > '0.0'; >> SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f; >> ! ERROR: value out of range: overflow >> SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f; >> ERROR: value out of range: overflow >> SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5; >> --- 382,396 ---- >> SET f1 = FLOAT8_TBL.f1 * '-1' >> WHERE FLOAT8_TBL.f1 > '0.0'; >> SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f; >> ! bad | ?column? >> ! -----+------------------ >> ! | 0 >> ! | -3.484e+201 >> ! | -1.0043e+203 >> ! | -Infinity >> ! | -1.2345678901234 >> ! (5 rows) >> ! >> SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f; >> ERROR: value out of range: overflow >> SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5; >> >> ====================================================================== >> >> >> > > > This is apparently an optimization bug in the compiler. If I turn optimization off (CFLAGS=-O0) it goes away. Ick. > > So at the moment I'm a bit blocked. I can't really file a bug because the compiler can't currently be used to build postgres,I don't have time to construct a self-contained test case, and I don't want to commit changes to enable the compileruntil the issue is solved. > > FYI I've been testing with the attached patch. We'll need to construct a configure test for HAVE_CRTDEFS_H. > > cheers > > andrew > > > <mingw-w64-32.patch> > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
On Thu, Dec 8, 2011 at 12:46 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > This is apparently an optimization bug in the compiler. If I turn > optimization off (CFLAGS=-O0) it goes away. Ick. > > So at the moment I'm a bit blocked. I can't really file a bug because the > compiler can't currently be used to build postgres, I don't have time to > construct a self-contained test case, and I don't want to commit changes to > enable the compiler until the issue is solved. If we're talking about adding support for a previously-unsupported configuration, it seems to me that it would be fine to commit a patch that made everything work, but for the compiler bug. We could refrain from stating that we officially support that configuration until the compiler bug is fixed, or even document the existence of the bug. We can't be responsible for other people's broken code, but I don't necessarily see that as a reason not to commit a prerequisite patch. Otherwise, as you say, there's a chicken-and-egg problem, and who does that help? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 12/09/2011 03:11 PM, Robert Haas wrote: > On Thu, Dec 8, 2011 at 12:46 PM, Andrew Dunstan<andrew@dunslane.net> wrote: >> This is apparently an optimization bug in the compiler. If I turn >> optimization off (CFLAGS=-O0) it goes away. Ick. >> >> So at the moment I'm a bit blocked. I can't really file a bug because the >> compiler can't currently be used to build postgres, I don't have time to >> construct a self-contained test case, and I don't want to commit changes to >> enable the compiler until the issue is solved. > If we're talking about adding support for a previously-unsupported > configuration, it seems to me that it would be fine to commit a patch > that made everything work, but for the compiler bug. We could refrain > from stating that we officially support that configuration until the > compiler bug is fixed, or even document the existence of the bug. We > can't be responsible for other people's broken code, but I don't > necessarily see that as a reason not to commit a prerequisite patch. > Otherwise, as you say, there's a chicken-and-egg problem, and who does > that help? > Yeah, fair enough. I'll work on that. cheers andrew
On -10/01/37 20:59, Andrew Dunstan wrote: >>> This is apparently an optimization bug in the compiler. If I turn >>> optimization off (CFLAGS=-O0) it goes away. Ick. >>> >>> So at the moment I'm a bit blocked. I can't really file a bug because >>> the >>> compiler can't currently be used to build postgres, I don't have time to >>> construct a self-contained test case, and I don't want to commit >>> changes to >>> enable the compiler until the issue is solved. >> If we're talking about adding support for a previously-unsupported >> configuration, it seems to me that it would be fine to commit a patch >> that made everything work, but for the compiler bug. We could refrain >> from stating that we officially support that configuration until the >> compiler bug is fixed, or even document the existence of the bug. We >> can't be responsible for other people's broken code, but I don't >> necessarily see that as a reason not to commit a prerequisite patch. >> Otherwise, as you say, there's a chicken-and-egg problem, and who does >> that help? >> > > > Yeah, fair enough. I'll work on that. Definitely do this (and then file a bug report with the project). I've worked with both Kai and NightStrike from the MingW-W64 project to fix previous bugs, and as long as they can build the offending source themselves then they are very helpful and quick to respond. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs
Am Freitag, 9. Dezember 2011, 15:31:17 schrieb Andrew Dunstan: > Yeah, fair enough. I'll work on that. Many thanks for reviewing, tweaking and commiting the patch! One thing I wonder about, is this snippet. Is the define really needed now? * The Mingw64 headers choke if this is already defined - they* define it themselves.*/ #if !defined(__MINGW64_VERSION_MAJOR) || defined(WIN32_ONLY_COMPILER) #define _WINSOCKAPI_ #endif #include <winsock2.h> Kind regards, Lars Kanis
On 12/12/2011 06:43 AM, Mark Cave-Ayland wrote: >>> configuration, it seems to me that it would be fine to commit a patch >>> that made everything work, but for the compiler bug. We could refrain >>> from stating that we officially support that configuration until the >>> compiler bug is fixed, or even document the existence of the bug. We >>> can't be responsible for other people's broken code, but I don't >>> necessarily see that as a reason not to commit a prerequisite patch. >>> Otherwise, as you say, there's a chicken-and-egg problem, and who does >>> that help? >>> >> >> >> Yeah, fair enough. I'll work on that. > If we're talking about adding support for a previously-unsupported > > Definitely do this (and then file a bug report with the project). I've > worked with both Kai and NightStrike from the MingW-W64 project to fix > previous bugs, and as long as they can build the offending source > themselves then they are very helpful and quick to respond. > > > Done and done (see <https://sourceforge.net/tracker/?func=detail&aid=3458244&group_id=202880&atid=983354>). cheers andrew
On 12/12/2011 09:54 AM, Lars Kanis wrote: > Am Freitag, 9. Dezember 2011, 15:31:17 schrieb Andrew Dunstan: >> Yeah, fair enough. I'll work on that. > Many thanks for reviewing, tweaking and commiting the patch! > One thing I wonder about, is this snippet. Is the define really needed now? > > * The Mingw64 headers choke if this is already defined - they > * define it themselves. > */ > #if !defined(__MINGW64_VERSION_MAJOR) || defined(WIN32_ONLY_COMPILER) > #define _WINSOCKAPI_ > #endif > #include<winsock2.h> > > > As previously discussed, unless you can prove it's not needed I don't want to remove it, on the ""if it ain't broke don't fix it" principle. I believe it is needed for at least some older compilers (specifically some of those used by buildfarm animals narwhal, frogmouth, mastodon, hamerkop and currawong), and it doesn't appear to be hurting anything. As you can see above it's been disabled for all Mingw-w64 compilers. If it's really bugging people we can try disabling it and see if any of those break, but honestly we have far uglier things that we carry for legacy reasons :-) cheers andrew
Am Montag, 12. Dezember 2011, 10:19:46 schrieb Andrew Dunstan: > > On 12/12/2011 09:54 AM, Lars Kanis wrote: > > Am Freitag, 9. Dezember 2011, 15:31:17 schrieb Andrew Dunstan: > >> Yeah, fair enough. I'll work on that. > > Many thanks for reviewing, tweaking and commiting the patch! > > One thing I wonder about, is this snippet. Is the define really needed now? > > > > * The Mingw64 headers choke if this is already defined - they > > * define it themselves. > > */ > > #if !defined(__MINGW64_VERSION_MAJOR) || defined(WIN32_ONLY_COMPILER) > > #define _WINSOCKAPI_ > > #endif > > #include<winsock2.h> > > > > > > > > As previously discussed, unless you can prove it's not needed I don't > want to remove it, on the ""if it ain't broke don't fix it" principle. I > believe it is needed for at least some older compilers (specifically > some of those used by buildfarm animals narwhal, frogmouth, mastodon, > hamerkop and currawong), and it doesn't appear to be hurting anything. > As you can see above it's been disabled for all Mingw-w64 compilers. Ok. Thanks for clarification. Kind regards, Lars
On 12/14/2011 04:43 AM, Mark Cave-Ayland wrote: > On 12/12/11 15:00, Andrew Dunstan wrote: > >>>> Yeah, fair enough. I'll work on that. >>> If we're talking about adding support for a previously-unsupported >>> >>> Definitely do this (and then file a bug report with the project). I've >>> worked with both Kai and NightStrike from the MingW-W64 project to fix >>> previous bugs, and as long as they can build the offending source >>> themselves then they are very helpful and quick to respond. >>> >>> >>> >> >> Done and done (see >> <https://sourceforge.net/tracker/?func=detail&aid=3458244&group_id=202880&atid=983354>). >> > > Hi Andrew, > > Did you see Kai's update on the ticket? If this is the case, I know > that we have seen similar bugs with PostGIS and the work-around has > been to add -ffloat-store to the compiler flags for the affected files > if that helps? > > > Hmm. Yeah, if I remove -O0 and instead set CFLAGS to -ffloat-store the error goes away. So, would we want to use that just for this file, or for the whole build? FTR, the comment in the bug reads: AFAICS from report, the issue happens with value 1e200 (as invalid range). The issue I might see here (as it doesn'toccure with x64 version, which uses sse instructions instead of x87) is that x87 registers internally have higherprecision then 32-bit. So failures in range occure on conversion from FPU register down to memory store. For x64SSE this is different, as here math operations are really just done in specified precission. Have you checked, ifyou get different result by using on 32-bit explicit SSE instructions? As things seems to work at -O0, but not at -On (with n > 0), it is pretty unlikely that runtime-functions itself arecausing this issue. So therefore my guess goes here for internal/external precision of used FPU. cheers andrew
On 14/12/11 13:59, Andrew Dunstan wrote: > Hmm. Yeah, if I remove -O0 and instead set CFLAGS to -ffloat-store the > error goes away. > > So, would we want to use that just for this file, or for the whole build? Well the latest documentation for gcc gives 2 options: -ffloat-store and -fexcess-precision=<style> which are documented at http://gcc.gnu.org/onlinedocs/gcc-4.6.2/gcc/Optimize-Options.html#Optimize-Options. For PostGIS we only applied the flags for specific files, because of severe performance warnings in older versions of the gcc documentation such as this: http://www.delorie.com/gnu/docs/gcc/gcc_10.html. I have no idea whether these warnings still hold true or not for more modern compiler versions. ISTM that the best solution would be to determine whether or not -fexcess-precision=standard does the right thing (and also determine the performance hit) or take a look at the "excess precision" notes in the older documentation to see if parts of the code can be rearranged to eliminate the effect. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs
Andrew Dunstan <andrew@dunslane.net> writes: >>> Done and done (see >>> <https://sourceforge.net/tracker/?func=detail&aid=3458244&group_id=202880&atid=983354>). >> Did you see Kai's update on the ticket? If this is the case, I know >> that we have seen similar bugs with PostGIS and the work-around has >> been to add -ffloat-store to the compiler flags for the affected files >> if that helps? > Hmm. Yeah, if I remove -O0 and instead set CFLAGS to -ffloat-store the > error goes away. Hmm, we have been bit by that recently elsewhere: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=ff68b256a533b398e3420750f34d161aeee4e099 I suspect what you are looking at is not at all mingw-specific but will soon start showing up on other x86 platforms. I see from the bug report that that's gcc 4.7.0, which hasn't made it into most distros yet but surely will soon. > So, would we want to use that just for this file, or for the whole build? -ffloat-store is a brute force solution, I think, and would affect old versions of gcc that don't exhibit any problems. I would suggest altering configure to see whether the compiler recognizes -fexcess-precision=standard and adding that to CFLAGS if so. regards, tom lane
On Wed, Dec 14, 2011 at 11:14 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > -ffloat-store is a brute force solution, I think, and would affect old > versions of gcc that don't exhibit any problems. I would suggest > altering configure to see whether the compiler recognizes > -fexcess-precision=standard and adding that to CFLAGS if so. Would it be better to change either the code or the test case to be less sensitive to this issue? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Dec 14, 2011 at 11:14 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> -ffloat-store is a brute force solution, I think, and would affect old
>> versions of gcc that don't exhibit any problems. I would suggest
>> altering configure to see whether the compiler recognizes
>> -fexcess-precision=standard and adding that to CFLAGS if so.
> Would it be better to change either the code or the test case to be
> less sensitive to this issue?
AFAICS it's really impractical to do that. The code Andrew is having
problems with is essentially
double a,b,c;...a = b * c;if (isinf(a)) throw error;
and the problem is that the multiplication result overflows in double
precision, but not in the wider-than-double register precision.
Therefore, if a is in a register and the isinf() primitive inspects the
register, it will return false, even though when the value gets stored
to memory it will become an infinity.
I don't see anything we can do to the code that avoids this issue.
You might think that explicitly casting b * c to double would help,
but our experiments in connection with the planner Assert case proved
it didn't. The only other thing we could possibly do is move the
multiplication into a separate subroutine, but what's to stop the
compiler from inlining that and generating the same code anyway?
Basically, what's going on here is that the gcc boys have decided
that speed trumps both sanity and conformance to the letter of the C
standard, unless you turn on compiler switches that say "please act
sane". So we'd better do that, unless you'd like to be dealing with
this type of issue for the rest of the project's lifespan. It's much
the same type of problem as with -fno-strict-aliasing, except that
someday we might consider biting the bullet and dealing with that piece
of insanity-in-the-name-of-speed. Floating-point performance is not
interesting enough for Postgres' purposes that I can imagine that we'd
ever want to deal with this kind of gotcha to improve FP speed.
regards, tom lane
On Wed, Dec 14, 2011 at 12:36 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > AFAICS it's really impractical to do that. The code Andrew is having > problems with is essentially > > double a,b,c; > ... > a = b * c; > if (isinf(a)) throw error; > > and the problem is that the multiplication result overflows in double > precision, but not in the wider-than-double register precision. > Therefore, if a is in a register and the isinf() primitive inspects the > register, it will return false, even though when the value gets stored > to memory it will become an infinity. Uh, wow. That really is pretty insane. How is anyone supposed to write sensible code around that non-API? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
> Uh, wow. That really is pretty insane. How is anyone supposed to
> write sensible code around that non-API?
Usability seems to be very low on the gcc project's list of goals
these days.
Personally I think this sort of thing might be fine if it were triggered
by -ffast-math or something like that. But as a default behavior it's
entirely ridiculous.
regards, tom lane
On 12/14/2011 11:14 AM, Tom Lane wrote: > -ffloat-store is a brute force solution, I think, and would affect old > versions of gcc that don't exhibit any problems. I would suggest > altering configure to see whether the compiler recognizes > -fexcess-precision=standard and adding that to CFLAGS if so. OK, this and the associated configure change seems to do the trick: diff --git a/configure.in b/configure.in index 9cf084d..b29bb61 100644 --- a/configure.in +++ b/configure.in @@-437,6 +437,8 @@ if test "$GCC" = yes -a "$ICC" = no; then PGAC_PROG_CC_CFLAGS_OPT([-fno-strict-aliasing]) # Disable optimizations that assume no overflow; needed for gcc 4.3+ PGAC_PROG_CC_CFLAGS_OPT([-fwrapv]) + # DisableFP optimizations that cause isinf errors on gcc 4.5+ + PGAC_PROG_CC_CFLAGS_OPT([-fexcess-precision=standard]) elif test "$ICC" = yes; then # Intel's compiler has a bug/misoptimization in checking for # division by NAN(NaN == 0), -mp1 fixes it, so add it to the CFLAGS. I guess we should backpatch it? cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> + # Disable FP optimizations that cause isinf errors on gcc 4.5+
> + PGAC_PROG_CC_CFLAGS_OPT([-fexcess-precision=standard])
Looks sane to me, except "isinf errors" is an awfully narrow reading of
the problem. Maybe just say "assorted errors"? Also, do we know that
gcc 4.5 poses the issue? I'm only aware of reports for 4.6 and 4.7.
> I guess we should backpatch it?
+1. Back branches will see these same problems as soon as anybody
tries to compile them with latest-n-greatest gcc.
regards, tom lane
On 12/14/2011 03:09 PM, Tom Lane wrote: > Andrew Dunstan<andrew@dunslane.net> writes: >> + # Disable FP optimizations that cause isinf errors on gcc 4.5+ >> + PGAC_PROG_CC_CFLAGS_OPT([-fexcess-precision=standard]) > Looks sane to me, except "isinf errors" is an awfully narrow reading of > the problem. Maybe just say "assorted errors"? Also, do we know that > gcc 4.5 poses the issue? I'm only aware of reports for 4.6 and 4.7. It looked to me like this switch landed in gcc 4.5 because they were getting problems like this. See <http://gcc.gnu.org/ml/gcc-patches/2008-11/msg00105.html> >> I guess we should backpatch it? > +1. Back branches will see these same problems as soon as anybody > tries to compile them with latest-n-greatest gcc. > > Yeah. Will do. cheers andrew