Thread: LIBPQ on Windows and large Queries

LIBPQ on Windows and large Queries

From
"Steve Howe"
Date:
Hello all,
       Is anybody trying to solve the 8191 bytes query limit from libpq
windows port ???       We've discussed this topic on "Large queries - again" thread and it
seems like nobody got interested on fixing it.       All Windows applications that rely on libpq are broken because of
this issue (ODBC applications are fine btw).       I can also do any kind of testing under Windows (and actually I'll
do it anyway). I wonder if this limitation also applies to the unix libpq
library ???       Jan, Tom, Bruce - any news on this ?

Best Regards,
Steve Howe




RE: LIBPQ on Windows and large Queries

From
Khoa Do
Date:
    I'm surprised you are even able to run postgres in windows.  I
didn't even know postgres supported windows.  Could you kindly point me to
instructions on how to run it and build the postgres souce on windows?  If
nobody will try to fix it, then maybe we should just try it ourseleves and
post some patch to it.

-----Original Message-----
From: Steve Howe [mailto:howe@carcass.dhs.org]
Sent: Wednesday, July 25, 2001 9:25 PM
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] LIBPQ on Windows and large Queries


Hello all,
       Is anybody trying to solve the 8191 bytes query limit from libpq
windows port ???       We've discussed this topic on "Large queries - again" thread and it
seems like nobody got interested on fixing it.       All Windows applications that rely on libpq are broken because of
this issue (ODBC applications are fine btw).       I can also do any kind of testing under Windows (and actually I'll
do it anyway). I wonder if this limitation also applies to the unix libpq
library ???       Jan, Tom, Bruce - any news on this ?

Best Regards,
Steve Howe



---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster


RE: LIBPQ on Windows and large Queries

From
Joel Burton
Date:
On Thu, 26 Jul 2001, Khoa Do wrote:

>     I'm surprised you are even able to run postgres in windows.  I
> didn't even know postgres supported windows.  Could you kindly point me to
> instructions on how to run it and build the postgres souce on windows?  If
> nobody will try to fix it, then maybe we should just try it ourseleves and
> post some patch to it.
> 
> -----Original Message-----
> From: Steve Howe [mailto:howe@carcass.dhs.org]
> Sent: Wednesday, July 25, 2001 9:25 PM
> To: pgsql-hackers@postgresql.org
> Subject: [HACKERS] LIBPQ on Windows and large Queries
> 
> 
> Hello all,
> 
>         Is anybody trying to solve the 8191 bytes query limit from libpq
> windows port ???
>         We've discussed this topic on "Large queries - again" thread and it
> seems like nobody got interested on fixing it.
>         All Windows applications that rely on libpq are broken because of
> this issue (ODBC applications are fine btw).
>         I can also do any kind of testing under Windows (and actually I'll
> do it anyway). I wonder if this limitation also applies to the unix libpq
> library ???
>         Jan, Tom, Bruce - any news on this ?

Steve's question was about the Postgres library (libpq.dll). I can't
confirm that is hasn't been TOAST'ed (for >8192 chars).

PostgreSQL does, however, run just peachy under Windows. Has for a long
time w/Windows NT and Windows 2000; recently, works with Windows 98
too. (Can't vouch for WinME, never touched the thing.)
 www.cygwin.com

Can download it as part of the Cygwin package. You'll need to install
CygIPC (easily found via google, a simple binary install). Whole thing is
pretty much of a snap nowadays.

Of course, would you want to run a serious database under Windows 98?

-- 
Joel Burton   <jburton@scw.org>
Director of Information Systems, Support Center of Washington



Re: LIBPQ on Windows and large Queries

From
Tom Lane
Date:
"Steve Howe" <howe@carcass.dhs.org> writes:
>         Is anybody trying to solve the 8191 bytes query limit from libpq
> windows port ???

I think it's *your* responsibility, at least to identify what's going
on.  This is an open source project, and that means you can and should
fix problems that affect you.

FWIW, if the problem is real (which I still misdoubt), it seems like
it would have to be related to needing to flush multiple output
bufferloads during a single PQsendQuery.  This works just fine on
everything but Windows --- why would it fail there?  And why would
no one but you have complained of it before?  I have no ideas.
        regards, tom lane


Re: LIBPQ on Windows and large Queries

From
Joerg Hessdoerfer
Date:
At 01:36 27.07.2001 -0400, you wrote:
>"Steve Howe" <howe@carcass.dhs.org> writes:
> >         Is anybody trying to solve the 8191 bytes query limit from libpq
> > windows port ???
>
>I think it's *your* responsibility, at least to identify what's going
>on.  This is an open source project, and that means you can and should
>fix problems that affect you.
>
>FWIW, if the problem is real (which I still misdoubt), it seems like
>it would have to be related to needing to flush multiple output
>bufferloads during a single PQsendQuery.  This works just fine on
>everything but Windows --- why would it fail there?  And why would
>no one but you have complained of it before?  I have no ideas.

[...]

To go on about this, I use psql 7.1.2 for toying around on Win2K, and have not
had this problem. I just evaluated using TOASTED blobs (written in a text 
column
as base64 encoded).

I did use the cygwin libpq though. Probably I'll get around compiling 
native libpq
and try that with my test cases.

Greetings,        Joerg



Re: LIBPQ on Windows and large Queries

From
Hiroshi Inoue
Date:
Tom Lane wrote:
> 
> "Steve Howe" <howe@carcass.dhs.org> writes:
> >         Is anybody trying to solve the 8191 bytes query limit from libpq
> > windows port ???
> 
> FWIW, if the problem is real (which I still misdoubt), 

Yes it's real.
Error handling seems the cause. When "Error: pqReadData() --
read() failed: errno=0 No error" occurs WSAGetLastError()
returns WSAEWOULDBLOCK. If EWOULDBLOCK exists and errno ==
EWOULDBLOCK, pqReadData() returns 0 or 1 not -1.
I added the code errno = WSAGetLastError(); and 
#define    EWOULDBLOCK WSAEWOULDBLOCK.
After that I encountered another error "pqFlush() --  
couldn't send data: errno=0". Then WSAGetLastError() also
returns WSAEWOULDBLOCK. After adding *errno = WSAGetLastError();*
the insertion was successful.

regards,
Hiroshi Inoue


Re: LIBPQ on Windows and large Queries

From
Tom Lane
Date:
Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> Yes it's real.
> Error handling seems the cause.

Hmm, are you working with CVS tip or 7.1.* sources?  I'm wondering
how this interacts with the recent patch to #define errno as
WSAGetLastError() on WIN32...
        regards, tom lane


Re: LIBPQ on Windows and large Queries

From
Hiroshi Inoue
Date:
Tom Lane wrote:
> 
> Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> > Yes it's real.
> > Error handling seems the cause.
> 
> Hmm, are you working with CVS tip or 7.1.* sources?  I'm wondering
> how this interacts with the recent patch to #define errno as
> WSAGetLastError() on WIN32...
> 

Oops sorry, I could have no cvs access for 3 weeks.
Well haven't this problem solved already ?

regards,
Hiroshi Inoue


Re: LIBPQ on Windows and large Queries

From
Tom Lane
Date:
Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> Well haven't this problem solved already ?

I'm not sure.  Steve, have you tried current sources?
        regards, tom lane


Re: LIBPQ on Windows and large Queries

From
"Steve Howe"
Date:
Hello all,
       I was in a trip and just arrived, and will do it real soon.


Best Regards,
Steve Howe

----- Original Message ----- 
From: "Tom Lane" <tgl@sss.pgh.pa.us>
To: "Hiroshi Inoue" <Inoue@tpf.co.jp>
Cc: "Steve Howe" <howe@carcass.dhs.org>; <pgsql-hackers@postgresql.org>
Sent: Sunday, July 29, 2001 8:57 PM
Subject: Re: [HACKERS] LIBPQ on Windows and large Queries 


> Hiroshi Inoue <Inoue@tpf.co.jp> writes:
> > Well haven't this problem solved already ?
> 
> I'm not sure.  Steve, have you tried current sources?
> 
> regards, tom lane
>