Thread: invalid string enlargement request

invalid string enlargement request

From
Glenn Holmer
Date:
Not sure if this is a JDBC or general Postgres question, but we're
losing a web app's connection to Postgres with these errors:

invalid string enlargement request size 1358954494
AbortTransaction and not in in-progress state
invalid frontend message type 41

This is with Postgres 7.4.7 and driver pg74.213.jdbc3.jar.

Anybody have any ideas?  Should I take this to the general list?

--
____________________________________________________________
Glenn Holmer                          gholmer@weycogroup.com
Software Engineer                        phone: 414-908-1809
Weyco Group, Inc.                          fax: 414-908-1601

Re: invalid string enlargement request

From
Oliver Jowett
Date:
Glenn Holmer wrote:
> Not sure if this is a JDBC or general Postgres question, but we're
> losing a web app's connection to Postgres with these errors:
>
> invalid string enlargement request size 1358954494
> AbortTransaction and not in in-progress state

This is the backend trying to allocate a very large string for some
reason, perhaps because of a silly-sized protocol message being received
or sent. Are you working with large bytea data at all?

> invalid frontend message type 41

And this is the protocol stream getting out of sync, possibly because of
the above error.

> This is with Postgres 7.4.7 and driver pg74.213.jdbc3.jar.

Can you try with the latest (build 310) driver and see if the problem
persists?

If so, can you send a testcase that demonstrates the problem? Or at
least the query code that fails plus the complete exception you get.

-O

Re: invalid string enlargement request

From
Glenn Holmer
Date:
Sorry Oliver, this was supposed to go to the list...

Oliver Jowett wrote:
> Glenn Holmer wrote:
>> Not sure if this is a JDBC or general Postgres question, but we're
>> losing a web app's connection to Postgres with these errors:
>>
>> invalid string enlargement request size 1358954494
>> AbortTransaction and not in in-progress state
>
> This is the backend trying to allocate a very large string for some
> reason, perhaps because of a silly-sized protocol message being received
> or sent. Are you working with large bytea data at all?

Not that I know of, this is an e-commerce package that we
customized.

>> invalid frontend message type 41
>
> And this is the protocol stream getting out of sync, possibly because of
> the above error.
>
>> This is with Postgres 7.4.7 and driver pg74.213.jdbc3.jar.
>
> Can you try with the latest (build 310) driver and see if the problem
> persists?
>
> If so, can you send a testcase that demonstrates the problem? Or at
> least the query code that fails plus the complete exception you get.


We can't upgrade the driver because we get this:

2005-03-17 10:10:22,961 WARN  net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 38
Message: SQL Error: 0, SQLState: 42883
2005-03-17 10:10:22,962 ERROR net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 46
Message: ERROR: operator does not exist: smallint = boolean


Here is more of the log with pg74.213.jdbc3.jar:

2005-03-16 14:30:43,239 WARN  net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 38  Message: SQL Error: 0, SQLState: 08S01
2005-03-16 14:30:44,273 ERROR net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 46  Message: The backend has broken the connection. Possibly the action you have att
empted has caused it to close.
2005-03-16 14:30:44,278 WARN  net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 38  Message: SQL Error: 0, SQLState: 08S01
2005-03-16 14:30:58,561 WARN  net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 38  Message: SQL Error: 0, SQLState: XX000
2005-03-16 14:30:58,607 ERROR net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: logExceptions - Line: 46  Message: ERROR: invalid string enlargement request size 1358954494

a little later on, we see:

2005-03-16 14:30:59,140 ERROR net.sf.hibernate.util.JDBCExceptionReporter - Meth
od: <init> - Line: 38  Message: Could not execute query
org.postgresql.util.PSQLException: ERROR: invalid string enlargement request siz
e 1358954494

       at org.postgresql.util.PSQLException.parseServerError(PSQLException.java
:139)
       at org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:152)
       at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:100)
       at org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)
       at org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Stat
ement.java:517)
       at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Stat
ement.java:50)
       at org.postgresql.jdbc1.AbstractJdbc1Statement.executeQuery(AbstractJdbc
1Statement.java:233)
       at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(Dele
gatingPreparedStatement.java:92)
       at net.sf.hibernate.impl.BatcherImpl.getResultSet(BatcherImpl.java:87)
       at net.sf.hibernate.loader.Loader.getResultSet(Loader.java:800)
       at net.sf.hibernate.loader.Loader.doQuery(Loader.java:189)


The other possible culprit is that one of our developers just started
using the Windows version of pgAdmin III, is that known to cause any
problems?

--
____________________________________________________________
Glenn Holmer                          gholmer@weycogroup.com
Software Engineer                        phone: 414-908-1809
Weyco Group, Inc.                          fax: 414-908-1601


Re: invalid string enlargement request

From
Oliver Jowett
Date:
Glenn Holmer wrote:

> We can't upgrade the driver because we get this:
>
> 2005-03-17 10:10:22,961 WARN
> net.sf.hibernate.util.JDBCExceptionReporter - Meth
> od: logExceptions - Line: 38  Message: SQL Error: 0, SQLState: 42883
> 2005-03-17 10:10:22,962 ERROR
> net.sf.hibernate.util.JDBCExceptionReporter - Meth
> od: logExceptions - Line: 46  Message: ERROR: operator does not exist:
> smallint = boolean

Without more details I'd have to say this is a hibernate or schema
mapping bug -- it looks like it is using setBoolean() where you should
be using setShort() (or vice versa). This only shows up in newer drivers
because they more tightly bind types to parameters (due to using the V3
extended query protocol to pass parameters to the server)

This behaviour is unlikely to change, so you might want to look at
tracking down the cause. As a last resort, you might be able to create
implicit casts between smallint<->boolean.

> Here is more of the log with pg74.213.jdbc3.jar:
>
> 2005-03-16 14:30:43,239 WARN
> net.sf.hibernate.util.JDBCExceptionReporter - Meth
> od: logExceptions - Line: 38  Message: SQL Error: 0, SQLState: 08S01
> 2005-03-16 14:30:44,273 ERROR
> net.sf.hibernate.util.JDBCExceptionReporter - Meth
> od: logExceptions - Line: 46  Message: The backend has broken the
> connection. Possibly the action you have att
> empted has caused it to close.

What is in the server logs around this point? This might be a backend
crash, or might just be the protocol stream getting out of sync again.

> The other possible culprit is that one of our developers just started
> using the Windows version of pgAdmin III, is that known to cause any
> problems?

Not that I know of.

To diagnose this further against the old driver I am going to need
either a testcase, the query code, or a tcpdump of the offending connection.

-O

Re: smallint vs. boolean (was invalid string enlargement

From
Glenn Holmer
Date:
On Fri, 2005-03-18 at 09:37 +1300, Oliver Jowett wrote:
> Glenn Holmer wrote:
>
> > We can't upgrade the driver because we get this:
> >
> > 2005-03-17 10:10:22,961 WARN
> > net.sf.hibernate.util.JDBCExceptionReporter - Meth
> > od: logExceptions - Line: 38  Message: SQL Error: 0, SQLState: 42883
> > 2005-03-17 10:10:22,962 ERROR
> > net.sf.hibernate.util.JDBCExceptionReporter - Meth
> > od: logExceptions - Line: 46  Message: ERROR: operator does not exist:
> > smallint = boolean
>
> Without more details I'd have to say this is a hibernate or schema
> mapping bug -- it looks like it is using setBoolean() where you should
> be using setShort() (or vice versa). This only shows up in newer drivers
> because they more tightly bind types to parameters (due to using the V3
> extended query protocol to pass parameters to the server)
>
> This behaviour is unlikely to change, so you might want to look at
> tracking down the cause. As a last resort, you might be able to create
> implicit casts between smallint<->boolean.

Yes, the database has some fields defined as smallint and set to 0 for
false or either 1 for true; the Hibernate DTOs have these fields
declared as boolean.

Is there any way around this without rewriting everything?

--
____________________________________________________________
Glenn Holmer                          gholmer@weycogroup.com
Software Engineer                        phone: 414-908-1809
Weyco Group, Inc.                          fax: 414-908-1601



Re: smallint vs. boolean (was invalid string enlargement

From
Oliver Jowett
Date:
Glenn Holmer wrote:
> On Fri, 2005-03-18 at 09:37 +1300, Oliver Jowett wrote:

>>This behaviour is unlikely to change, so you might want to look at
>>tracking down the cause. As a last resort, you might be able to create
>>implicit casts between smallint<->boolean.
>
>
> Yes, the database has some fields defined as smallint and set to 0 for
> false or either 1 for true; the Hibernate DTOs have these fields
> declared as boolean.
>
> Is there any way around this without rewriting everything?

Change the column type so it is a boolean, change the hibernate mapping
so it calls setShort(), or create the implicit cast as I suggested.

-O

Re: smallint vs. boolean (was invalid string enlargement

From
Glenn Holmer
Date:
On Wed, 2005-04-13 at 10:09 +1200, Oliver Jowett wrote:

> Change the column type so it is a boolean, change the hibernate mapping
> so it calls setShort(), or create the implicit cast as I suggested.

I got it to work with some PL/pgSQL functions, implicit casts, and
overloading the = operator (smallint, boolean).  I'll strongly recommend
to the vendor that they rewrite the Postgres DDL for their product to
use booleans for boolean fields, but in the mean time are there any side
effects I should worry about?

--
____________________________________________________________
Glenn Holmer                          gholmer@weycogroup.com
Software Engineer                        phone: 414-908-1809
Weyco Group, Inc.                          fax: 414-908-1601