Re: Fwd: Re: unicode in 7.1 - Mailing list pgsql-general

If you are getting a parse error as you said in a latter message, then
you should set "debug_print_query = true" in your postgresql.conf file.
  This will cause the sql statements to be printed in the server log
file and you then might be able to figure out why the statement sent
couldn't be parsed.

I noticed that the code snipit you sent below seems to have an error in
it.  You select 'TEST' as test, but then later you do a
getString("title") which should fail since there isn't a selected column
called "title".

thanks,
--Barry


Culley Harrelson wrote:
> I just finished rebuilding my development environment and have
> duplicated the error.  I have a database created with the -E UNICODE
> flag and I am using the jdbc7.1-1.2.jar driver.  Data just gets
> truncated with a non-latin character without the charSet property
> switch, and I get a sql error when I leave in the property switch. Here
> is a snap of my code:
>
> --------------------------
> Connection conn = null;
> Statement stmt = null;
> ResultSet rs = null;
> Properties props = new Properties();
> Class.forName ("org.postgresql.Driver");
> props.put("user","login");
> props.put("password","password");
> props.put("charSet","UNICODE");
> conn = DriverManager.getConnection("jdbc:postgresql://url", props);
>
> stmt = conn.createStatement();
> rs = stmt.executeQuery("select 'TEST' as test");
> rs.next();
> String dummy = "";
> dummy = rs.getString("title");
> conn.close();
> ---------------------------------------
>
> And when I do this I get the following SQL error:
>
> ---------------------------------------
>
> *Internal Servlet Error:
> *javax.servlet.ServletException:
>         at
> org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:459)
>         at
> _0002fzz_0002ejspzz_jsp_464._jspService(_0002fzz_0002ejspzz_jsp_464.java:97)
>         at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
>         at
> org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServlet.java:177)
>         at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:318)
>         at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:391)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
>         at
> org.apache.tomcat.core.ServletWrapper.doService(ServletWrapper.java:404)
>         at org.apache.tomcat.core.Handler.service(Handler.java:286)
>         at
> org.apache.tomcat.core.ServletWrapper.service(ServletWrapper.java:372)
>         at
> org.apache.tomcat.core.ContextManager.internalService(ContextManager.java:797)
>         at
> org.apache.tomcat.core.ContextManager.service(ContextManager.java:743)
>         at
> org.apache.tomcat.service.http.HttpConnectionHandler.processConnection(HttpConnectionHandler.java:210)
>         at
> org.apache.tomcat.service.TcpWorkerThread.runIt(PoolTcpEndpoint.java:416)
>         at
> org.apache.tomcat.util.ThreadPool$ControlRunnable.run(ThreadPool.java:498)
>         at java.lang.Thread.run(Thread.java:484)
> *Root cause:*
> java.sql.SQLException:
>         at org.postgresql.Connection.ExecSQL(Connection.java:533)
>         at org.postgresql.jdbc2.Statement.execute(Statement.java:294)
>         at org.postgresql.jdbc2.Statement.executeQuery(Statement.java:59)
>         at
> _0002fzz_0002ejspzz_jsp_464._jspService(_0002fzz_0002ejspzz_jsp_464.java:77)
>         at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:119)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
>         at
> org.apache.jasper.servlet.JspServlet$JspServletWrapper.service(JspServlet.java:177)
>         at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:318)
>         at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:391)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
>         at
> org.apache.tomcat.core.ServletWrapper.doService(ServletWrapper.java:404)
>         at org.apache.tomcat.core.Handler.service(Handler.java:286)
>         at
> org.apache.tomcat.core.ServletWrapper.service(ServletWrapper.java:372)
>         at
> org.apache.tomcat.core.ContextManager.internalService(ContextManager.java:797)
>         at
> org.apache.tomcat.core.ContextManager.service(ContextManager.java:743)
>         at
> org.apache.tomcat.service.http.HttpConnectionHandler.processConnection(HttpConnectionHandler.java:210)
>         at
> org.apache.tomcat.service.TcpWorkerThread.runIt(PoolTcpEndpoint.java:416)
>         at
> org.apache.tomcat.util.ThreadPool$ControlRunnable.run(ThreadPool.java:498)
>         at java.lang.Thread.run(Thread.java:484)
>
> ---------------------------------------
>
> If I take out the encoding line it works but truncates at any non-latin
> character.
>
> culley
>
>
>
> At 07:33 PM 9/11/01 -0700, you wrote:
>
>> Culley,
>>
>> I would like to help, but I'm not sure what I can do to help.  If
>> there is anything I can do, please let me know.
>>
>> thanks,
>> --Barry
>>
>>
>> Culley Harrelson wrote:
>>
>>> Yes, psql works fine and displays non-English characters fine.
>>> Culley
>>> At 06:05 PM 9/11/01 -0700, you wrote:
>>>
>>>> If that is the classpath you are using then you should be using the
>>>> correct code.  Can you sucessfully select the data through psql?
>>>>
>>>> thanks,
>>>> --Barry
>>>>
>>>> Culley Harrelson wrote:
>>>>
>>>>> No this problem was fixed when I rebuild the database with UNICODE
>>>>> encoding.  No more '?'.  Now the data just gets *truncated* when it
>>>>> hits a non-Latin character.
>>>>> Here is my classpath:
>>>>>
/usr/jdk1.3/jre/lib/rt.jar:/usr/jdk1.3/lib/tools.jar:/usr/jakarta-oro-2.0.1/jakarta-oro-2.0.1.jar:/usr/jakarta-regexp-1.2/jakarta-regexp-1.2.jar:/usr/jakarta-servletapi-3.2/lib/servlet.jar:/usr/jakarta-tomcat-3.2.1/lib/jasper.jar:/usr/jakarta-tomcat-3.2.1/lib/webserver.jar:/usr/Jetspeed-1.1/Jetspeed.jar:/usr/cocoon-1.8/lib/stylebook-1.0-b2.jar:/usr/cocoon-1.8/bin/cocoon.jar:/usr/xerces-1_2_0/xerces.jar:/usr/jakarta-ant/lib/ant.jar:/usr/jakarta-ant/lib/jaxp.jar:/usr/jakarta-ant/lib/optional.jar:/usr/jakarta-ant/lib/parser.jar:/usr/xalan_1_2_D01/xalan.jar:/usr/xalan_1_2_D01/bsf.jar:/usr/xalan_1_2_D01/bsfengines.jar:/usr/aoserv/lib/acme.jar:/usr/aoserv/lib/aocode-public.jar:/usr/aoserv/lib/aoserv-private.jar:/usr/aoserv/lib/damud.jar:/usr/aoserv/lib/danshome.jar:/usr/castor/castor-0.8.8.jar:/usr/cos/lib/cos.jar:/usr/ecs-1.3.3/ecs-1.3.3.jar:/usr/freemarker152/freemarker-jdk1.2.jar:/usr/gnu.regexp-1.0.8/lib/gnu-regexp-1.0.8.jar:/usr/jaf-1.0.1/activation.jar:/usr/jakarta-slid
e-
>>>>
>> 1.
>>
>>>>
>>>>
0m4/slide/lib/webdav.jar:/usr/jakarta-slide-1.0m4/slide/lib/slide.jar:/usr/javachart/jars/kcServlet.jar:/usr/javamail-1.1.3/mail.jar:*/usr/jdbc-2.0/jdbc-2.0.jar*:/usr/jsse1.0.2/lib/jcert.jar:/usr/jsse1.0.2/lib/jnet.jar:/usr/jsse1.0.2/lib/jsse.jar:/usr/Apache-Jyve-20000907/src/java:/usr/mm.mysql-2.0.4/mm-mysql.jar:/usr/openxml/openxml-1.2.jar:/usr/ozone-0.5.5:*/usr/share/pgsql/jdbc7.1-1.2.jar*:/usr/pop3-1.1.1/pop3.jar:/usr/soap-2_0/lib/soap.jar:/usr/spfc-0.2.0/bin/spfc.jar:/usr/Apache-Turbine-20000907/src/java:/usr/village-1.3/village-1.3.jar:/usr/webmacro/webmacro.jar:/usr/xang_0_0_2/fesi.zip:/usr/xang_0_0_2/xang.jar:/usr/xmlrpc-java/lib/xmlrpc.jar:/usr/ApacheJSSI-1.1.2/src/java/ApacheJSSI.jar:/usr/interclient/interclient.jar:/usr/PoolMan-1.4.1/lib/PoolMan.jar:/usr/fop-0_13_0/fop_bin_0_13_0.jar:/usr/fop-0_13_0/lib/xml.jar:/wwwgroup/culley/classes:/www/flashcard/classes:/www/banyan/classes

>>>>
>>>>
>>>>> In the same directory as the 7.1-1.2 driver is a copy of the old
>>>>> driver.  Is there someway I could still be hitting the old one?
>>>>> An interesting anomoly-- When I hit my remote production postgres
>>>>> server from my local tomcat (normally I'm using tomcat on the same
>>>>> machine as the database) non-English characters *were* displayed
>>>>> correctly.  I was through the jdbc7.0-1.2.jar driver though!  Very
>>>>> confusing...
>>>>> I'm currently updating my local dinky little linux box with
>>>>> postgres 7.1.3 so my dev environment more closely matches the
>>>>> production.
>>>>> Debugging should be easier...
>>>>> culley
>>>>>
>>>>> At 04:33 PM 9/11/01 -0700, you wrote:
>>>>>
>>>>>> Culley,
>>>>>>
>>>>>> That all looks fine.  I don't understand why you would be having
>>>>>> problems with these settings.  Are you sure you are using the 7.1
>>>>>> jdbc drivers?  One other thing I can think of to check is to do a
>>>>>> 'show client_encoding'.  This should also return UNICODE.
>>>>>>
>>>>>> So to restate your problem, you get the 8bit characters returned
>>>>>> as ?'s through jdbc, but it works correctly through psql.  Is this
>>>>>> a correct restatement of your problem?
>>>>>>
>>>>>> thanks,
>>>>>> --Barry
>>>>>>
>>>>>> Culley Harrelson wrote:
>>>>>>
>>>>>>>
>>>>>>> At 11:45 AM 9/11/01 -0700, you wrote:
>>>>>>>
>>>>>>>> select getdatabaseencoding();
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> UNICODE
>>>>>>>
>>>>>>>> select datname, encoding from pg_database;
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  flashcard     |        5
>>>>>>> All other databases on the server had 0 for encoding.
>>>>>>> Unfortunately my dev environment still has postgres 7.0.  I
>>>>>>> updated my connection pool object and it works on dev but not on
>>>>>>> prod with 7.1.  I really don't want to update my dev environment
>>>>>>> untill I can do the whole machine from scratch but I fear I might
>>>>>>> have to in order to debug this...
>>>>>>> Culley
>>>>>>>
>>>>>>> _________________________________________________________
>>>>>>> Do You Yahoo!?
>>>>>>> Get your free @yahoo.com address at http://mail.yahoo.com
>>>>>>> <http://mail.yahoo.com/> <http://mail.yahoo.com/>
>>>>>>
>>>
>>> _________________________________________________________
>>> Do You Yahoo!?
>>> Get your free @yahoo.com address at http://mail.yahoo.com
>>> <http://mail.yahoo.com/>
>>