Thread: minor changes in psql's \encoding command
Hi all,
While developing one of our applicatins,
we found that psql's meta command \encoding
return different result to what
"SHOW CLIENT_ENCODING" command return.
And it make us somewhat difficult to determine
our problem. After take a look at source of psql
and libpq, I found that the \encoding command
only use the encoding setting while connection was made.
don't know why, but a few changes make \encoding command
synchronized with "SHOW CLIENT_ENCODING" &
"SET CLIENT_ENCODING TO 'something'"
please apply it if it's useful.
regards laser
*** fe-connect.c.old Sun Jan 5 09:34:00 2003
--- fe-connect.c Sun Jan 5 09:46:25 2003
***************
*** 2730,2739 ****
}
int
! PQclientEncoding(const PGconn *conn)
{
if (!conn || conn->status != CONNECTION_OK)
return -1;
return conn->client_encoding;
}
--- 2730,2754 ----
}
int
! PQclientEncoding(PGconn *conn)
{
+ static char query[] = "show client_encoding";
+ PGresult *res;
+ char *encoding;
+
if (!conn || conn->status != CONNECTION_OK)
return -1;
+ res = PQexec(conn, query);
+
+ if(res == (PGresult *) NULL)
+ return -1;
+ if(res->resultStatus != PGRES_TUPLES_OK)
+ return -1;
+ else{
+ encoding = PQgetvalue(res, 0, 0);
+ conn->client_encoding = pg_char_to_encoding(encoding);
+ }
+ PQclear(res);
return conn->client_encoding;
}
*** command.c.old Sun Jan 5 10:05:29 2003
--- command.c Sun Jan 5 10:06:17 2003
***************
*** 455,463 ****
{
char *encoding = scan_option(&string, OT_NORMAL, NULL, false);
! if (!encoding)
/* show encoding */
puts(pg_encoding_to_char(pset.encoding));
else
{
/* set encoding */
--- 455,465 ----
{
char *encoding = scan_option(&string, OT_NORMAL, NULL, false);
! if (!encoding){
! pset.encoding = PQclientEncoding(pset.db);
/* show encoding */
puts(pg_encoding_to_char(pset.encoding));
+ }
else
{
/* set encoding */
*** libpq-fe.h.old Sun Jan 5 09:47:59 2003
--- libpq-fe.h Sun Jan 5 09:48:22 2003
***************
*** 227,233 ****
extern char *PQerrorMessage(const PGconn *conn);
extern int PQsocket(const PGconn *conn);
extern int PQbackendPID(const PGconn *conn);
! extern int PQclientEncoding(const PGconn *conn);
extern int PQsetClientEncoding(PGconn *conn, const char *encoding);
#ifdef USE_SSL
--- 227,233 ----
extern char *PQerrorMessage(const PGconn *conn);
extern int PQsocket(const PGconn *conn);
extern int PQbackendPID(const PGconn *conn);
! extern int PQclientEncoding(PGconn *conn);
extern int PQsetClientEncoding(PGconn *conn, const char *encoding);
#ifdef USE_SSL
Weiping He <laser@zhengmai.com.cn> writes:
> int
> ! PQclientEncoding(PGconn *conn)
> {
> + static char query[] = "show client_encoding";
> + PGresult *res;
> + char *encoding;
> +
> if (!conn || conn->status != CONNECTION_OK)
> return -1;
> + res = PQexec(conn, query);
> +
> + if(res == (PGresult *) NULL)
> + return -1;
> + if(res->resultStatus != PGRES_TUPLES_OK)
> + return -1;
> + else{
> + encoding = PQgetvalue(res, 0, 0);
> + conn->client_encoding = pg_char_to_encoding(encoding);
> + }
> + PQclear(res);
> return conn->client_encoding;
> }
I'm a bit dissatisfied with this, as (a) it fails utterly with pre-7.3
servers; (b) it fails if the server is in transaction-abort state;
(c) it fails if the client is in the middle of a query cycle already;
(d) it changes what had been an extremely cheap call into an extremely
expensive one. (I shudder to think of the implications for an app that
calls it in an inner loop, as for example per-character during display
of results.)
This is quite a large change from the old behavior to support what seems
an unusual corner case. How many clients have need to change encoding
on the fly?
I'd prefer to just document that PQclientEncoding is untrustworthy if
the client sets the encoding directly (rather than via PGCLIENTENCODING
or PQsetClientEncoding), and similarly that psql's \encoding is not
trustworthy if one goes behind its back.
regards, tom lane
Tom Lane wrote: > > I'm a bit dissatisfied with this, as (a) it fails utterly with pre-7.3 > servers; (b) it fails if the server is in transaction-abort state; > (c) it fails if the client is in the middle of a query cycle already; > (d) it changes what had been an extremely cheap call into an extremely > expensive one. (I shudder to think of the implications for an app that > calls it in an inner loop, as for example per-character during display > of results.) > > This is quite a large change from the old behavior to support what seems > an unusual corner case. How many clients have need to change encoding > on the fly? > > I'd prefer to just document that PQclientEncoding is untrustworthy if > the client sets the encoding directly (rather than via PGCLIENTENCODING > or PQsetClientEncoding), and similarly that psql's \encoding is not > trustworthy if one goes behind its back. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster > emm, understood your concern. documentation it seems better. let me see if there better way to solve our problem. regards laser
I said:
> I'd prefer to just document that PQclientEncoding is untrustworthy if
> the client sets the encoding directly (rather than via PGCLIENTENCODING
> or PQsetClientEncoding), and similarly that psql's \encoding is not
> trustworthy if one goes behind its back.
On looking closer, both libpq and psql assume that their current
internal encoding settings accurately describe the strings they get
from the backend. So hacking up PQclientEncoding() wouldn't be enough
to fix all the problems anyway.
I think we need to document that doing "SET client_encoding" directly is
hazardous to your health with either of these interfaces, and perhaps
with others as well. Anyone know how JDBC and ODBC deal with encoding?
regards, tom lane
Here is an email from December 15 reporting the problem. Seems we need to address it somehow. Should we throw a warning when the do it? --------------------------------------------------------------------------- Tom Lane wrote: > I said: > > I'd prefer to just document that PQclientEncoding is untrustworthy if > > the client sets the encoding directly (rather than via PGCLIENTENCODING > > or PQsetClientEncoding), and similarly that psql's \encoding is not > > trustworthy if one goes behind its back. > > On looking closer, both libpq and psql assume that their current > internal encoding settings accurately describe the strings they get > from the backend. So hacking up PQclientEncoding() wouldn't be enough > to fix all the problems anyway. > > I think we need to document that doing "SET client_encoding" directly is > hazardous to your health with either of these interfaces, and perhaps > with others as well. Anyone know how JDBC and ODBC deal with encoding? > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Don't 'kill -9' the postmaster > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania 19073
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Should we throw a warning when the do it?
How? The backend can't throw such a warning (it has no way to know if
the SET is coming from the client library or a higher level). The only
way to do it would be for the client library to parse every command
passed through it to look for "SET client_encoding" ... which is
expensive and still not very bulletproof.
Perhaps in the still-mythical 7.4 protocol revision, we could consider
adding a field to backend result messages to show the current client
encoding. But I misdoubt whether it's worth adding a couple bytes to
*every* backend message to make life safe for clients who both want to
change the encoding on the fly, and want to do so behind the back of the
low-level libraries they're using. That's a cost paid by everyone to
satisfy a very small group.
I really think that documentation to the effect of "don't do it this
way, do it that other way" is the appropriate solution.
regards, tom lane