Thread: weird - invalid string enlargement request size
A simple query: select prec.bl_considerada_prioritaria AS "A proposta é considerada prioritária por Conselho Municipal ou Estadual" from consultaprevia2008.tab_urb_asse_prec prec The result: ERROR: invalid string enlargement request size 1073741823 But.. If i replace the accented char, leaving the query like: select prec.bl_considerada_prioritaria AS "A proposta e considerada prioritaria por Conselho Municipal ou Estadual" from consultaprevia2008.tab_urb_asse_prec prec More weird yet: This is part of a large query. I have the keyword "as", using accented chars in some other places, and works ok. My pg version: "PostgreSQL 8.2.5 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)" This is a restored backup from 8.1 . In 8.1 it works ok. []'s - Walter
Walter Cruz escribió: > A simple query: > > select prec.bl_considerada_prioritaria > AS "A proposta é considerada prioritária por Conselho Municipal ou Estadual" > from consultaprevia2008.tab_urb_asse_prec prec > > The result: ERROR: invalid string enlargement request size 1073741823 Weird. Probably encoding mismatch. What are your locale and encoding settings, and what encoding is the actual client? And what client is it? Easiest way to show server settings is: select name, setting from pg_settings where name ~~ any ( array['lc_%', '%_encoding']); -- Alvaro Herrera http://www.advogato.org/person/alvherre "La Primavera ha venido. Nadie sabe como ha sido" (A. Machado)
Yes, encoding mismatch [:)] "client_encoding";"UNICODE" "lc_collate";"pt_BR.UTF-8" "lc_ctype";"pt_BR.UTF-8" "lc_messages";"pt_BR.UTF_8" "lc_monetary";"pt_BR.UTF_8" "lc_numeric";"pt_BR.UTF_8" "lc_time";"pt_BR.UTF_8" "server_encoding";"LATIN1" I'm with some troubles to set the encoding In my PHP program, but this is not hackers issue :) I posted on hackers cause I think that was a bug, os something. []'s - Walter On Dec 4, 2007 3:46 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote: > Walter Cruz escribió: > > A simple query: > > > > select prec.bl_considerada_prioritaria > > AS "A proposta é considerada prioritária por Conselho Municipal ou Estadual" > > from consultaprevia2008.tab_urb_asse_prec prec > > > > The result: ERROR: invalid string enlargement request size 1073741823 > > Weird. Probably encoding mismatch. What are your locale and encoding > settings, and what encoding is the actual client? And what client is > it? Easiest way to show server settings is: > > select name, setting > from pg_settings > where name ~~ any ( array['lc_%', '%_encoding']); > > -- > Alvaro Herrera http://www.advogato.org/person/alvherre > "La Primavera ha venido. Nadie sabe como ha sido" (A. Machado) >
"Walter Cruz" <walter.php@gmail.com> writes:
> I posted on hackers cause I think that was a bug, os something.
Yeah, I think so too. Can you extract a reproducible test case?
regards, tom lane
On Dec 4, 2007 7:19 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > "Walter Cruz" <walter.php@gmail.com> writes: > > I posted on hackers cause I think that was a bug, os something. > > Yeah, I think so too. Can you extract a reproducible test case? I'm trying to extract that case! []'s - Walter
Yes, I did. What have I done. My Linux version: Debian Etch. My Postgres version: "PostgreSQL 8.2.5 on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)" It was compiled by myself. My initdb was: initdb -E LATIN1 --locale=pt_BR By that initdb, the $LANG of the system was pt_BR.UTF-8 . The pt_BR was not even enabled. A simple query that shows the problem: select true AS "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" (it's needed this exact number of chars, with one of them accented.) The output of the SQL of Alvaro in this server is: "client_encoding";"UNICODE" "lc_collate";"pt_BR.UTF-8" "lc_ctype";"pt_BR.UTF-8" "lc_messages";"pt_BR" "lc_monetary";"pt_BR" "lc_numeric";"pt_BR" "lc_time";"pt_BR" "server_encoding";"LATIN1" (I have enabled the pt_BR locale before the tests, but lc_collate and lc_type are still pt_BR.UTF-8, due to initdb). Just after that, I changed my $LANG to pt_BR, did a initdb. The query select true AS "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; returns: template1-# ; NOTICE: identifier "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" will be truncated to "áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"áaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa On this second initdb: template1=# select name, setting template1-# from pg_settings template1-# where name ~~ any ( array['lc_%', '%_encoding']); name | setting -----------------+---------client_encoding | LATIN1lc_collate | pt_BRlc_ctype | pt_BRlc_messages | pt_BRlc_monetary | pt_BRlc_numeric | pt_BRlc_time | pt_BRserver_encoding | LATIN1 Is that information enough? I can give you more details, if needed. []'s - Walter
"Walter Cruz" <walter.php@gmail.com> writes:
> My initdb was:
> initdb -E LATIN1 --locale=pt_BR
> By that initdb, the $LANG of the system was pt_BR.UTF-8 .
> A simple query that shows the problem:
> select true AS "�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
OK, I was able to reproduce this here:
postgres=# select true AS "�aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
;
ERROR: invalid string enlargement request size 1073741823
What is happening is that vsnprintf() is failing because it doesn't like
its input data. elog.c's EVALUATE_MESSAGE macro thinks that the only
possible reason for failure is that the output buffer isn't large
enough, so it enlarges and tries again, leading soon to the palloc error
displayed above.
Possibly we should put some effort into producing a more useful error
message here, but I'm reluctant to fool with it, because our historical
experience is that vsnprintf's behavior just isn't very consistent
across platforms.
In any case, the bottom-line reason why you're having a problem is that
the database encoding (LATIN1) is inconsistent with the encoding
specified by the server's locale (UTF-8), thus allowing vsnprintf to see
what it thinks is bad data.
PG 8.3 contains defenses to prevent that sort of inconsistency, and
I think that's probably all we need to do about this.
regards, tom lane