Thread: chr() function leads to OOM / killed connection with 8.1, 8.2
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, I found something that I believe to be a bug in postgresql handling of the chr function. This function takes an ascii code and returns the character. Now it seems that a value greater than 191 seems to cause trouble with a backend instance of postgresql. I verified this on postgres 8.1.8, 8.1.9 and 8.2.4. I could not trigger the behaviour with 8.1.3 or 8.1.5. I did not test other versions. The effects are: 8.1.8 / 8.1.9: transport=> select id,msisdn,replace(msisdn, chr(192), 'FF') from msisdnmap limit 2; ERROR: out of memory DETAIL: Failed on request of size 17. The backend process consumes a lot memory and uses up all available CPU cycles. The logfile gives OOM statistics. I can provide that if needed. 8.2.4: dwh=> select replace(customer_id, chr(192), 'FF') from sms_customer limit 1; server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request. The connection to the server was lost. Attempting reset: Failed. !> the logfile says: 2007-07-19 18:07:49 CEST @[5385]LOG: server process (PID 15351) was terminated by signal 11 2007-07-19 18:07:49 CEST @[5385]LOG: terminating any other active server processes 2007-07-19 18:07:49 CEST dwh@dwh[15352]FATAL: the database system is in recovery mode 2007-07-19 18:07:49 CEST @[5385]LOG: all server processes terminated; reinitializing 2007-07-19 18:07:49 CEST @[15353]LOG: database system was interrupted at 2007-07-19 12:29:33 CEST 2007-07-19 18:07:49 CEST @[15353]LOG: checkpoint record is at 9/6552A6C0 2007-07-19 18:07:49 CEST @[15353]LOG: redo record is at 9/6552A6C0; undo record is at 0/0; shutdown TRUE 2007-07-19 18:07:49 CEST @[15353]LOG: next transaction ID: 0/2581; next OID: 24576 2007-07-19 18:07:49 CEST @[15353]LOG: next MultiXactId: 1; next MultiXactOffset: 0 2007-07-19 18:07:49 CEST @[15353]LOG: database system was not properly shut down; automatic recovery in progress 2007-07-19 18:07:50 CEST @[15353]LOG: record with zero length at 9/6552A708 2007-07-19 18:07:50 CEST @[15353]LOG: redo is not required 2007-07-19 18:07:50 CEST @[15353]LOG: database system is ready This behavior is independent of column type, so it should be easy to reproduce. Please verify that this is indeed a bug and not a mistake on our side. Thank you. - -- Regards, Wiktor Wodecki net mobile AG, Zollhof 17, 40221 Duesseldorf, Germany 923B DCF8 070C 9FDD 5E05 9AE3 E923 5A35 182C 9783 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGn45Z6SNaNRgsl4MRAko2AJ9WZN0vUjs9f8IqYDJLzLFsk1szRwCfeTEp WWaquyaMz0xjMORmfDTRtAA= =eDv/ -----END PGP SIGNATURE-----
Wiktor Wodecki <wiktor.wodecki@Net-m.de> writes: > I found something that I believe to be a bug in postgresql handling of > the chr function. This function takes an ascii code and returns the > character. What database encoding are you testing in? regards, tom lane
Tom Lane wrote: > Wiktor Wodecki <wiktor.wodecki@Net-m.de> writes: >> I found something that I believe to be a bug in postgresql handling of >> the chr function. This function takes an ascii code and returns the >> character. > > What database encoding are you testing in? FWIW, I can reproduce this with UTF-8, on REL_8_2_STABLE. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki@enterprisedb.com> writes: > FWIW, I can reproduce this with UTF-8, on REL_8_2_STABLE. I can reproduce an out-of-memory condition (basically, replace() is going into an infinite loop because of the invalid input) but I'm not seeing any crash. regards, tom lane
Tom Lane wrote: > Heikki Linnakangas <heikki@enterprisedb.com> writes: >> FWIW, I can reproduce this with UTF-8, on REL_8_2_STABLE. > > I can reproduce an out-of-memory condition (basically, replace() is > going into an infinite loop because of the invalid input) but I'm > not seeing any crash. replace_text reads past the end of source string, byte by byte (or character by character, not sure), and eventually tries to read from an invalid address which causes a segfault. It happens here when start_posn == 367368. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki@enterprisedb.com> writes: > Tom Lane wrote: >> I can reproduce an out-of-memory condition (basically, replace() is >> going into an infinite loop because of the invalid input) but I'm >> not seeing any crash. > replace_text reads past the end of source string, byte by byte (or > character by character, not sure), and eventually tries to read from an > invalid address which causes a segfault. It happens here when start_posn > == 367368. Hm, must be memory-layout-dependent. On mine, the output string buffer is growing fast enough to ensure there's still RAM to read, up till the kernel says no more. Anyway the problem is that pg_utf2wchar_with_len silently drops the trailing incomplete character in its input, causing text_position_next to think the pattern is empty, causing an infinite loop because curr_posn never advances. replace_text already tried to guard against empty pattern, but it doesn't know about this case. What I intend to do to fix this is to modify the users of text_position_next to believe the string lengths saved by text_position_setup, rather than using TEXTLEN() to compute the lengths. This will effectively make replace_text and friends consistently act as though the partial character isn't there. In the long run it might be better to make pg_utf2wchar_with_len throw an error for bad input, but I'm quite unsure of the consequences of that, in view of the existing comment "not ours to throw error". Anyway such a potentially-significant behavioral change doesn't seem like a good idea to back-patch. (We seem to have this bug in one form or another clear back to 7.3...) regards, tom lane
Tom Lane wrote: > Heikki Linnakangas <heikki@enterprisedb.com> writes: >> Tom Lane wrote: >>> I can reproduce an out-of-memory condition (basically, replace() is >>> going into an infinite loop because of the invalid input) but I'm >>> not seeing any crash. > >> replace_text reads past the end of source string, byte by byte (or >> character by character, not sure), and eventually tries to read from an >> invalid address which causes a segfault. It happens here when start_posn >> == 367368. > > Hm, must be memory-layout-dependent. On mine, the output string buffer > is growing fast enough to ensure there's still RAM to read, up till the > kernel says no more. > > Anyway the problem is that pg_utf2wchar_with_len silently drops the > trailing incomplete character in its input, causing text_position_next > to think the pattern is empty, causing an infinite loop because > curr_posn never advances. replace_text already tried to guard against > empty pattern, but it doesn't know about this case. > > What I intend to do to fix this is to modify the users of > text_position_next to believe the string lengths saved by > text_position_setup, rather than using TEXTLEN() to compute > the lengths. This will effectively make replace_text and > friends consistently act as though the partial character isn't there. > > In the long run it might be better to make pg_utf2wchar_with_len throw > an error for bad input, but I'm quite unsure of the consequences of > that, in view of the existing comment "not ours to throw error". > Anyway such a potentially-significant behavioral change doesn't seem > like a good idea to back-patch. (We seem to have this bug in one form > or another clear back to 7.3...) I agree we should do the above changes for the sake of robustness, but isn't the real problem here that chr function can return invalid byte sequences? That was actually discussed a while back (starting at http://archives.postgresql.org/pgsql-hackers/2007-04/msg00010.php), but that was inconclusive. IMHO chr should at the very least not return invalid byte sequences. Limiting it to ascii range is not a bad idea either, though that might break some applications. Is there any other known loopholes to get invalid data in the database? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki@enterprisedb.com> writes: > I agree we should do the above changes for the sake of robustness, but > isn't the real problem here that chr function can return invalid byte > sequences? That was actually discussed a while back (starting at > http://archives.postgresql.org/pgsql-hackers/2007-04/msg00010.php), but > that was inconclusive. My thought about that is that there will *always* be ways to get invalidly encoded text into the database (via bugs if nothing else), and so we had better write code that doesn't crash on it. Allowing chr() to act the way it does at least makes it relatively easy to test such cases. > Is there any other known loopholes to get invalid data in the database? The convert() function is a loophole by definition. Someday we may be able to label individual strings with their encodings, which would allow convert() to behave in a less unsafe way. I'd be a bit inclined to define chr() as returning LATIN1, or some other popular single-byte encoding, when that happens. regards, tom lane
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tom Lane wrote: > Wiktor Wodecki <wiktor.wodecki@Net-m.de> writes: >> I found something that I believe to be a bug in postgresql handling of >> the chr function. This function takes an ascii code and returns the >> character. > > What database encoding are you testing in? As Heikki already found out, this is tested on UTF-8. - -- Regards, Wiktor Wodecki net mobile AG, Zollhof 17, 40221 Duesseldorf, Germany 923B DCF8 070C 9FDD 5E05 9AE3 E923 5A35 182C 9783 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGoFhR6SNaNRgsl4MRAmELAKDBDIAu5jflbFD5691hN78bEZNILwCglBzi 7v+UX+eR1G80EK8foIThNTM= =e/A5 -----END PGP SIGNATURE-----