Thread: Insert Assertion Failed in strcoll_l.c:112
Hi,
The following is written to /var/log/postgresql/postgresql-9.1-main.log:
127.0.0.1(59800) INSERT: strcoll_l.c:112: __strcoll_l: Assertion `((uintptr_t)
indirect) % __alignof__ (indirect[0]) == 0' failed.
2012-12-23 08:54:54 SGT LOG: server process (PID 1964) was terminated by signal
6: Aborted
2012-12-23 08:54:54 SGT LOG: terminating any other active server processes
2012-12-23 08:54:54 SGT WARNING: terminating connection because of crash of ano
ther server process
2012-12-23 08:54:54 SGT DETAIL: The postmaster has commanded this server proces
s to roll back the current transaction and exit, because another server process
exited abnormally and possibly corrupted shared memory.
2012-12-23 08:54:54 SGT HINT: In a moment you should be able to reconnect to th
e database and repeat your command.
Error occurring on Ubuntu 12.04 with PostgreSQL 9.1 on Dell PowerEdge R710
Same code executes successfully on Ubuntu 12.10 with PostgreSQL 9.1 on Dell Precision 390.
Application is a libpq c-application. Application is inserting a large number of records into the DB, for later query. Uses a thread pool and connection pool. Connection pool establishes connections up-front and hands them to threads when required. Connection pools creates 5 prepared statements to assist with the inserts on each connection. 70 Connections in the connection pool, up to 150 threads in the thread pool. Mutex's manage contention on the connection pool. Application successfully inserts 100+ records in the first table and 1000+ records in the second table before failing. After the above error all connections begin to report 'no connection' to the database.
Configuration in /etc/postgresql/9.1/main/postgresql.conf is as follows:
fysnc =on
full_page_writes = on
max_connections = 150
max_prepared_transactions = 350
max_locks_per_transaction = 10
Configuration in /etc/sysctl.conf :
kernel.shmmax=131072000
Feels like a configuration issue. Any suggestions on where to start looking would be appreciated.
Kind regards,
Michael
The following is written to /var/log/postgresql/postgresql-9.1-main.log:
127.0.0.1(59800) INSERT: strcoll_l.c:112: __strcoll_l: Assertion `((uintptr_t)
indirect) % __alignof__ (indirect[0]) == 0' failed.
2012-12-23 08:54:54 SGT LOG: server process (PID 1964) was terminated by signal
6: Aborted
2012-12-23 08:54:54 SGT LOG: terminating any other active server processes
2012-12-23 08:54:54 SGT WARNING: terminating connection because of crash of ano
ther server process
2012-12-23 08:54:54 SGT DETAIL: The postmaster has commanded this server proces
s to roll back the current transaction and exit, because another server process
exited abnormally and possibly corrupted shared memory.
2012-12-23 08:54:54 SGT HINT: In a moment you should be able to reconnect to th
e database and repeat your command.
Error occurring on Ubuntu 12.04 with PostgreSQL 9.1 on Dell PowerEdge R710
Same code executes successfully on Ubuntu 12.10 with PostgreSQL 9.1 on Dell Precision 390.
Application is a libpq c-application. Application is inserting a large number of records into the DB, for later query. Uses a thread pool and connection pool. Connection pool establishes connections up-front and hands them to threads when required. Connection pools creates 5 prepared statements to assist with the inserts on each connection. 70 Connections in the connection pool, up to 150 threads in the thread pool. Mutex's manage contention on the connection pool. Application successfully inserts 100+ records in the first table and 1000+ records in the second table before failing. After the above error all connections begin to report 'no connection' to the database.
Configuration in /etc/postgresql/9.1/main/postgresql.conf is as follows:
fysnc =on
full_page_writes = on
max_connections = 150
max_prepared_transactions = 350
max_locks_per_transaction = 10
Configuration in /etc/sysctl.conf :
kernel.shmmax=131072000
Feels like a configuration issue. Any suggestions on where to start looking would be appreciated.
Kind regards,
Michael
Michael Arnold <myk321@gmail.com> writes: > The following is written to /var/log/postgresql/postgresql-9.1-main.log: > 127.0.0.1(59800) INSERT: strcoll_l.c:112: __strcoll_l: Assertion > `((uintptr_t) > indirect) % __alignof__ (indirect[0]) == 0' failed. This appears to be a failure inside glibc (specifically strcoll_l()). There is certainly noplace in Postgres proper that would print such a message. > Error occurring on Ubuntu 12.04 with PostgreSQL 9.1 on Dell PowerEdge R710 > Same code executes successfully on Ubuntu 12.10 with PostgreSQL 9.1 on Dell > Precision 390. I would take that to mean this is a glibc bug that was fixed between Ubuntu releases 12.04 and 12.10. Update the older system and see if it doesn't go away. regards, tom lane
Thanks Tom, but no luck this time: both boxes now on Ubuntu 12.10, but same error. Also removed -march gcc switch, likewise with no impact on error. Kind regards, Michael From: Tom Lane Sent: 24/12/2012 1:16 AM To: Michael Arnold Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Insert Assertion Failed in strcoll_l.c:112 Michael Arnold <myk321@gmail.com> writes: > The following is written to /var/log/postgresql/postgresql-9.1-main.log: > 127.0.0.1(59800) INSERT: strcoll_l.c:112: __strcoll_l: Assertion > `((uintptr_t) > indirect) % __alignof__ (indirect[0]) == 0' failed. This appears to be a failure inside glibc (specifically strcoll_l()). There is certainly noplace in Postgres proper that would print such a message. > Error occurring on Ubuntu 12.04 with PostgreSQL 9.1 on Dell PowerEdge R710 > Same code executes successfully on Ubuntu 12.10 with PostgreSQL 9.1 on Dell > Precision 390. I would take that to mean this is a glibc bug that was fixed between Ubuntu releases 12.04 and 12.10. Update the older system and see if it doesn't go away. regards, tom lane
Michael Arnold <myk321@gmail.com> writes: > Thanks Tom, but no luck this time: both boxes now on Ubuntu 12.10, but > same error. Hm. Are they using identical locale settings? Can you extract a self-contained test case? It's still a glibc bug AFAICS, but without a test case you'll not likely get any traction reporting to the glibc guys either. regards, tom lane
Thanks Tom, but no luck this time: both boxes now on Ubuntu 12.10, but same error. Also removed -march gcc switch, likewise with no impact on error. Kind regards, Michael From: Tom Lane Sent: 24/12/2012 1:16 AM To: Michael Arnold Cc: pgsql-general@postgresql.org Subject: Re: [GENERAL] Insert Assertion Failed in strcoll_l.c:112 Michael Arnold <myk321@gmail.com> writes: > The following is written to /var/log/postgresql/postgresql-9.1-main.log: > 127.0.0.1(59800) INSERT: strcoll_l.c:112: __strcoll_l: Assertion > `((uintptr_t) > indirect) % __alignof__ (indirect[0]) == 0' failed. This appears to be a failure inside glibc (specifically strcoll_l()). There is certainly noplace in Postgres proper that would print such a message. > Error occurring on Ubuntu 12.04 with PostgreSQL 9.1 on Dell PowerEdge R710 > Same code executes successfully on Ubuntu 12.10 with PostgreSQL 9.1 on Dell > Precision 390. I would take that to mean this is a glibc bug that was fixed between Ubuntu releases 12.04 and 12.10. Update the older system and see if it doesn't go away. regards, tom lane
Michael Arnold <myk321@gmail.com> writes: > The locale is the same on both machines: "en_SG.UTF-8" > I've extracted the smallest set of code which still produces the > error. Unfortunately its still more than 1000 lines of code, as > without the thread and connection pools the error goes away. If only > 1 thread or only 1 connection is used the error also goes away. Hm. I ran this several times on a Fedora 16 box with locale en_SG.utf8, and didn't see any assertion failures. I did see a fair amount of ERROR: invalid byte sequence for encoding "UTF8": 0xf0 0x66 0x02 STATEMENT: INSERT INTO TABLE1( FIELD1, FIELD2, FIELD3, FIELD4) VALUES( $1, $2, $3, $4); with varying specific data being complained of. Some investigation disclosed the reason for that --- the test program is setting up the strings to be inserted like this: strncpy(DataFileInfo->Field1, "AAA", 3); strncpy(DataFileInfo->Field2, "BBB", 3); which of course fails to copy any trailing \0 into the struct, so unless the struct happens to contain zeroes to start with (which is likely but not guaranteed for a freshly-malloced block), you get strings that appear to contain platform-specific garbage. Changing the lengths to "4" made the encoding complaints go away. So at this point I surmise that the issue is triggered by unexpected non-ASCII data, but it's going to be unlikely that anybody else can reproduce it on the basis of this example. Also, this bug in your test case may or may not have much to do with the original problem... regards, tom lane