Thread: BUG #14218: pg_logical_slot_get_changes causes segmentation fault
BUG #14218: pg_logical_slot_get_changes causes segmentation fault
From
alexey.kuntsevich@gmail.com
Date:
VGhlIGZvbGxvd2luZyBidWcgaGFzIGJlZW4gbG9nZ2VkIG9uIHRoZSB3ZWJz aXRlOgoKQnVnIHJlZmVyZW5jZTogICAgICAxNDIxOApMb2dnZWQgYnk6ICAg ICAgICAgIEFsZXhleSBLdW50c2V2aWNoCkVtYWlsIGFkZHJlc3M6ICAgICAg YWxleGV5Lmt1bnRzZXZpY2hAZ21haWwuY29tClBvc3RncmVTUUwgdmVyc2lv bjogOS41LjIKT3BlcmF0aW5nIHN5c3RlbTogICBEZWJpYW4gSmVzc2llIDgu NCB4NjQKRGVzY3JpcHRpb246ICAgICAgICAKCldlIGVuYWJsZWQgbG9naWNh bCByZXBsaWNhdGlvbiBvbiBvdXIgcG9zdGdyZXNxbCBjbHVzdGVyLCBjcmVh dGVkIGEgbmV3CnJlcGxpY2F0aW9uIHNsb3Qgd2l0aCB2YW5pbGxhIHRlc3Rf ZGVjb2RpbmcgZGVjb2RlciBhbmQgcmFuIGFuIGFwcCB0aGF0CnBvbGxzIDEw MDAwIHJvd3MgZnJvbSB0aGlzIHNsb3Qgb25jZSBwZXIgc2Vjb25kLiBJdCBy YW4gZmluZSBmb3IgYSBkYXksCnN1cnZpdmVkIHNldmVyYWwgYnVsayB1cGxv YWRzIG9mIH4xbWxuIHJvd3MgdW50aWwgd2UgZGlkIGJ1bGsgdXBsb2FkIG9m Cn44bWxuIHJvd3MgYXQgb25jZS4gRHVyaW5nIHRoZSB1cGxvYWQgb3VyIHBv c3RncmVzcWwgaW5zdGFuY2UgZGlzY29ubmVjdGVkCmFsbCBjbGllbnRzIGFu ZCByZXBvcnRlZCB0aGF0IGl0IGlzIGluIHJlY292ZXJ5LiBDb3JlIGR1bXAg d2FzIGNyZWF0ZWQgYW5kCndoZW4gd2UgY2hlY2tlZCB0aGUgbG9ncyB3ZSBz YXcNCg0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQgWzIzNTk4XTogWzI4LTFd IGRiPSx1c2VyPSxhcHA9LGNsaWVudD0gTE9HOiAgMDAwMDA6CnNlcnZlciBw cm9jZXNzIChQSUQgODM2OSkgd2FzIHRlcm1pbmF0ZWQgYnkgc2lnbmFsIDEx OiBTZWdtZW50YXRpb24gZmF1bHQNCjIwMTYtMDYtMjggMjI6MTY6NTUgR01U IFsyMzU5OF06IFsyOS0xXSBkYj0sdXNlcj0sYXBwPSxjbGllbnQ9IERFVEFJ TDogCkZhaWxlZCBwcm9jZXNzIHdhcyBydW5uaW5nOiBTRUxFQ1QgKiBGUk9N CnBnX2xvZ2ljYWxfc2xvdF9nZXRfY2hhbmdlcygncmVncmVzc2lvbl9zbG90 JywgTlVMTCwgTlVMTCkgTElNSVQgMTAwMDA7DQoyMDE2LTA2LTI4IDIyOjE2 OjU1IEdNVCBbMjM1OThdOiBbMzAtMV0gZGI9LHVzZXI9LGFwcD0sY2xpZW50 PSBMT0NBVElPTjogCkxvZ0NoaWxkRXhpdCwgcG9zdG1hc3Rlci5jOjM0NzEN CjIwMTYtMDYtMjggMjI6MTY6NTUgR01UIFsyMzU5OF06IFszMS0xXSBkYj0s dXNlcj0sYXBwPSxjbGllbnQ9IExPRzogIDAwMDAwOgp0ZXJtaW5hdGluZyBh bnkgb3RoZXIgYWN0aXZlIHNlcnZlciBwcm9jZXNzZXMNCjIwMTYtMDYtMjgg MjI6MTY6NTUgR01UIFsyMzU5OF06IFszMi0xXSBkYj0sdXNlcj0sYXBwPSxj bGllbnQ9IExPQ0FUSU9OOiAKSGFuZGxlQ2hpbGRDcmFzaCwgcG9zdG1hc3Rl ci5jOjMxOTENCjIwMTYtMDYtMjggMjI6MTY6NTUgR01UIFsyNDg0NV06IFs4 NS0xXQpkYj1kdyx1c2VyPWFwcCxhcHA9bW9uaXRvcl9ub2RlLGNsaWVudD08 aGlkZGVuIGlwMT4gV0FSTklORzogIDU3UDAyOgp0ZXJtaW5hdGluZyBjb25u ZWN0aW9uIGJlY2F1c2Ugb2YgY3Jhc2ggb2YgYW5vdGhlciBzZXJ2ZXIgcHJv Y2Vzcw0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQgWzI0ODQ1XTogWzg2LTFd CmRiPWR3LHVzZXI9YXBwLGFwcD1tb25pdG9yX25vZGUsY2xpZW50PTxoaWRk ZW4gaXAxPiBERVRBSUw6ICBUaGUgcG9zdG1hc3RlcgpoYXMgY29tbWFuZGVk IHRoaXMgc2VydmVyIHByb2Nlc3MgdG8gcm9sbCBiYWNrIHRoZSBjdXJyZW50 IHRyYW5zYWN0aW9uIGFuZApleGl0LCBiZWNhdXNlIGFub3RoZXIgc2VydmVy IHByb2Nlc3MgZXhpdGVkIGFibm9ybWFsbHkgYW5kIHBvc3NpYmx5CmNvcnJ1 cHRlZCBzaGFyZWQgbWVtb3J5Lg0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQg WzI0ODQ1XTogWzg3LTFdCmRiPWR3LHVzZXI9YXBwLGFwcD1tb25pdG9yX25v ZGUsY2xpZW50PTxoaWRkZW4gaXAxPiBISU5UOiAgSW4gYSBtb21lbnQgeW91 CnNob3VsZCBiZSBhYmxlIHRvIHJlY29ubmVjdCB0byB0aGUgZGF0YWJhc2Ug YW5kIHJlcGVhdCB5b3VyIGNvbW1hbmQuDQoyMDE2LTA2LTI4IDIyOjE2OjU1 IEdNVCBbMjQ4NDVdOiBbODgtMV0KZGI9ZHcsdXNlcj1hcHAsYXBwPW1vbml0 b3Jfbm9kZSxjbGllbnQ9PGhpZGRlbiBpcDE+IExPQ0FUSU9OOiAgcXVpY2tk aWUsCnBvc3RncmVzLmM6MjYxMg0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQg WzI0ODQ5XTogWzE5My0xXQpkYj1kdyx1c2VyPWFwcCxhcHA9bW9uaXRvcl9u b2RlLGNsaWVudD08aGlkZGVuIGlwMj4gV0FSTklORzogIDU3UDAyOgp0ZXJt aW5hdGluZyBjb25uZWN0aW9uIGJlY2F1c2Ugb2YgY3Jhc2ggb2YgYW5vdGhl ciBzZXJ2ZXIgcHJvY2Vzcw0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQgWzI0 ODQ5XTogWzE5NC0xXQpkYj1kdyx1c2VyPWFwcCxhcHA9bW9uaXRvcl9ub2Rl LGNsaWVudD08aGlkZGVuIGlwMj4gREVUQUlMOiAgVGhlIHBvc3RtYXN0ZXIK aGFzIGNvbW1hbmRlZCB0aGlzIHNlcnZlciBwcm9jZXNzIHRvIHJvbGwgYmFj ayB0aGUgY3VycmVudCB0cmFuc2FjdGlvbiBhbmQKZXhpdCwgYmVjYXVzZSBh bm90aGVyIHNlcnZlciBwcm9jZXNzIGV4aXRlZCBhYm5vcm1hbGx5IGFuZCBw b3NzaWJseQpjb3JydXB0ZWQgc2hhcmVkIG1lbW9yeS4NCjIwMTYtMDYtMjgg MjI6MTY6NTUgR01UIFsyNDg0OV06IFsxOTUtMV0KZGI9ZHcsdXNlcj1hcHAs YXBwPW1vbml0b3Jfbm9kZSxjbGllbnQ9PGhpZGRlbiBpcDI+IEhJTlQ6ICBJ biBhIG1vbWVudCB5b3UKc2hvdWxkIGJlIGFibGUgdG8gcmVjb25uZWN0IHRv IHRoZSBkYXRhYmFzZSBhbmQgcmVwZWF0IHlvdXIgY29tbWFuZC4NCjIwMTYt MDYtMjggMjI6MTY6NTUgR01UIFsyNDg0OV06IFsxOTYtMV0KZGI9ZHcsdXNl cj1hcHAsYXBwPW1vbml0b3Jfbm9kZSxjbGllbnQ9PGhpZGRlbiBpcDI+IExP Q0FUSU9OOiAgcXVpY2tkaWUsCnBvc3RncmVzLmM6MjYxMg0KMjAxNi0wNi0y OCAyMjoxNjo1NSBHTVQgWzgzNzBdOiBbNTgwNTktMV0KZGI9ZHcsdXNlcj1h cHAsYXBwPUJvbnVzU2VydmljZSxjbGllbnQ9PGhpZGRlbiBpcDI+IFdBUk5J Tkc6ICA1N1AwMjoKdGVybWluYXRpbmcgY29ubmVjdGlvbiBiZWNhdXNlIG9m IGNyYXNoIG9mIGFub3RoZXIgc2VydmVyIHByb2Nlc3MNCjIwMTYtMDYtMjgg MjI6MTY6NTUgR01UIFs4MzcwXTogWzU4MDYwLTFdCmRiPWR3LHVzZXI9YXBw LGFwcD1Cb251c1NlcnZpY2UsY2xpZW50PTxoaWRkZW4gaXAyPiBERVRBSUw6 ICBUaGUgcG9zdG1hc3RlcgpoYXMgY29tbWFuZGVkIHRoaXMgc2VydmVyIHBy b2Nlc3MgdG8gcm9sbCBiYWNrIHRoZSBjdXJyZW50IHRyYW5zYWN0aW9uIGFu ZApleGl0LCBiZWNhdXNlIGFub3RoZXIgc2VydmVyIHByb2Nlc3MgZXhpdGVk IGFibm9ybWFsbHkgYW5kIHBvc3NpYmx5CmNvcnJ1cHRlZCBzaGFyZWQgbWVt b3J5Lg0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQgWzgzNzBdOiBbNTgwNjEt MV0KZGI9ZHcsdXNlcj1hcHAsYXBwPUJvbnVzU2VydmljZSxjbGllbnQ9PGhp ZGRlbiBpcDI+IEhJTlQ6ICBJbiBhIG1vbWVudCB5b3UKc2hvdWxkIGJlIGFi bGUgdG8gcmVjb25uZWN0IHRvIHRoZSBkYXRhYmFzZSBhbmQgcmVwZWF0IHlv dXIgY29tbWFuZC4NCjIwMTYtMDYtMjggMjI6MTY6NTUgR01UIFs4MzcwXTog WzU4MDYyLTFdCmRiPWR3LHVzZXI9YXBwLGFwcD1Cb251c1NlcnZpY2UsY2xp ZW50PTxoaWRkZW4gaXAyPiBMT0NBVElPTjogIHF1aWNrZGllLApwb3N0Z3Jl cy5jOjI2MTINCjIwMTYtMDYtMjggMjI6MTY6NTUgR01UIFsyNDg0OF06IFs2 My0xXQpkYj1kdyx1c2VyPWFwcCxhcHA9bW9uaXRvcl9ub2RlLGNsaWVudD08 aGlkZGVuIGlwMj4gV0FSTklORzogIDU3UDAyOgp0ZXJtaW5hdGluZyBjb25u ZWN0aW9uIGJlY2F1c2Ugb2YgY3Jhc2ggb2YgYW5vdGhlciBzZXJ2ZXIgcHJv Y2Vzcw0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQgWzI0ODQ4XTogWzY0LTFd CmRiPWR3LHVzZXI9YXBwLGFwcD1tb25pdG9yX25vZGUsY2xpZW50PTxoaWRk ZW4gaXAyPiBERVRBSUw6ICBUaGUgcG9zdG1hc3RlcgpoYXMgY29tbWFuZGVk IHRoaXMgc2VydmVyIHByb2Nlc3MgdG8gcm9sbCBiYWNrIHRoZSBjdXJyZW50 IHRyYW5zYWN0aW9uIGFuZApleGl0LCBiZWNhdXNlIGFub3RoZXIgc2VydmVy IHByb2Nlc3MgZXhpdGVkIGFibm9ybWFsbHkgYW5kIHBvc3NpYmx5CmNvcnJ1 cHRlZCBzaGFyZWQgbWVtb3J5Lg0KMjAxNi0wNi0yOCAyMjoxNjo1NSBHTVQg WzI0ODQ4XTogWzY1LTFdCmRiPWR3LHVzZXI9YXBwLGFwcD1tb25pdG9yX25v ZGUsY2xpZW50PTxoaWRkZW4gaXAyPiBISU5UOiAgSW4gYSBtb21lbnQgeW91 CnNob3VsZCBiZSBhYmxlIHRvIHJlY29ubmVjdCB0byB0aGUgZGF0YWJhc2Ug YW5kIHJlcGVhdCB5b3VyIGNvbW1hbmQuDQoNCldlJ3JlIGFibGUgdG8gcmVw cm9kdWNlIHRoZSBpc3N1ZSBhdG0uIAoK
On Wed, Jun 29, 2016 at 12:10 AM, <alexey.kuntsevich@gmail.com> wrote: > We enabled logical replication on our postgresql cluster, created a new > replication slot with vanilla test_decoding decoder and ran an app that > polls 10000 rows from this slot once per second. It ran fine for a day, > survived several bulk uploads of ~1mln rows until we did bulk upload of > ~8mln rows at once. During the upload our postgresql instance disconnected > all clients and reported that it is in recovery. Core dump was created and > when we checked the logs we saw Can you show a backtrace from the coredump? https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD -- Peter Geoghegan
Hi, Peter Here is the backtrace: Reading symbols from /usr/lib/postgresql/9.5/bin/postgres...(no debugging symbols found)...done. [New LWP 33731] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `postgres: docker turf 172.30.32.56(50865) SELECT Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007fdd58521f20 in ReorderBufferCommit () (gdb) bt #0 0x00007fdd58521f20 in ReorderBufferCommit () #1 0x00007fdd5851d7b0 in LogicalDecodingProcessRecord () #2 0x00007fdd5851f1c5 in ?? () #3 0x00007fdd58466ac2 in ExecMakeTableFunctionResult () #4 0x00007fdd5847be52 in ?? () #5 0x00007fdd58468c73 in ExecScan () #6 0x00007fdd58461498 in ExecProcNode () #7 0x00007fdd5845e34e in standard_ExecutorRun () #8 0x00007fdd5856b1ff in ?? () #9 0x00007fdd5856c808 in PortalRun () #10 0x00007fdd58569501 in PostgresMain () #11 0x00007fdd58303c31 in ?? () #12 0x00007fdd5850d54e in PostmasterMain () #13 0x00007fdd58304db7 in main () On Wed, Jun 29, 2016 at 9:19 AM, Peter Geoghegan <pg@heroku.com> wrote: > On Wed, Jun 29, 2016 at 12:10 AM, <alexey.kuntsevich@gmail.com> wrote: > > We enabled logical replication on our postgresql cluster, created a new > > replication slot with vanilla test_decoding decoder and ran an app that > > polls 10000 rows from this slot once per second. It ran fine for a day, > > survived several bulk uploads of ~1mln rows until we did bulk upload of > > ~8mln rows at once. During the upload our postgresql instance > disconnected > > all clients and reported that it is in recovery. Core dump was created > and > > when we checked the logs we saw > > Can you show a backtrace from the coredump? > > > https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD > > -- > Peter Geoghegan > -- Best regards, Alexey Kuntsevich
On Wed, Jun 29, 2016 at 4:55 AM, Alexey Kuntsevich <alexey.kuntsevich@gmail.com> wrote: > > Here is the backtrace: > > Reading symbols from /usr/lib/postgresql/9.5/bin/postgres...(no debugging > symbols found)...done. That isn't very useful, since you don't have debugging symbols. See that Wiki page. -- Peter Geoghegan