Thread: BUG #14180: Segmentation fault on replication slave
VGhlIGZvbGxvd2luZyBidWcgaGFzIGJlZW4gbG9nZ2VkIG9uIHRoZSB3ZWJz aXRlOgoKQnVnIHJlZmVyZW5jZTogICAgICAxNDE4MApMb2dnZWQgYnk6ICAg ICAgICAgIEJvIMOYcnN0ZWQgQW5kcmVzZW4KRW1haWwgYWRkcmVzczogICAg ICBib2FAbmVvZ3JpZC5kawpQb3N0Z3JlU1FMIHZlcnNpb246IDkuNS4zCk9w ZXJhdGluZyBzeXN0ZW06ICAgVWJ1bnR1IDE2LjA0IExUUwpEZXNjcmlwdGlv bjogICAgICAgIAoKSGVsbG8sDQoNCldlIGhhdmUgYSByZXBsaWNhdGlvbiBz bG90IHNldHVwIHdoZXJlIHRoZSByZXBsaWNhdGlvbiBjYXVzZXMgYSBzZWdt ZW50YXRpb24KZmF1bHQgd2l0aGluIGVpZ2h0IGhvdXJzIGFmdGVyIGEgcmVi dWlsZCBvZiB0aGUgc2xhdmUuDQoNCkluIHRoZSBmb2xsb3dpbmcgdGhlIG1h c3RlciBpcyBJUCAxMC4wLjAuMiBhbmQgdGhlIHNsYXZlIGlzIElQIDEwLjAu MC4zLg0KDQpPbiB0aGUgbWFzdGVyIHdlIGhhdmUgdGhlIGRlZmF1bHRzIGFu ZCB0aGUgZm9sbG93aW5nOg0KDQovZXRjL3Bvc3RncmVzcWwvOS41L21haW4v cG9zdGdyZXNxbC5jb25mDQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tDQpsaXN0ZW5fYWRkcmVzc2VzID0gJyonDQpwb3J0ID0g NTQzMw0Kd2FsX2xldmVsID0gYXJjaGl2ZQ0KYXJjaGl2ZV9tb2RlID0gb24N CmFyY2hpdmVfY29tbWFuZCA9ICd0ZXN0ICEgLWYgL21udC9wb3N0Z3Jlc19h cmNoaXZlLyVmICYmIGNwICVwCi9tbnQvcG9zdGdyZXNfYXJjaGl2ZS8lZicN Cm1heF93YWxfc2VuZGVycyA9IDUNCndhbF9rZWVwX3NlZ21lbnRzID0gNDAw MA0KbWF4X3JlcGxpY2F0aW9uX3Nsb3RzID0gMQ0KdGltZXpvbmUgPSAnVVRD Jw0KLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQ0K DQovZXRjL3Bvc3RncmVzcWwvOS41L21haW4vcGdfaGJhLmNvbmYNCi0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0NCmhvc3QgICAg cmVwbGljYXRpb24gICAgIHBvc3RncmVzICAgICAgICAgMTAuMC4wLjMvMzIg ICAgICAgIHRydXN0DQotLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tDQoNCk9uIHRoZSBzbGF2ZSB3ZSBoYXZlIHRoZSBkZWZhdWx0 cyBhbmQgdGhlIHRpbWV6b25lIGNoYW5nZWQ6DQoNCi9ldGMvcG9zdGdyZXNx bC85LjUvbWFpbi9wb3N0Z3Jlc3FsLmNvbmYuDQotLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQp0aW1lem9uZSA9ICdVVEMnDQot LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tDQoNCk9u IHRoZSBtYXN0ZXIgd2UgcnVuIHRoZSBTUUwgcXVlcnk6DQoNClNFTEVDVCAq IEZST00gcGdfY3JlYXRlX3BoeXNpY2FsX3JlcGxpY2F0aW9uX3Nsb3QoJ3Ns YXZlJyk7DQoNCk9uIHRoZSBzbGF2ZSB3ZSBydW4gdGhlIGNvbW1hbmQ6DQoN CnBnX2Jhc2ViYWNrdXAgLVAgLVIgLVggc3RyZWFtIC1jIGZhc3QgLWggMTAu MC4wLjIgLXAgNTQzMyAtVSBwb3N0Z3JlcyAtRAovdmFyL2xpYi9wb3N0Z3Jl c3FsLzkuNS9tYWluDQoNCkFmdGVyIHRoaXMgcmVjb3ZlcnkuY29uZiBsb29r cyBsaWtlIHRoaXMgKHdoZXJlIHdlIGFkZGVkIHRoZSBzbG90IGxpbmUpOg0K DQpzdGFuZGJ5X21vZGUgPSAnb24nDQpwcmltYXJ5X2Nvbm5pbmZvID0gJ3Vz ZXI9cG9zdGdyZXMgaG9zdD0xMC4wLjAuMiBwb3J0PTU0MzMgc3NsbW9kZT1w cmVmZXIKc3NsY29tcHJlc3Npb249MSBrcmJzcnZuYW1lPXBvc3RncmVzJw0K cHJpbWFyeV9zbG90X25hbWUgPSAnc2xhdmUnDQoNClRoZW4gd2UgZml4IHRo ZSBvd25lcnNoaXAgYW5kIHN0YXJ0IHRoZSBzbGF2ZSBkYXRhYmFzZS4gQWZ0 ZXIgYSB3aGlsZSAtCmFueXRoaW5nIGJldHdlZW4gdGVuIG1pbnV0ZXMgYW5k IGVpZ2h0IGhvdXJzIHdlIGdldCB0aGlzIGVycm9yIGluIHRoZSBsb2cKZmls ZToNCg0KMjAxNi0wNi0wMyAwNTo1NToyNyBVVEMgWzI3MzAzLTRdIExPRzog IHN0YXJ0dXAgcHJvY2VzcyAoUElEIDI3MzA1KSB3YXMKdGVybWluYXRlZCBi eSBzaWduYWwgMTE6IFNlZ21lbnRhdGlvbiBmYXVsdA0KMjAxNi0wNi0wMyAw NTo1NToyNyBVVEMgWzI3MzAzLTVdIExPRzogIHRlcm1pbmF0aW5nIGFueSBv dGhlciBhY3RpdmUgc2VydmVyCnByb2Nlc3Nlcw0KDQpJZiB3ZSBhdHRhY2gg d2l0aCBnZGIgYmVmb3JlIHRoZSBzZWdtZW50YXRpb24gZmF1bHQgd2UgZ2V0 Og0KDQojIGdkYiAtcCAzMDUyNA0KR05VIGdkYiAoVWJ1bnR1IDcuMTEtMHVi dW50dTEpIDcuMTENCkNvcHlyaWdodCAoQykgMjAxNiBGcmVlIFNvZnR3YXJl IEZvdW5kYXRpb24sIEluYy4NCkxpY2Vuc2UgR1BMdjMrOiBHTlUgR1BMIHZl cnNpb24gMyBvciBsYXRlcgo8aHR0cDovL2dudS5vcmcvbGljZW5zZXMvZ3Bs Lmh0bWw+DQpUaGlzIGlzIGZyZWUgc29mdHdhcmU6IHlvdSBhcmUgZnJlZSB0 byBjaGFuZ2UgYW5kIHJlZGlzdHJpYnV0ZSBpdC4NClRoZXJlIGlzIE5PIFdB UlJBTlRZLCB0byB0aGUgZXh0ZW50IHBlcm1pdHRlZCBieSBsYXcuICBUeXBl ICJzaG93IGNvcHlpbmciDQphbmQgInNob3cgd2FycmFudHkiIGZvciBkZXRh aWxzLg0KVGhpcyBHREIgd2FzIGNvbmZpZ3VyZWQgYXMgIng4Nl82NC1saW51 eC1nbnUiLg0KVHlwZSAic2hvdyBjb25maWd1cmF0aW9uIiBmb3IgY29uZmln dXJhdGlvbiBkZXRhaWxzLg0KRm9yIGJ1ZyByZXBvcnRpbmcgaW5zdHJ1Y3Rp b25zLCBwbGVhc2Ugc2VlOg0KPGh0dHA6Ly93d3cuZ251Lm9yZy9zb2Z0d2Fy ZS9nZGIvYnVncy8+Lg0KRmluZCB0aGUgR0RCIG1hbnVhbCBhbmQgb3RoZXIg ZG9jdW1lbnRhdGlvbiByZXNvdXJjZXMgb25saW5lIGF0Og0KPGh0dHA6Ly93 d3cuZ251Lm9yZy9zb2Z0d2FyZS9nZGIvZG9jdW1lbnRhdGlvbi8+Lg0KRm9y IGhlbHAsIHR5cGUgImhlbHAiLg0KVHlwZSAiYXByb3BvcyB3b3JkIiB0byBz ZWFyY2ggZm9yIGNvbW1hbmRzIHJlbGF0ZWQgdG8gIndvcmQiLg0KQXR0YWNo aW5nIHRvIHByb2Nlc3MgMzA1MjQNClJlYWRpbmcgc3ltYm9scyBmcm9tIC91 c3IvbGliL3Bvc3RncmVzcWwvOS41L2Jpbi9wb3N0Z3Jlcy4uLlJlYWRpbmcg c3ltYm9scwpmcm9tCi91c3IvbGliL2RlYnVnLy5idWlsZC1pZC9jNi83NDQ0 Y2FlMmRiYzZiY2FjNDZlODA1MjkyMWMwMWMwNjc4MGQ3Mi5kZWJ1Zy4uLmRv bmUuDQpkb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9saWIveDg2 XzY0LWxpbnV4LWdudS9saWJ4bWwyLnNvLjIuLi5SZWFkaW5nCnN5bWJvbHMg ZnJvbQovdXNyL2xpYi9kZWJ1Zy8uYnVpbGQtaWQvYTEvNTVjN2JjMzQ1ZDBl MGI3MTFiZTA5MTIwMjA0YmQ4OGY0NzVmOWUuZGVidWcuLi5kb25lLg0KZG9u ZS4NClJlYWRpbmcgc3ltYm9scyBmcm9tIC9saWIveDg2XzY0LWxpbnV4LWdu dS9saWJwYW0uc28uMC4uLihubyBkZWJ1Z2dpbmcKc3ltYm9scyBmb3VuZCku Li5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL2xpYi94ODZfNjQtbGlu dXgtZ251L2xpYnNzbC5zby4xLjAuMC4uLlJlYWRpbmcgc3ltYm9scwpmcm9t Ci91c3IvbGliL2RlYnVnLy5idWlsZC1pZC84Mi8yNzU0Njk1ZTRiMzFhZTgy OTM3MjU4YmRmZjNkNTJlZmEwYmEzNi5kZWJ1Zy4uLmRvbmUuDQpkb25lLg0K UmVhZGluZyBzeW1ib2xzIGZyb20gL2xpYi94ODZfNjQtbGludXgtZ251L2xp YmNyeXB0by5zby4xLjAuMC4uLlJlYWRpbmcKc3ltYm9scyBmcm9tCi91c3Iv bGliL2RlYnVnLy5idWlsZC1pZC9iNy81YTk2YzU5YmUxYjViNTRmYmYxYTkx ZWQ3MjJiZWM5NDA2Mjg4ZS5kZWJ1Zy4uLmRvbmUuDQpkb25lLg0KUmVhZGlu ZyBzeW1ib2xzIGZyb20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJn c3NhcGlfa3JiNS5zby4yLi4uKG5vCmRlYnVnZ2luZyBzeW1ib2xzIGZvdW5k KS4uLmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvbGliL3g4Nl82NC1s aW51eC1nbnUvbGlicnQuc28uMS4uLlJlYWRpbmcgc3ltYm9scyBmcm9tCi91 c3IvbGliL2RlYnVnLy9saWIveDg2XzY0LWxpbnV4LWdudS9saWJydC0yLjIz LnNvLi4uZG9uZS4NCmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvbGli L3g4Nl82NC1saW51eC1nbnUvbGliZGwuc28uMi4uLlJlYWRpbmcgc3ltYm9s cyBmcm9tCi91c3IvbGliL2RlYnVnLy9saWIveDg2XzY0LWxpbnV4LWdudS9s aWJkbC0yLjIzLnNvLi4uZG9uZS4NCmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMg ZnJvbSAvbGliL3g4Nl82NC1saW51eC1nbnUvbGlibS5zby42Li4uUmVhZGlu ZyBzeW1ib2xzIGZyb20KL3Vzci9saWIvZGVidWcvL2xpYi94ODZfNjQtbGlu dXgtZ251L2xpYm0tMi4yMy5zby4uLmRvbmUuDQpkb25lLg0KUmVhZGluZyBz eW1ib2xzIGZyb20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJsZGFw X3ItMi40LnNvLjIuLi5SZWFkaW5nCnN5bWJvbHMgZnJvbQovdXNyL2xpYi9k ZWJ1Zy8uYnVpbGQtaWQvYWQvZjZmNDFmMjIzZDQyMTkzMTY1ZmEwYzU1ODcx ZjAyZDkxNWZiMTkuZGVidWcuLi5kb25lLg0KZG9uZS4NClJlYWRpbmcgc3lt Ym9scyBmcm9tIC9saWIveDg2XzY0LWxpbnV4LWdudS9saWJjLnNvLjYuLi5S ZWFkaW5nIHN5bWJvbHMgZnJvbQovdXNyL2xpYi9kZWJ1Zy8vbGliL3g4Nl82 NC1saW51eC1nbnUvbGliYy0yLjIzLnNvLi4uZG9uZS4NCmRvbmUuDQpSZWFk aW5nIHN5bWJvbHMgZnJvbSAvdXNyL2xpYi94ODZfNjQtbGludXgtZ251L2xp YmljdXVjLnNvLjU1Li4uUmVhZGluZwpzeW1ib2xzIGZyb20KL3Vzci9saWIv ZGVidWcvLmJ1aWxkLWlkLzMyLzNlNDg3ODA3M2JiNGUwZDdiMTc0YWUyNGUz ODNlYzVlMDVkNjhhLmRlYnVnLi4uZG9uZS4NCmRvbmUuDQpSZWFkaW5nIHN5 bWJvbHMgZnJvbSAvbGliL3g4Nl82NC1saW51eC1nbnUvbGliei5zby4xLi4u UmVhZGluZyBzeW1ib2xzIGZyb20KL3Vzci9saWIvZGVidWcvLmJ1aWxkLWlk LzM0LzBiN2I0NjNmOTgxYjhhMGZiMzQ1MTc1MWY4ODFkZjFiMGMyZjc0LmRl YnVnLi4uZG9uZS4NCmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvbGli L3g4Nl82NC1saW51eC1nbnUvbGlibHptYS5zby41Li4uKG5vIGRlYnVnZ2lu ZwpzeW1ib2xzIGZvdW5kKS4uLmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJv bSAvbGliL3g4Nl82NC1saW51eC1nbnUvbGliYXVkaXQuc28uMS4uLihubyBk ZWJ1Z2dpbmcKc3ltYm9scyBmb3VuZCkuLi5kb25lLg0KUmVhZGluZyBzeW1i b2xzIGZyb20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJrcmI1LnNv LjMuLi4obm8gZGVidWdnaW5nCnN5bWJvbHMgZm91bmQpLi4uZG9uZS4NClJl YWRpbmcgc3ltYm9scyBmcm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUv bGliazVjcnlwdG8uc28uMy4uLihubwpkZWJ1Z2dpbmcgc3ltYm9scyBmb3Vu ZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL2xpYi94ODZfNjQt bGludXgtZ251L2xpYmNvbV9lcnIuc28uMi4uLlJlYWRpbmcgc3ltYm9scwpm cm9tIC91c3IvbGliL2RlYnVnLy9saWIveDg2XzY0LWxpbnV4LWdudS9saWJj b21fZXJyLnNvLjIuMS4uLmRvbmUuDQpkb25lLg0KUmVhZGluZyBzeW1ib2xz IGZyb20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJrcmI1c3VwcG9y dC5zby4wLi4uKG5vCmRlYnVnZ2luZyBzeW1ib2xzIGZvdW5kKS4uLmRvbmUu DQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvbGliL3g4Nl82NC1saW51eC1nbnUv bGlicHRocmVhZC5zby4wLi4uUmVhZGluZyBzeW1ib2xzCmZyb20KL3Vzci9s aWIvZGVidWcvLmJ1aWxkLWlkL2I3Lzc4NDdjYzljYWNiY2EzYjU3NTNkMGQy NWEzMmU1Nzk1YWZlNzViLmRlYnVnLi4uZG9uZS4NCmRvbmUuDQpbVGhyZWFk IGRlYnVnZ2luZyB1c2luZyBsaWJ0aHJlYWRfZGIgZW5hYmxlZF0NClVzaW5n IGhvc3QgbGlidGhyZWFkX2RiIGxpYnJhcnkgIi9saWIveDg2XzY0LWxpbnV4 LWdudS9saWJ0aHJlYWRfZGIuc28uMSIuDQpSZWFkaW5nIHN5bWJvbHMgZnJv bSAvbGliNjQvbGQtbGludXgteDg2LTY0LnNvLjIuLi5SZWFkaW5nIHN5bWJv bHMgZnJvbQovdXNyL2xpYi9kZWJ1Zy8vbGliL3g4Nl82NC1saW51eC1nbnUv bGQtMi4yMy5zby4uLmRvbmUuDQpkb25lLg0KUmVhZGluZyBzeW1ib2xzIGZy b20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJsYmVyLTIuNC5zby4y Li4uUmVhZGluZwpzeW1ib2xzIGZyb20KL3Vzci9saWIvZGVidWcvLmJ1aWxk LWlkLzZiLzlmNDA2MWExZDQ0ODEzYTU0ZGE0ZGJiMDA4OGY1MjlkOGQ3OGVh LmRlYnVnLi4uZG9uZS4NCmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAv bGliL3g4Nl82NC1saW51eC1nbnUvbGlicmVzb2x2LnNvLjIuLi5SZWFkaW5n IHN5bWJvbHMKZnJvbSAvdXNyL2xpYi9kZWJ1Zy8vbGliL3g4Nl82NC1saW51 eC1nbnUvbGlicmVzb2x2LTIuMjMuc28uLi5kb25lLg0KZG9uZS4NClJlYWRp bmcgc3ltYm9scyBmcm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvbGli c2FzbDIuc28uMi4uLihubyBkZWJ1Z2dpbmcKc3ltYm9scyBmb3VuZCkuLi5k b25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9saWIveDg2XzY0LWxp bnV4LWdudS9saWJnc3NhcGkuc28uMy4uLihubwpkZWJ1Z2dpbmcgc3ltYm9s cyBmb3VuZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9s aWIveDg2XzY0LWxpbnV4LWdudS9saWJnbnV0bHMuc28uMzAuLi4obm8KZGVi dWdnaW5nIHN5bWJvbHMgZm91bmQpLi4uZG9uZS4NClJlYWRpbmcgc3ltYm9s cyBmcm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvbGliaWN1ZGF0YS5z by41NS4uLihubwpkZWJ1Z2dpbmcgc3ltYm9scyBmb3VuZCkuLi5kb25lLg0K UmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdu dS9saWJzdGRjKysuc28uNi4uLihubwpkZWJ1Z2dpbmcgc3ltYm9scyBmb3Vu ZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL2xpYi94ODZfNjQt bGludXgtZ251L2xpYmdjY19zLnNvLjEuLi5SZWFkaW5nIHN5bWJvbHMKZnJv bSAvdXNyL2xpYi9kZWJ1Zy8vbGliL3g4Nl82NC1saW51eC1nbnUvbGliZ2Nj X3Muc28uMS4uLmRvbmUuDQpkb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20g L2xpYi94ODZfNjQtbGludXgtZ251L2xpYmtleXV0aWxzLnNvLjEuLi4obm8g ZGVidWdnaW5nCnN5bWJvbHMgZm91bmQpLi4uZG9uZS4NClJlYWRpbmcgc3lt Ym9scyBmcm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvbGliaGVpbW50 bG0uc28uMC4uLihubwpkZWJ1Z2dpbmcgc3ltYm9scyBmb3VuZCkuLi5kb25l Lg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9saWIveDg2XzY0LWxpbnV4 LWdudS9saWJrcmI1LnNvLjI2Li4uKG5vIGRlYnVnZ2luZwpzeW1ib2xzIGZv dW5kKS4uLmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvdXNyL2xpYi94 ODZfNjQtbGludXgtZ251L2xpYmFzbjEuc28uOC4uLihubyBkZWJ1Z2dpbmcK c3ltYm9scyBmb3VuZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20g L3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJoY3J5cHRvLnNvLjQuLi4o bm8KZGVidWdnaW5nIHN5bWJvbHMgZm91bmQpLi4uZG9uZS4NClJlYWRpbmcg c3ltYm9scyBmcm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvbGlicm9r ZW4uc28uMTguLi4obm8KZGVidWdnaW5nIHN5bWJvbHMgZm91bmQpLi4uZG9u ZS4NClJlYWRpbmcgc3ltYm9scyBmcm9tIC91c3IvbGliL3g4Nl82NC1saW51 eC1nbnUvbGlicDExLWtpdC5zby4wLi4uKG5vCmRlYnVnZ2luZyBzeW1ib2xz IGZvdW5kKS4uLmRvbmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvdXNyL2xp Yi94ODZfNjQtbGludXgtZ251L2xpYmlkbi5zby4xMS4uLihubyBkZWJ1Z2dp bmcKc3ltYm9scyBmb3VuZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZy b20gL3Vzci9saWIveDg2XzY0LWxpbnV4LWdudS9saWJ0YXNuMS5zby42Li4u KG5vIGRlYnVnZ2luZwpzeW1ib2xzIGZvdW5kKS4uLmRvbmUuDQpSZWFkaW5n IHN5bWJvbHMgZnJvbSAvdXNyL2xpYi94ODZfNjQtbGludXgtZ251L2xpYm5l dHRsZS5zby42Li4uKG5vCmRlYnVnZ2luZyBzeW1ib2xzIGZvdW5kKS4uLmRv bmUuDQpSZWFkaW5nIHN5bWJvbHMgZnJvbSAvdXNyL2xpYi94ODZfNjQtbGlu dXgtZ251L2xpYmhvZ3dlZWQuc28uNC4uLihubwpkZWJ1Z2dpbmcgc3ltYm9s cyBmb3VuZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9s aWIveDg2XzY0LWxpbnV4LWdudS9saWJnbXAuc28uMTAuLi4obm8gZGVidWdn aW5nCnN5bWJvbHMgZm91bmQpLi4uZG9uZS4NClJlYWRpbmcgc3ltYm9scyBm cm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvbGlid2luZC5zby4wLi4u KG5vIGRlYnVnZ2luZwpzeW1ib2xzIGZvdW5kKS4uLmRvbmUuDQpSZWFkaW5n IHN5bWJvbHMgZnJvbSAvdXNyL2xpYi94ODZfNjQtbGludXgtZ251L2xpYmhl aW1iYXNlLnNvLjEuLi4obm8KZGVidWdnaW5nIHN5bWJvbHMgZm91bmQpLi4u ZG9uZS4NClJlYWRpbmcgc3ltYm9scyBmcm9tIC91c3IvbGliL3g4Nl82NC1s aW51eC1nbnUvbGliaHg1MDkuc28uNS4uLihubyBkZWJ1Z2dpbmcKc3ltYm9s cyBmb3VuZCkuLi5kb25lLg0KUmVhZGluZyBzeW1ib2xzIGZyb20gL3Vzci9s aWIveDg2XzY0LWxpbnV4LWdudS9saWJzcWxpdGUzLnNvLjAuLi5SZWFkaW5n CnN5bWJvbHMgZnJvbQovdXNyL2xpYi9kZWJ1Zy8uYnVpbGQtaWQvZDkvNzgy YmEwMjNjYWVjMjZiMTVkODY3NmUzYTVkMDdiNTVlMTIxZWYuZGVidWcuLi5k b25lLg0KZG9uZS4NClJlYWRpbmcgc3ltYm9scyBmcm9tIC9saWIveDg2XzY0 LWxpbnV4LWdudS9saWJjcnlwdC5zby4xLi4uUmVhZGluZyBzeW1ib2xzCmZy b20gL3Vzci9saWIvZGVidWcvL2xpYi94ODZfNjQtbGludXgtZ251L2xpYmNy eXB0LTIuMjMuc28uLi5kb25lLg0KZG9uZS4NClJlYWRpbmcgc3ltYm9scyBm cm9tIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvbGliZmZpLnNvLjYuLi5S ZWFkaW5nIHN5bWJvbHMKZnJvbQovdXNyL2xpYi9kZWJ1Zy8uYnVpbGQtaWQv OWQvOWM5NThmMWY0ODk0YWZlZjZhZWNkOTBkMWM0MzBlYTI5YWMzNGYuZGVi dWcuLi5kb25lLg0KZG9uZS4NClJlYWRpbmcgc3ltYm9scyBmcm9tIC9saWIv eDg2XzY0LWxpbnV4LWdudS9saWJuc3NfZmlsZXMuc28uMi4uLlJlYWRpbmcK c3ltYm9scyBmcm9tCi91c3IvbGliL2RlYnVnLy9saWIveDg2XzY0LWxpbnV4 LWdudS9saWJuc3NfZmlsZXMtMi4yMy5zby4uLmRvbmUuDQpkb25lLg0KMHgw MDAwN2Y4MTk5MjVkZTcwIGluIF9fcG9sbF9ub2NhbmNlbCAoKSBhdAouLi9z eXNkZXBzL3VuaXgvc3lzY2FsbC10ZW1wbGF0ZS5TOjg0DQo4NCAgICAgIC4u L3N5c2RlcHMvdW5peC9zeXNjYWxsLXRlbXBsYXRlLlM6IE5vIHN1Y2ggZmls ZSBvciBkaXJlY3RvcnkuDQooZ2RiKSBzZXQgcGFnaW5hdGlvbiBvZmYNCihn ZGIpIHNldCBsb2dnaW5nIGZpbGUgL3RtcC9nZGIubG9nDQooZ2RiKSBzZXQg bG9nZ2luZyBvbg0KQ29weWluZyBvdXRwdXQgdG8gL3RtcC9nZGIubG9nDQoo Z2RiKSBoYW5kbGUgU0lHVVNSMSBub3N0b3ANClNpZ25hbCAgICAgICAgU3Rv cCAgICAgIFByaW50ICAgUGFzcyB0byBwcm9ncmFtIERlc2NyaXB0aW9uDQpT SUdVU1IxICAgICAgIE5vICAgICAgICBZZXMgICAgIFllcyAgICAgICAgICAg ICBVc2VyIGRlZmluZWQgc2lnbmFsIDENCihnZGIpIGhhbmRsZSBTSUdVU1Ix IG5vcHJpbnQNClNpZ25hbCAgICAgICAgU3RvcCAgICAgIFByaW50ICAgUGFz cyB0byBwcm9ncmFtIERlc2NyaXB0aW9uDQpTSUdVU1IxICAgICAgIE5vICAg ICAgICBObyAgICAgIFllcyAgICAgICAgICAgICBVc2VyIGRlZmluZWQgc2ln bmFsIDENCihnZGIpIGNvbnQNCkNvbnRpbnVpbmcuDQoNClByb2dyYW0gcmVj ZWl2ZWQgc2lnbmFsIFNJR1NFR1YsIFNlZ21lbnRhdGlvbiBmYXVsdC4NCl9i dF9yZXN0b3JlX3BhZ2UgKHBhZ2U9MHg3ZjgxNmZjZTJiNDAgIiIsIGZyb209 MHg1NWEwOTQ1YWJiNzAgIlwwMzYiLApsZW49PG9wdGltaXplZCBvdXQ+KSBh dAovYnVpbGQvcG9zdGdyZXNxbC05LjUteHA5dXRIL3Bvc3RncmVzcWwtOS41 LTkuNS4zL2J1aWxkLy4uL3NyYy9iYWNrZW5kL2FjY2Vzcy9uYnRyZWUvbmJ0 eGxvZy5jOjU3DQo1NyAgICAgCi9idWlsZC9wb3N0Z3Jlc3FsLTkuNS14cDl1 dEgvcG9zdGdyZXNxbC05LjUtOS41LjMvYnVpbGQvLi4vc3JjL2JhY2tlbmQv YWNjZXNzL25idHJlZS9uYnR4bG9nLmM6Ck5vIHN1Y2ggZmlsZSBvciBkaXJl Y3RvcnkuDQooZ2RiKSBidA0KIzAgIF9idF9yZXN0b3JlX3BhZ2UgKHBhZ2U9 MHg3ZjgxNmZjZTJiNDAgIiIsIGZyb209MHg1NWEwOTQ1YWJiNzAgIlwwMzYi LApsZW49PG9wdGltaXplZCBvdXQ+KSBhdAovYnVpbGQvcG9zdGdyZXNxbC05 LjUteHA5dXRIL3Bvc3RncmVzcWwtOS41LTkuNS4zL2J1aWxkLy4uL3NyYy9i YWNrZW5kL2FjY2Vzcy9uYnRyZWUvbmJ0eGxvZy5jOjU3DQojMSAgMHgwMDAw MDAwMDAwMDAwMDAwIGluID8/ICgpDQooZ2RiKSBwIGZyb20NCiQxID0gMHg1 NWEwOTQ1YWJiNzAgIlwwMzYiDQooZ2RiKSBwIGVuZA0KJDIgPSAweDU1YTA5 NDVhYzkyOCAiXDMwNWNPIg0KKGdkYikgcCBpDQokMyA9IDMzMjQNCihnZGIp IHAgbGVuDQokNCA9IDxvcHRpbWl6ZWQgb3V0Pg0KKGdkYikgcCAmaXR1cGRh dGENCiQ1ID0gKEluZGV4VHVwbGVEYXRhICopIDB4N2ZmZTgzZWE4NGUwDQoo Z2RiKSBwIGl0ZW1zDQokNiA9IHsweDAgPHJlcGVhdHMgNDA4IHRpbWVzPn0N CihnZGIpIHAgJml0ZW1zDQokNyA9IChJdGVtICgqKVs0MDhdKSAweDdmZmU4 M2VhODgyMA0KKGdkYikgcCBpdGVtc3oNCiQ4ID0gPG9wdGltaXplZCBvdXQ+ DQooZ2RiKSBydW4NClRoZSBwcm9ncmFtIGJlaW5nIGRlYnVnZ2VkIGhhcyBi ZWVuIHN0YXJ0ZWQgYWxyZWFkeS4NClN0YXJ0IGl0IGZyb20gdGhlIGJlZ2lu bmluZz8gKHkgb3IgbikgeQ0KU3RhcnRpbmcgcHJvZ3JhbTogL3Vzci9saWIv cG9zdGdyZXNxbC85LjUvYmluL3Bvc3RncmVzDQpbVGhyZWFkIGRlYnVnZ2lu ZyB1c2luZyBsaWJ0aHJlYWRfZGIgZW5hYmxlZF0NClVzaW5nIGhvc3QgbGli dGhyZWFkX2RiIGxpYnJhcnkgIi9saWIveDg2XzY0LWxpbnV4LWdudS9saWJ0 aHJlYWRfZGIuc28uMSIuDQoicm9vdCIgZXhlY3V0aW9uIG9mIHRoZSBQb3N0 Z3JlU1FMIHNlcnZlciBpcyBub3QgcGVybWl0dGVkLg0KVGhlIHNlcnZlciBt dXN0IGJlIHN0YXJ0ZWQgdW5kZXIgYW4gdW5wcml2aWxlZ2VkIHVzZXIgSUQg dG8gcHJldmVudA0KcG9zc2libGUgc3lzdGVtIHNlY3VyaXR5IGNvbXByb21p c2UuICBTZWUgdGhlIGRvY3VtZW50YXRpb24gZm9yDQptb3JlIGluZm9ybWF0 aW9uIG9uIGhvdyB0byBwcm9wZXJseSBzdGFydCB0aGUgc2VydmVyLg0KW0lu ZmVyaW9yIDEgKHByb2Nlc3MgMTg4NykgZXhpdGVkIHdpdGggY29kZSAwMV0N CihnZGIpIHF1aXQKCg==
Hi, thanks for reporting this issue. On 2016-06-07 09:16:18 +0000, boa@neogrid.dk wrote: > Program received signal SIGSEGV, Segmentation fault. > _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 "\036", > len=<optimized out>) at > /build/postgresql-9.5-xp9utH/postgresql-9.5-9.5.3/build/../src/backend/access/nbtree/nbtxlog.c:57 > 57 > /build/postgresql-9.5-xp9utH/postgresql-9.5-9.5.3/build/../src/backend/access/nbtree/nbtxlog.c: > No such file or directory. > (gdb) bt > #0 _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 "\036", > len=<optimized out>) at > /build/postgresql-9.5-xp9utH/postgresql-9.5-9.5.3/build/../src/backend/access/nbtree/nbtxlog.c:57 > #1 0x0000000000000000 in ?? () > (gdb) p from > $1 = 0x55a0945abb70 "\036" > (gdb) p end > $2 = 0x55a0945ac928 "\305cO" > (gdb) p i > $3 = 3324 > (gdb) p len > $4 = <optimized out> > (gdb) p &itupdata > $5 = (IndexTupleData *) 0x7ffe83ea84e0 > (gdb) p items > $6 = {0x0 <repeats 408 times>} > (gdb) p &items > $7 = (Item (*)[408]) 0x7ffe83ea8820 > (gdb) p itemsz > $8 = <optimized out> Uhm, this is a bit odd. There's no backtrace, but types and such are known? I guess you do have the debug symbols installed? Greetings, Andres Freund
> On 2016-06-07 19:21, Andres Freund wrote: > > Program received signal SIGSEGV, Segmentation fault. > > _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 "\036", > > len=<optimized out>) at > > /build/postgresql-9.5-xp9utH/postgresql-9.5- > 9.5.3/build/../src/backend/access/nbtree/nbtxlog.c:57 > > 57 > > /build/postgresql-9.5-xp9utH/postgresql-9.5- > 9.5.3/build/../src/backend/access/nbtree/nbtxlog.c: > > No such file or directory. > > (gdb) bt > > #0 _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 > > "\036", len=<optimized out>) at > > /build/postgresql-9.5-xp9utH/postgresql-9.5-9.5.3/build/../src/backend > > /access/nbtree/nbtxlog.c:57 > > #1 0x0000000000000000 in ?? () > > (gdb) p from > > $1 = 0x55a0945abb70 "\036" > > (gdb) p end > > $2 = 0x55a0945ac928 "\305cO" > > (gdb) p i > > $3 = 3324 > > (gdb) p len > > $4 = <optimized out> > > (gdb) p &itupdata > > $5 = (IndexTupleData *) 0x7ffe83ea84e0 > > (gdb) p items > > $6 = {0x0 <repeats 408 times>} > > (gdb) p &items > > $7 = (Item (*)[408]) 0x7ffe83ea8820 > > (gdb) p itemsz > > $8 = <optimized out> > > Uhm, this is a bit odd. There's no backtrace, but types and such are known? I > guess you do have the debug symbols installed? Yeah, it's confusing. I installed the result of ./list-dbgsym-packages-v2.1.sh -p $pid which gave: libcomerr2-dbg libffi6-dbg libgcc1-dbg libicu55-dbg libldap-2.4-2-dbg libsqlite3-0-dbg libssl1.0.0-dbg libxml2-dbg postgresql-9.5-dbgzlib1g-dbg Not sure what else I can do short of recompiling postgresql mysql. Greetings, Bo Ørsted Andresen
On 2016-06-07 17:36:38 +0000, Bo Ørsted Andresen wrote: > > On 2016-06-07 19:21, Andres Freund wrote: > > > Program received signal SIGSEGV, Segmentation fault. > > > _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 "\036", > > > len=<optimized out>) at > > > /build/postgresql-9.5-xp9utH/postgresql-9.5- > > 9.5.3/build/../src/backend/access/nbtree/nbtxlog.c:57 > > > 57 > > > /build/postgresql-9.5-xp9utH/postgresql-9.5- > > 9.5.3/build/../src/backend/access/nbtree/nbtxlog.c: > > > No such file or directory. > > > (gdb) bt > > > #0 _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 > > > "\036", len=<optimized out>) at > > > /build/postgresql-9.5-xp9utH/postgresql-9.5-9.5.3/build/../src/backend > > > /access/nbtree/nbtxlog.c:57 > > > #1 0x0000000000000000 in ?? () > > > (gdb) p from > > > $1 = 0x55a0945abb70 "\036" > > > (gdb) p end > > > $2 = 0x55a0945ac928 "\305cO" > > > (gdb) p i > > > $3 = 3324 > > > (gdb) p len > > > $4 = <optimized out> > > > (gdb) p &itupdata > > > $5 = (IndexTupleData *) 0x7ffe83ea84e0 > > > (gdb) p items > > > $6 = {0x0 <repeats 408 times>} > > > (gdb) p &items > > > $7 = (Item (*)[408]) 0x7ffe83ea8820 > > > (gdb) p itemsz > > > $8 = <optimized out> > > > > Uhm, this is a bit odd. There's no backtrace, but types and such are known? I > > guess you do have the debug symbols installed? > > Yeah, it's confusing. I installed the result of ./list-dbgsym-packages-v2.1.sh -p $pid which gave: > > libcomerr2-dbg libffi6-dbg libgcc1-dbg libicu55-dbg libldap-2.4-2-dbg libsqlite3-0-dbg libssl1.0.0-dbg libxml2-dbg postgresql-9.5-dbgzlib1g-dbg > > Not sure what else I can do short of recompiling postgresql mysql. Any chance the running version of postgres is out of date with the installed binaries / debug symbols?
> On 2016-06-07 19:41, Andres Freund wrote: > > > > Program received signal SIGSEGV, Segmentation fault. > > > > _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 > > > > "\036", len=<optimized out>) at > > > > /build/postgresql-9.5-xp9utH/postgresql-9.5- > > > 9.5.3/build/../src/backend/access/nbtree/nbtxlog.c:57 > > > > 57 > > > > /build/postgresql-9.5-xp9utH/postgresql-9.5- > > > 9.5.3/build/../src/backend/access/nbtree/nbtxlog.c: > > > > No such file or directory. > > > > (gdb) bt > > > > #0 _bt_restore_page (page=0x7f816fce2b40 "", from=0x55a0945abb70 > > > > "\036", len=<optimized out>) at > > > > /build/postgresql-9.5-xp9utH/postgresql-9.5-9.5.3/build/../src/bac > > > > kend > > > > /access/nbtree/nbtxlog.c:57 > > > > #1 0x0000000000000000 in ?? () > > > > (gdb) p from > > > > $1 = 0x55a0945abb70 "\036" > > > > (gdb) p end > > > > $2 = 0x55a0945ac928 "\305cO" > > > > (gdb) p i > > > > $3 = 3324 > > > > (gdb) p len > > > > $4 = <optimized out> > > > > (gdb) p &itupdata > > > > $5 = (IndexTupleData *) 0x7ffe83ea84e0 > > > > (gdb) p items > > > > $6 = {0x0 <repeats 408 times>} > > > > (gdb) p &items > > > > $7 = (Item (*)[408]) 0x7ffe83ea8820 > > > > (gdb) p itemsz > > > > $8 = <optimized out> > > > > > > Uhm, this is a bit odd. There's no backtrace, but types and such are > > > known? I guess you do have the debug symbols installed? > > > > Yeah, it's confusing. I installed the result of ./list-dbgsym-packages-v2.1.sh - > p $pid which gave: > > > > libcomerr2-dbg libffi6-dbg libgcc1-dbg libicu55-dbg libldap-2.4-2-dbg > > libsqlite3-0-dbg libssl1.0.0-dbg libxml2-dbg postgresql-9.5-dbg > > zlib1g-dbg > > > > Not sure what else I can do short of recompiling postgresql mysql. > > Any chance the running version of postgres is out of date with the installed > binaries / debug symbols? You mean that I upgraded without restarting postgres before the segfault? Before this I have had the same problem with Ubuntu 14.04 with postgresql 9.5.3 from a ppa. I then tried building a new VMbased on Ubuntu 16.04 on the 3rd of June to see if there was something wrong with the slave VM. So it's a clean installwhere I installed latest postgresql once and ran it. I've reproduced the problem three times since that happened withone reboot in between. Of course what hasn't changed during all of this is the master. The version of postgresql-client-9.5, libpq5, postgreql-9.5, postgresql-contrib-9.5 and postgresql-9.5-dbg are all at 9.5.3-0ubuntu0.16.04. Regards, Bo Ørsted Andresen
On 2016-06-07 17:53:46 +0000, Bo Ørsted Andresen wrote: > > On 2016-06-07 19:41, Andres Freund wrote: > > > Not sure what else I can do short of recompiling postgresql mysql. > > > > Any chance the running version of postgres is out of date with the installed > > binaries / debug symbols? > > You mean that I upgraded without restarting postgres before the segfault? Yes, that's what I was wondering. But alas, that's aparently not the reason. This is going to be a bit more complicated, sorry :( Could you try to reproduce the problem, and do 'p/x ReadRecPtr'? That should give you something like 0x5434343496. If you rewrite this as first-four-bytes/last-four-bytes e.g. 54/34343496 you get the LSN. With that, could you try pg_xlogdump -p /path/to/data/directory -s 54/34343496 -n 100 and send the output? Regards, Andres
> On 2016-06-07 20:00, Andres Freund wrote: > > > On 2016-06-07 19:41, Andres Freund wrote: > > > > Not sure what else I can do short of recompiling postgresql mysql. > > > > > > Any chance the running version of postgres is out of date with the > > > installed binaries / debug symbols? > > > > You mean that I upgraded without restarting postgres before the segfault? > > Yes, that's what I was wondering. But alas, that's aparently not the reason. > > This is going to be a bit more complicated, sorry :( > > Could you try to reproduce the problem, and do 'p/x ReadRecPtr'? That > should give you something like 0x5434343496. If you rewrite this as first-four- > bytes/last-four-bytes e.g. 54/34343496 you get the LSN. With that, could you > try pg_xlogdump -p /path/to/data/directory -s 54/34343496 -n 100 and send > the output? Will do. May take a while. Regards, Bo Ørsted Andresen
On 2016-06-07 18:04:50 +0000, Bo Ørsted Andresen wrote: > > On 2016-06-07 20:00, Andres Freund wrote: > > > > On 2016-06-07 19:41, Andres Freund wrote: > > > > > Not sure what else I can do short of recompiling postgresql mysql. > > > > > > > > Any chance the running version of postgres is out of date with the > > > > installed binaries / debug symbols? > > > > > > You mean that I upgraded without restarting postgres before the segfault? > > > > Yes, that's what I was wondering. But alas, that's aparently not the reason. > > > > This is going to be a bit more complicated, sorry :( > > > > Could you try to reproduce the problem, and do 'p/x ReadRecPtr'? That > > should give you something like 0x5434343496. If you rewrite this as first-four- > > bytes/last-four-bytes e.g. 54/34343496 you get the LSN. With that, could you > > try pg_xlogdump -p /path/to/data/directory -s 54/34343496 -n 100 and send > > the output? > > Will do. May take a while. To clarify: After the crash recovery continues successfully? Or do you have to scrap te standby? Regards, Andres
Bo Ørsted Andresen <boa@neogrid.dk> writes: >> On 2016-06-07 19:41, Andres Freund wrote: >> Any chance the running version of postgres is out of date with the installed >> binaries / debug symbols? > You mean that I upgraded without restarting postgres before the segfault? I think the reason for the lack of useful backtrace info is that we've smashed the stack. Note that the original report shows i == 3324 which is much larger than the available length of the local items[] array (408). So presumably, the passed-in "len" was bogus (much too large). If you're prepared to build a custom version of Postgres, you could try adding this to _bt_restore_page(): /* Need to copy tuple header due to alignment considerations */ memcpy(&itupdata, from, sizeof(IndexTupleData)); itemsz = IndexTupleDSize(itupdata); itemsz = MAXALIGN(itemsz); + if (i >= lengthof(items)) + elog(PANIC, "too many items on btree page"); + items[i] = (Item) from; itemsizes[i] = itemsz; i++; from += itemsz; and then you should get a core dump before the stack is clobbered. I wonder whether we shouldn't add such a check to the regular sources... regards, tom lane
> On 2016-06-07 20:07, Andres Freund wrote: > > > > > > Not sure what else I can do short of recompiling postgresql mysql. > > > > > > > > > > Any chance the running version of postgres is out of date with > > > > > the installed binaries / debug symbols? > > > > > > > > You mean that I upgraded without restarting postgres before the > segfault? > > > > > > Yes, that's what I was wondering. But alas, that's aparently not the > reason. > > > > > > This is going to be a bit more complicated, sorry :( > > > > > > Could you try to reproduce the problem, and do 'p/x ReadRecPtr'? > > > That should give you something like 0x5434343496. If you rewrite > > > this as first-four- bytes/last-four-bytes e.g. 54/34343496 you get > > > the LSN. With that, could you try pg_xlogdump -p > > > /path/to/data/directory -s 54/34343496 -n 100 and send the output? > > > > Will do. May take a while. > > To clarify: After the crash recovery continues successfully? Or do you have to > scrap te standby? The crash recovery does not continue successfully. I don't know of a way to attach in gdb to the process that crashes beforeit already crashed, which does not involve scrapping the standby. Regards, Bo Ørsted Andresen
On 2016-06-07 18:15:14 +0000, Bo Ørsted Andresen wrote: > The crash recovery does not continue successfully. I don't know of a way to attach in gdb to the process that crashes beforeit already crashed, which does not involve scrapping the standby. gdb --args /path/to/postgres --single postgres -D /path/to/datadir (gdb) run should probably do the trick. Regards, Andres
> Fra: [mailto:andres@anarazel.de] > Sendt: 07 June 2016 20:17 > Til: Bo Ørsted Andresen <boa@neogrid.dk> > Cc: pgsql-bugs@postgresql.org > Emne: Re: SV: [BUGS] BUG #14180: Segmentation fault on replication slave > > On 2016-06-07 20:17, Andres Freund wrote: > > The crash recovery does not continue successfully. I don't know of a way to > attach in gdb to the process that crashes before it already crashed, which > does not involve scrapping the standby. > > gdb --args /path/to/postgres --single postgres -D /path/to/datadir > (gdb) run > > should probably do the trick. Will try that next time, thanks. Unfornately it's gone. :( Regards, Bo Ørsted Andresen
> On 2016-06-07 20:08, Tom Lane wrote: > I think the reason for the lack of useful backtrace info is that we've smashed > the stack. Note that the original report shows i == 3324 which is much larger > than the available length of the local items[] array (408). > So presumably, the passed-in "len" was bogus (much too large). > > If you're prepared to build a custom version of Postgres, you could try adding > this to _bt_restore_page(): > > /* Need to copy tuple header due to alignment > considerations */ > memcpy(&itupdata, from, sizeof(IndexTupleData)); > itemsz = IndexTupleDSize(itupdata); > itemsz = MAXALIGN(itemsz); > > + if (i >= lengthof(items)) > + elog(PANIC, "too many items on btree page"); > + > items[i] = (Item) from; > itemsizes[i] = itemsz; > i++; > > from += itemsz; > > and then you should get a core dump before the stack is clobbered. > > I wonder whether we shouldn't add such a check to the regular sources... Will give it a shot tomorrow. Regards, Bo Ørsted Andresen
> On 2016-06-07 20:00, Andres Freund wrote: > > > > Not sure what else I can do short of recompiling postgresql mysql. > > > > > > Any chance the running version of postgres is out of date with the > > > installed binaries / debug symbols? > > > > You mean that I upgraded without restarting postgres before the segfault? > > Yes, that's what I was wondering. But alas, that's aparently not the reason. > > This is going to be a bit more complicated, sorry :( > > Could you try to reproduce the problem, and do 'p/x ReadRecPtr'? That > should give you something like 0x5434343496. If you rewrite this as first-four- > bytes/last-four-bytes e.g. 54/34343496 you get the LSN. With that, could you > try pg_xlogdump -p /path/to/data/directory -s 54/34343496 -n 100 and send > the output? Output attached. Thanks, Bo Ørsted Andresen
Attachment
> > On 2016-06-07 20:08, Tom Lane wrote: > > I think the reason for the lack of useful backtrace info is that we've > > smashed the stack. Note that the original report shows i == 3324 > > which is much larger than the available length of the local items[] array > (408). > > So presumably, the passed-in "len" was bogus (much too large). > > > > If you're prepared to build a custom version of Postgres, you could > > try adding this to _bt_restore_page(): > > > > /* Need to copy tuple header due to alignment > considerations */ > > memcpy(&itupdata, from, sizeof(IndexTupleData)); > > itemsz = IndexTupleDSize(itupdata); > > itemsz = MAXALIGN(itemsz); > > > > + if (i >= lengthof(items)) > > + elog(PANIC, "too many items on btree page"); > > + > > items[i] = (Item) from; > > itemsizes[i] = itemsz; > > i++; > > > > from += itemsz; > > > > and then you should get a core dump before the stack is clobbered. > > > > I wonder whether we shouldn't add such a check to the regular sources... Logged: LOG: started streaming WAL from primary at 631/7000000 on timeline 1 PANIC: too many items on btree page CONTEXT: xlog redo Btree/SPLIT_R: level 0, firstright 139 Bacttrace: # gdb -p 10069 GNU gdb (Ubuntu 7.11-0ubuntu1) 7.11 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word". Attaching to process 10069 Reading symbols from /usr/local/pgsql/bin/postgres...done. Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/librt-2.23.so...done. done. Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.23.so...done. done. Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libm-2.23.so...done. done. Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.23.so...done. done. Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols from /usr/lib/debug/.build-id/b7/7847cc9cacbca3b5753d0d25a32e5795afe75b.debug...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/ld-2.23.so...done. done. Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libnss_files-2.23.so...done. done. 0x00007ffff73f3e70 in __poll_nocancel () at ../sysdeps/unix/syscall-template.S:84 84 ../sysdeps/unix/syscall-template.S: No such file or directory. (gdb) set pagination off (gdb) set logging file /tmp/debuglog-20160608-2.txt (gdb) set logging on Copying output to /tmp/debuglog-20160608-2.txt. (gdb) handle SIGUSR1 nostop Signal Stop Print Pass to program Description SIGUSR1 No Yes Yes User defined signal 1 (gdb) handle SIGUSR1 noprint Signal Stop Print Pass to program Description SIGUSR1 No No Yes User defined signal 1 (gdb) cont Continuing. (gdb) Program received signal SIGABRT, Aborted. 0x00007ffff732e418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x00007ffff732e418 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x00007ffff733001a in __GI_abort () at abort.c:89 #2 0x000000000078ccaa in errfinish (dummy=dummy@entry=0) at elog.c:551 #3 0x000000000079074a in elog_finish (elevel=elevel@entry=22, fmt=fmt@entry=0x7cb187 "too many items on btree page") atelog.c:1368 #4 0x00000000004ae437 in _bt_restore_page (page=page@entry=0x7fffefa2cb40 "", from=<optimized out>, from@entry=0xc52e70"\036", len=<optimized out>) at nbtxlog.c:58 #5 0x00000000004ae8a4 in btree_xlog_split (onleft=onleft@entry=0 '\000', isroot=isroot@entry=0 '\000', record=record@entry=0xc3b840)at nbtxlog.c:241 #6 0x00000000004aee1c in btree_redo (record=0xc3b840) at nbtxlog.c:984 #7 0x00000000004d5c2b in StartupXLOG () at xlog.c:6825 #8 0x000000000064e212 in StartupProcessMain () at startup.c:215 #9 0x00000000004e3168 in AuxiliaryProcessMain (argc=argc@entry=2, argv=argv@entry=0x7fffffffe3e0) at bootstrap.c:418 #10 0x000000000064b698 in StartChildProcess (type=StartupProcess) at postmaster.c:5199 #11 0x000000000064dc84 in PostmasterMain (argc=argc@entry=3, argv=argv@entry=0xc1b9f0) at postmaster.c:1284 #12 0x0000000000467950 in main (argc=3, argv=0xc1b9f0) at main.c:228 Regards, Bo Ørsted Andresen