Thread: [w32] test_shm_mq test suite permanently burns connections slots
On a Windows or other EXEC_BACKEND build, the following eventually gets failures because all, or all but one, max_connections slot is consumed: for run in `seq 1 100`; do make -C contrib/test_shm_mq installcheck; done When I use max_connections=40, it fails on the sixth iteration. Only the six basic processes are actually running at that time. Thanks, nm -- Noah Misch EnterpriseDB http://www.enterprisedb.com
On Fri, Jul 25, 2014 at 3:25 PM, Noah Misch <noah@leadboat.com> wrote: > On a Windows or other EXEC_BACKEND build, the following eventually gets > failures because all, or all but one, max_connections slot is consumed: > > for run in `seq 1 100`; do make -C contrib/test_shm_mq installcheck; done > > When I use max_connections=40, it fails on the sixth iteration. Only the six > basic processes are actually running at that time. The tests start 7 workers each time, so that makes sense: 7 * 5 < 40 but 7 * 6 > 40. What I'm not sure is why they are leaking connection slots, and why they're only doing it in EXEC_BACKEND mode. A quick code audit didn't uncover any obvious explanation, so I'll try to reproduce and debug. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Mon, Jul 28, 2014 at 3:59 PM, Robert Haas <robertmhaas@gmail.com> wrote: > On Fri, Jul 25, 2014 at 3:25 PM, Noah Misch <noah@leadboat.com> wrote: >> On a Windows or other EXEC_BACKEND build, the following eventually gets >> failures because all, or all but one, max_connections slot is consumed: >> >> for run in `seq 1 100`; do make -C contrib/test_shm_mq installcheck; done >> >> When I use max_connections=40, it fails on the sixth iteration. Only the six >> basic processes are actually running at that time. > > The tests start 7 workers each time, so that makes sense: 7 * 5 < 40 > but 7 * 6 > 40. What I'm not sure is why they are leaking connection > slots, and why they're only doing it in EXEC_BACKEND mode. A quick > code audit didn't uncover any obvious explanation, so I'll try to > reproduce and debug. OK, I think I see the problem. In EXEC_BACKEND mode, SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has been set. InitProcess() therefore pulls the PGPROC for the worker from freeProcs rather than bgworkerFreeProcs. By exit time, IsBackgroundWorker has been set, so the PGPROC gets put back on the bgworkerFreeProcs list. Eventually there are no regular PGPROCs left; they've all been moved to the bgworkerFreeProcs list. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Robert Haas wrote: > OK, I think I see the problem. In EXEC_BACKEND mode, > SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has > been set. InitProcess() therefore pulls the PGPROC for the worker > from freeProcs rather than bgworkerFreeProcs. By exit time, > IsBackgroundWorker has been set, so the PGPROC gets put back on the > bgworkerFreeProcs list. Eventually there are no regular PGPROCs left; > they've all been moved to the bgworkerFreeProcs list. Doh. I'm surprised -- I tested a worker that crashed over and over to ensure PGPROCs were reused sanely. I guess I forgot to run it under EXEC_BACKEND. Are you fixing it? -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Jul 28, 2014 at 9:38 PM, Alvaro Herrera <alvherre@2ndquadrant.com> wrote: > Robert Haas wrote: >> OK, I think I see the problem. In EXEC_BACKEND mode, >> SubPostmasterMain() calls InitProcess() before IsBackgroundWorker has >> been set. InitProcess() therefore pulls the PGPROC for the worker >> from freeProcs rather than bgworkerFreeProcs. By exit time, >> IsBackgroundWorker has been set, so the PGPROC gets put back on the >> bgworkerFreeProcs list. Eventually there are no regular PGPROCs left; >> they've all been moved to the bgworkerFreeProcs list. > > Doh. I'm surprised -- I tested a worker that crashed over and over to > ensure PGPROCs were reused sanely. I guess I forgot to run it under > EXEC_BACKEND. > > Are you fixing it? Working on it now. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company