I tried running the application using the race detector but no data races were detected.
To further narrow this down I removed the call to the function in question from the pool
and instead run it one by one from a queue. This seems to have helped and the query didn't lock up anymore.
I took this one step further and left the locked up queries sitting there for a day and they eventually finished. Who would have thought a ~2 second query could blow up to ~20hrs.
On Thu, 2016-11-24 at 22:23 +0100, azhwkd wrote: > It should not be possible because a group does not return to the > update pool before the update hasn't finished. > I watched the queries in a postgres client and there was no overlap I > could see. > I don't really know what to make from this behavior, sometimes when I > start the application a few updates go through and eventually it will > lock up completely and sometimes it locks up immediately - always > with > heap_hot_search_buffer using ~20 of all CPU time on the system. > > Hello Sebastian, You stated that the application is written using go (www.golang.org). Are you able to run the application with the -race flag?
HTH, Rob
From wikipedia:-
Lack of race condition safety
There are no restrictions on how goroutines access shared data, making race conditions possible. Specifically, unless a program explicitly synchronizes via channels or other means, writes from one goroutine might be partly, entirely, or not at all visible to another, often with no guarantees about ordering of writes. Furthermore, Go's internal data structures like interface values, slice headers, hash tables, and string headers are not immune to race conditions, so type and memory safety can be violated in multithreaded programs that modify shared instances of those types without synchronization.
Instead of language support, safe concurrent programming thus relies on conventions; for example, Chisnall recommends an idiom called "aliases xor mutable", meaning that passing a mutable value (or pointer) over a channel signals a transfer of ownership over the value to its receiver.
