On Sun, Jul 19, 2015 at 9:53 AM, Andres Freund <andres@anarazel.de> wrote:
> Perhaps this could be moved into a virtual machine hosted by the
> postgres project infrastructure?
I would definitely be willing to start an external project to provide
additional coverage tests if I thought someone was inclined to run
them in this kind of environment. There *is* a place for scattergun
tests that just exercise rarely hit codepaths (or even frequently hit
codepaths that are not tested by the regression tests currently, often
due to requiring non-trivial amounts of data). Running the tests with
tools like Valgrind will catch a certain class of bug, like the one
reported and fixed on this thread.
The advantage of an external project is that being a committer isn't
tied to being a Postgres committer, and that it allows us to have
broader requirements to run all the tests. For example, it would be
acceptable to write some tests in Perl, and others in Python,
according to individual preference. Or, individual scripts could have
requirements that are only practical to satisfy with a
language-specific package manager like Python's pip, which would never
fly with the standard regression tests. I can imagine that kind of
thing being useful for generating a test-case involving very large
hash joins or sorts, where a procedural generation of the tests is
easier than having hundreds of megabytes of dummy data stored in git.
Weren't early versions of isolationtester written in Python + Twisted?
Noah suggested that Valgrind be extended to detect when a buffer was
read from despite not being pinned. This seems like a good idea to me,
but unless and until we exercise the code paths where this might be
happening, which tend to involve complex issues with concurrency, then
we won't get the full benefit of automated detection. I recall
noticing such a bug when reviewing the B-Tree page split patch that
went into 9.4. But the relevant code path isn't exercised at all, and
so even if Valgrind could have detected it, it probably wouldn't have
(the P_INCOMPLETE_SPLIT() path within _bt_moveright() has no
coverage).
I think that a lot of us are doing stuff like this privately already
to some extent, but we don't know about it because it's never
submitted because the requirements are too high.
--
Peter Geoghegan
