Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I think Simon's point is that showing a gain on specific test
> cases isn't a sufficient argument.
Ah, if that's what he's been trying to get at, I'm curious who
disagrees with that. I wouldn't have thought anyone on this list
would.
> What we need to know about this sort of change is what is the
> distributed overhead that is going to be paid by *everybody*,
> whether their queries benefit from the optimization or not.
Certainly we need to test whether Heikki is right in the previously
non-quoted part of his post on this thread:
>> Note that we already have the visibility map, and the accesses
>> needed to update it are already there. Granted, we'll have to
>> change the logic slightly to make it crash safe, but I don't
>> expect that to add any meaningful overhead - the changes are
>> going to be where the bits are set, ie. vacuum, not when the bits
>> are cleared. Granted, we might also want to set the bits more
>> aggressively once they're used by index-only-scans. But done
>> correctly, just taking advantage of the VM that's already there
>> shouldn't add overhead to other operations.
> Isolated test cases (undoubtedly chosen to show off the
> optimization) are not adequate to form a picture of the overall
> cost and benefit.
Well, first, that hardly seems fair. I have many times seen people
make an effort to synthesize *worst* case benchmarks. Certainly any
regular on this list would know it is pointless to show only a best
case benchmark.
Second, we really need to make development of a performance testing
farm a priority at PGCon next week. The need for it just keeps
coming up over and over.
Third, Dan Ports has been working a great deal with DBT-2 running
PostgreSQL for the SSI patch, both as a stress tool to flush out
bugs and to get benchmarks numbers conforming to the published
requirements of that benchmark. I know from off-list emails that it
took a fair amount of work to get it running correctly with
PostgreSQL in his environment. We should probably try to draw on
that experience. (Of course that shouldn't be the *only* test in a
performance testing farm, but it's a good one to include.)
-Kevin
