Re: Set visibility map bit after HOT prune - Mailing list pgsql-hackers

On Thu, Dec 20, 2012 at 6:12 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Wed, Dec 19, 2012 at 12:26 PM, Pavan Deolasee
> <pavan.deolasee@gmail.com> wrote:
>> This can be handled by breaking 1-to-1 mapping on VM bit and
>> PD_ALL_VISIBLE bit. So seq scans will only look at PD_ALL_VISIBLE. It
>> was proposed by Andres up thread, but shot down by Tom and Simon. But
>> I still feel that was over reaction and there is a lot of merit in the
>> idea. As I said elsewhere, it will also help the case when there are
>> DEAD line pointers in a page. Today we can't mark such pages
>> all-visible, but if we break this mapping, we can do that.
>
> Sure, but you're zipping rather blithely past the disadvantages of
> such an approach.

Hmm. You're right. I did not think about the disadvantages and now
that you mention them, I feel they are important.

>  Jeff Davis recently proposed getting rid of
> PD_ALL_VISIBLE, and Tom and I both expressed considerable skepticism
> about that; this proposal has the same problems.  One of the major
> benefits of PD_ALL_VISIBLE is that, when it isn't set, inserts,
> updates, and deletes to the page can ignore the visibility map.  That
> means that a server under heavy concurrency is much less likely to
> encounter contention on the visibility map blocks.  Now, maybe that's
> not really a problem, but I sure haven't seen enough evidence to make
> me believe it.  If it's really true that PD_ALL_VISIBLE needn't fill
> this role, then Heikki wasted an awful lot of time implementing it,
> and I wasted an awful lot of time keeping it working when I made the
> visibility map crash-safe for IOS.  That could be true, but I tend to
> think it isn't.
>

Yeah, VM buffer contention can become prominent if we break the
invariant that page level bit status implies the vm bit status, at
least when its clear.OTOH IMHO we need some mechanism to address the
issue of aggressive clearing of the VM bits, but a very lame
corresponding set operation. Today we don't have much contention on
the VM page, but we must be sacrificing its usability in return. IOS
as well as vacuum optimizations using VMs will turn out not so useful
for many workloads. I'm very reluctant to suggest that we can solve
this my setting aside another page-level bit to track visibility of
tuples for heapscans. Or even have a bit in the tuple header itself to
track this information at that level to avoid repeated visibility
check for a tuple which is known to be visible to all current and
future transactions.

>> I would like to run some pgbench tests where we get the system in a
>> steady state such as all/most updates are HOT updates (not entirely
>> unlikely scenario for many real life cases). And then try running some
>> concurrent queries which can be executed via IOS. My gut feel is that,
>> today we will see slow and continuous drop in performance for these
>> queries because IOS will slowly stop working.
>
> If there are no vacuums, I agree.
>

And we expect vacuums to be very less or none. AFAIR in pgbench, it
now takes hours for accounts table to get chosen for vacuum and we
should be happy about it. But IOS are almost impossible for pgbench
kind of workloads today because of our aggressive strategy to clear
the VM bits.

Thanks,
Pavan




-- 
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee