Maybe we could check PD_ALL_VISIBLE before taking the buffer lock - if it appears to be set, then we pin the visibility map page before taking the buffer lock. Otherwise, we take the buffer lock at once. Either way, once we have the lock, we recheck the bit. Only if it's set and we haven't got a pin do we need to do the drop-lock-pin-reacquire-lock dance. Is that at all sensible?
But only lock can make sure the page has enough free space. If we try the drop-lock-...-lock dance, we may fall into a dead loop.