Re: MMAP Buffers - Mailing list pgsql-hackers

Robert Haas <robertmhaas@gmail.com> Monday 18 April 2011 03:06:17
> On Sun, Apr 17, 2011 at 5:32 PM, Radosław Smogura
>
> <rsmogura@softperience.eu> wrote:
> > Each process has simple "mirror" of shared descriptors.
> >
> > I "believe" that modifications to buffer content may be only done when
> > holding exclusive lock (with some simple exceptions) (+ MVCC), actually
> > I saw only two things that can change already loaded data and cause
> > damage, you have described (setting hint bits during scan, and vacuum -
> > 1st may only cause, I think, that two processes will ask for same
> > transaction statuses <except vacuum>, 2nd one is impossible as vacumm
> > requires exclusive pin). When buffer tag is changed the version of
> > buffer is bumped up, and checked against local version - this about
> > reading buffer.
>
> Yes, an exclusive lock is required for substantive content changes.
> But if vacuum cleaning up the buffer is an issue for your patch, then
> it's probably also a problem if someone grabs an exclusive content
> lock and deletes the tuple (by setting XMAX) and some other backend
> later sees the old buffer contents after having in the meanwhile taken
> a new snapshot; or if likewise someone grabs an exclusive-lock, adds a
> tuple, and then your backend takes a new snapshot and then sees the
> old buffer contents.  Basically, any time someone grabs an
> exclusive-lock and releases it, it's necessary for all observers to
> see the updated contents by the time the exclusive lock is released.
> > In other cases after obtaining lock check is done if buffer has
> > associated updatable buffer and if local "mirror" has it too, then swap
> > should take place.
>
> I think this check would have to be done every time someone
> share-locks the buffer, which seems rather expensive.
I don't treat as issues, but it's disadvantage.

> > Logic about updatable buffers is similar to "shared buffers", each
> > updatable buffer has pin count, and updatable buffer can't be free if
> > someone uses it, but in contrast to "normal buffers", updatable buffers
> > doesn't have any support for locking etc. Updatable buffers exists only
> > on free list, or when associated with buffer.
>
> I don't see how you're going to get away with removing buffer locks.
> They exist for a reason, and adding mmap() to the mix is going to
> require MORE locking, not less.
>
> > In future, I will change version to shared segment id, something like
> > relation's oid + block, but ids will have continuous numbering 1,2,3...,
> > so I will be able to bypass smgr/md during read, and tag version check -
> > this looks like faster solution.
>
> I don't understand this part at all.
Versioning is witch approach where I thought about really often changes of
mmaped areas, I allocated part of segments, but now the segment is mmaped with
reservation, to it's full possible size, addresses of segments can't change
(problem is only with segment deletion).

Regards,
Radek