Re: Adding probes for smgr - Mailing list pgsql-hackers

(2012/07/29 12:14), Tom Lane wrote:
> Peter Geoghegan <peter@2ndquadrant.com> writes:
>> On 28 July 2012 17:15, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> IMV smgr is pretty vestigial.  I wouldn't recommend loading more
>>> functionality onto that layer, because it's as likely as not that
>>> we'll just get rid of it someday.
> 
>> Agreed. I recently found myself reading a paper written by Stonebraker
>> back in the Berkeley days:
>> http://dislab2.hufs.ac.kr/dislab/seminar/2007/ERL-M87-06.pdf
>> This paper appears to have been published in about 1988, and it shows.
>> It's fairly obvious from reading the opening paragraph that the
>> original rationale for the design of the storage manager doesn't hold
>> these days. Of course, it's also obvious from reading the code, since
>> for example there is only one storage manager module.
> 
> Yeah.  There were actually two storage managers in what we inherited
> from Berkeley, but we soon got rid of the other one as being useless.
> (IIRC it was meant for magnetic-core memory ... anybody seen any of
> that lately?)

I remember that I had found mm.c when I started looking around the
storage manager code, maybe a decade ago. It seemed very interesting,
but I've never seen "magnetic-core memory" itself. :)

Anyway, I'm realizing that it's reasonable to add new probes into the
md module at this time.

Thanks for the comments.

>
> I think basically what happened since then is that the functionality
> Stonebraker et al imagined as being in per-storage-manager code all
> migrated into the kernel device drivers, or even down into the hardware
> itself.  (SSDs are *way* smarter than the average '80s storage device,
> and even those were an order of magnitude smarter than what they'd been
> ten years previously.  I used to do device drivers back in the 80's...)
> There's no longer any good reason to have anything but md.c, which isn't
> so much a "magnetic disk" interface as an "interface to something that
> has a Unix block device driver".

BTW, I'm still interested in keeping the storage manager architecture
"theoretically" pluggable. From my point of view, current pluggable
architecture was designed to serve for different storage manager with
by-passing the posix file system API.

I agree with that most of recent storage devices can be managed under
ordinally file systems, ext3/ext4 or xfs for examples.

However, I'm still curious to see new project which will intend to
extend storage manager to earn more performance for some special
storage devices by by-passing ordinally file system API.

For example, Fusion-IO is offering dedicated API for their PCI-Express
Flash storage, and I guess the database performace could tremendously
benefit from it. (I accept that it's just my curious though. :)

It's just a possible option, but keeping the storage architecture
pluggable does make sense to me even if the community could give only
one storage manager.

Regards,

> 
>> This state of affairs sort of reminds me of mcxt.c . The struct
>> MemoryContextData is described as an "abstract type" that can have
>> multiple implementations, despite the fact that since 2000 (and
>> perhaps earlier), the underlying type is invariably AllocSetContext. I
>> never investigated if that indirection still needs to exist, but I
>> suspect that it too is a candidate for refactoring. Do you agree?
> 
> Meh.  Having invented the MemoryContext interface, I am probably not
> the best-qualified person to evaluate it objectively.  The original
> thought was that we might have (a) a context type that could allocate
> storage in shared memory, and/or (b) a context type that could provide
> better allocation speed at a loss of storage efficiency (eg, lose the
> ability to pfree individual chunks).  Case (a) has never become
> practical given the inability of SysV-style shared memory to expand at
> all.  I don't know if that might change when/if we switch to some other
> shmem API.  The idea of a different allocation strategy for some usages
> still seems like something we'll want to do someday, though.
> 
>             regards, tom lane
> 


-- 
Satoshi Nagayasu <snaga@uptime.jp>
Uptime Technologies, LLC. http://www.uptime.jp