Re: [HACKERS] Pluggable storage - Mailing list pgsql-hackers

From Haribabu Kommi
Subject Re: [HACKERS] Pluggable storage
Date
Msg-id CAJrrPGf17rSrM_uonLpASZ3d3nLSWTPj_TD5S8yQc6OkVS2=VA@mail.gmail.com
Whole thread Raw
In response to Re: [HACKERS] Pluggable storage  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: [HACKERS] Pluggable storage
List pgsql-hackers

On Fri, Oct 27, 2017 at 4:06 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Oct 25, 2017 at 1:59 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
>> Another thing to consider is that, if we could replace satisfiesfunc,
>> it would probably break some existing code.  There are multiple places
>> in the code that compare snapshot->satisfies to
>> HeapTupleSatisfiesHistoricMVCC and HeapTupleSatisfiesMVCC.
>>
>> I think the storage API should just leave snapshots alone.  If a
>> storage engine wants to call HeapTupleSatisfiesVisibility() with that
>> snapshot, it can do so.  Otherwise it can switch on
>> snapshot->satisfies and handle each case however it likes.
>>
>
> How will it switch satisfies at runtime?  There are places where we
> might know which visibility function (*MVCC , *Dirty, etc) needs to be
> called, but I think there are other places (like heap_fetch) where it
> is not clear and we decide based on what is stored in
> snapshot->satisfies.

An alternative storage engine needs to provide its own implementation
of heap_fetch, and that replacement implementation can implement MVCC
and other snapshot behavior in any way it likes.


In the current set of patches, I changed the snapshot->satisfies function
point to an enum type, Based on the snapshot visibility type, internally
the storage AM will call the corresponding visibility function.

Additional changes that are done in the patches compared to earlier
patches apart from rebase.

0004-Adding tuple visibility:

Tuple visibility API functions are reduced to 3. It still needs further
optimization.

Tuple satisfies visibility check is added to heap function that don't have
currently.

0006-tuple-insert

Move the index tuple insertion logic inside storage Am with a function
pointer, applicable to insert and updates. yet to handle the insert index tuples
for multi insert scenario.

Removed the speculative finish API. Yet to remove abort API.


Known pending items:

1. Provide a generic new API like heap_fetch to remove heap_hot_search
and visibility functions usage.
2. Move toast table details into storage AM, Toast method depends on storage.
and also toast flattening function needs to be replaced with some generic
functions.
3. Provide new API to get the heaptuple from index or building the index.
may be the same API like heap_fetch may satisfy this requirement also.
4. Bulk insert functionality needs a separate API to deal with all storage AMs.
5. Provide a framework to add reloptions based on storage.
6. Needs a generic API to support rewrite the heap, (cluster command)

Regards,
Hari Babu
Fujitsu Australia
Attachment

pgsql-hackers by date:

Previous
From: Kyotaro HORIGUCHI
Date:
Subject: Re: [HACKERS] More stats about skipped vacuums
Next
From: Michael Paquier
Date:
Subject: Re: [HACKERS] [bug fix] postgres.exe crashes with access violation onWindows while starting up