Re: [HACKERS] Pluggable storage - Mailing list pgsql-hackers
| From | Haribabu Kommi |
|---|---|
| Subject | Re: [HACKERS] Pluggable storage |
| Date | |
| Msg-id | CAJrrPGfRDfvvjTvoQd-bs7-DbYBTx1UwxdTVmvPeGKZZS8prDw@mail.gmail.com Whole thread Raw |
| In response to | Re: [HACKERS] Pluggable storage (Amit Kapila <amit.kapila16@gmail.com>) |
| Responses |
Re: [HACKERS] Pluggable storage
|
| List | pgsql-hackers |
On Sun, Jul 23, 2017 at 4:10 PM, Amit Kapila <amit.kapila16@gmail.com> wrote:
On Wed, Jul 19, 2017 at 11:33 AM, Haribabu Kommi
<kommi.haribabu@gmail.com> wrote:
>
> I am finding out that eliminating the HeapTuple usage in the upper layers
> needs some major changes, How about not changing anything in the upper
> layers of storage currently and just support the pluggable tuple with one of
> the following approach for first version?
>
It is not very clear to me how any of the below alternatives are
better or worse as compare to the approach you are already working.
Do you mean to say that we will get away without changing all the
places which take HeapTuple as input or return it as output with some
of the below approaches?
Yes, With the following approaches, my intention is to reduce the changing
of HeapTuple everywhere to support the Pluggable Storage API with minimal
changes and later improvise further. But until unless we change the HeapTuple
everywhere properly, may be we can't see the benefits of pluggable storages.
> 1. Design an API that returns values/nulls array and convert that into a
> HeapTuple whenever it is required in the upper layers. All the existing
> heap form/deform tuples are used for every tuple with some adjustments.
>
So, this would have the additional cost of form/deform. Also, how
would it have lesser changes as compare to what you have described
earlier?
Yes, It have the additional cost of form/deform. It is the same approach that
is described earlier. But I have an intention of modifying everywhere the
HeapTuple is accessed. But with the other prototype changes of removing
HeapTuple usage from triggers, I realized that it needs some clear design
to proceed further, instead of combining those changes with pluggable
Storage API.
- heap_getnext function is kept as it as and it is used only for system table
access.
- heap_getnext_slot function is introduced to return the slot whenever the
data is found, otherwise NULL, This function is used in all the places from
Executor and etc.
- The TupleTableSlot structure is modified to contain a void* tuple instead of
HeapTuple. And also it contains the storagehanlder functions.
- heap_insert and etc function can take Slot as an argument and perform the
insert operation.
The cases where the TupleTableSlot is not possible to sent, form a HeapTuple
from the data and sent it and also note down that it is a HeapTuple data, not
the tuple from the storage.
> 2. Design an API that returns StorageTuple(void *) with first member
> represents the TYPE of the storage, so that corresponding registered
> function calls can be called to deform/form the tuple whenever there is
> a need of tuple.
>
Do you intend to say that we store such information in disk tuple or
only in the in-memory version of same?
Only in-memory version.
Also, what makes you think
that we would need hooks only for form and deform? Right now, in many
cases tuple will directly point to disk page and we deal with it by
retaining the pin on the corresponding buffer, what if some kinds of
tuple don't follow that rule? For ex. to support in-place updates, we
might always need a separate copy of tuple rather than the one
pointing to disk page.
In any of the approaches, except for system tables, we are going to remove
the direct disk access of the tuple. Either with replace tuple with slot or something,
no direct disk access will be removed. Otherwise it will be difficult to support,
and also it doesn't provide much performance benefit also.
All the storagehandler functions needs to be maintained seperately
and accessed by them using the hanlder ID.
heap_getnext function returns StorageTuple and not HeapTuple.
The StorageTuple can be mapped to HeapTuple or another Tuple
based on the first member type.
heap_insert etc function takes input as StorageTuple and internally
decides based on the type of the tuple to perform the insert operation.
Instead of storing the handler functions inside a relation/slot, the
function pointers can be accessed directly based on the storage
type data.
This works every case where the tuple is accessed, but the problem
is, it may need changes wherever the tuple is accessed.
> 3. Design an API that returns StorageTuple(void *) but the necessary
> format information of that tuple can be get from the tupledesc. wherever
> the tuple is present, there exists a tupledesc in most of the cases. How
> about adding some kind of information in tupledesc to find out the tuple
> format and call the necessary functions
>
heap_getnext function returns StorageTuple instead of HeapTuple. The tuple
type information is available in the TupleDesc structure.
All heap_insert and etc function accepts TupleTableSlot as input and perform
the insert operation. This approach is almost same as first approach except the
storage handler functions are stored in TupleDesc.
In case if the tuple is formed internally based on the value/nulls array, the formed
tuple is always the HeapTuple format and the same is updated in TupleDesc.
I am having doubt that passing the StorageTuple everywhere in case that location
doesn't contains a TupleDesc and direct access of the tuple may give problems.
> 4. Design an API to return always the StorageTuple and it converts to
> HeapTuple with a function hook if it gets registered (for heap storages
> this is not required to register the hook, because it is already a HeapTuple
> format). This function hook should be placed in the heap form/deform
> functions.
>
heap_getnext function always returns HeapTuple irrespective of the storage.
That means all other storage modules have to frame the HeapTuple from the
data and send it back to the server.
heap_insert and etc functions can work with same interfaces and convert
the HeapTuple internally and perform the insert operation according to their
storage requirement.
There are not many changes in the structures, just adding the storage handler
functions.
This approach is simple, but it doesn't provide much benefit in having a different
storage in my opinion.
I am preferring to go with the Original (first) approach and generate the HeapTuple
wherever it is necessary in upper layers, but the API will return a TupleTableSlot
with either void pointer to tuple or value/nulls array.
Please provide your views.
Regards,
Hari Babu
Fujitsu Australia
pgsql-hackers by date: