Re: Relation extension scalability - Mailing list pgsql-hackers

From Amit Kapila
Subject Re: Relation extension scalability
Date
Msg-id CAA4eK1K0=ToYcYpdDVNobKy2QaSd8f3B63hgvmW+m2mZVJJBYA@mail.gmail.com
Whole thread Raw
In response to Re: Relation extension scalability  (Robert Haas <robertmhaas@gmail.com>)
Responses Re: Relation extension scalability
List pgsql-hackers
On Fri, Mar 4, 2016 at 9:59 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Fri, Mar 4, 2016 at 12:06 AM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > I have tried the approach of group extend,
> >
> > 1. We convert the extension lock to TryLock and if we get the lock then
> > extend by one block.2.
> > 2. If don't get the Lock then use the Group leader concep where only one
> > process will extend for all, Slight change from ProcArrayGroupClear is that
> > here other than satisfying the requested backend we Add some extra blocks in
> > FSM, say GroupSize*10.
> > 3. So Actually we can not get exact load but still we have some factor like
> > group size tell us exactly the contention size and we extend in multiple of
> > that.
>
> This approach seems good to me, and the performance results look very
> positive.  The nice thing about this is that there is not a
> user-configurable knob; the system automatically determines when
> larger extensions are needed, which will mean that real-world users
> are much more likely to benefit from this.
>

I think one thing which needs more thoughts about this approach is that we need to maintain some number of slots so that group extend for different relations can happen in parallel.  Do we want to provide simultaneous extension for 1, 2, 3, 4, 5 or more number of relations?  I think providing it for three or four relations should be okay as higher the number we want to provide, bigger the size of PGPROC structure will be.

+GroupExtendRelation(PGPROC *proc, Relation relation, BulkInsertState bistate)

+{

+ volatile PROC_HDR *procglobal = ProcGlobal;

+ uint32 nextidx;

+ uint32 wakeidx;

+ int extraWaits = -1;

+ BlockNumber targetBlock;

+ int count = 0;

+

+ /* Add ourselves to the list of processes needing a group extend. */

+ proc->groupExtendMember = true;

..

..

+ /* We are the leader.  Acquire the lock on behalf of everyone. */

+ LockRelationForExtension(relation, ExclusiveLock);



To provide it for multiple relations, I think you need to advocate the reloid for relation in each proc and then get the relation descriptor for relation extension lock.



With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

pgsql-hackers by date:

Previous
From: Petr Jelinek
Date:
Subject: Re: VS 2015 support in src/tools/msvc
Next
From: Michael Paquier
Date:
Subject: Re: VS 2015 support in src/tools/msvc