From 0b55832169d2939345c40f26f193f977b399e181 Mon Sep 17 00:00:00 2001
From: Peter Geoghegan <pg@bowt.ie>
Date: Mon, 1 Oct 2018 15:51:53 -0700
Subject: [PATCH v6 2/6] Weigh suffix truncation when choosing a split point.

Add infrastructure to determine where the earliest difference appears
among a pair of tuples enclosing a candidate split point.  Use this
within _bt_findsplitloc() to weigh how effective suffix truncation will
be.  This is primarily useful because it maximizes the effectiveness of
suffix truncation.  This should not noticeably affect the balance of
free space within each half of the split.

_bt_findsplitloc() is also taught to care about the case where there are
many duplicates, making it hard to find a distinguishing split point.
_bt_findsplitloc() may even conclude that it isn't possible to avoid
filling a page entirely with duplicates, in which case it packs pages
full of duplicates very tightly.

The number of cycles added is not very noticeable, which is important,
since _bt_findsplitloc() is run while an exclusive (leaf page) buffer
lock is held.  We avoid using authoritative insertion scankey
comparisons, unlike suffix truncation proper.

This patch is required to credibly assess anything about the performance
of the patch series.  Applying the patches up to and including this
patch in the series is sufficient to see much better space utilization
and space reuse with cases where many duplicates are inserted.  (Cases
resulting in searches for free space among many pages full of
duplicates, where the search inevitably "gets tired" on the master
branch [1]).

[1] https://postgr.es/m/CAH2-Wzmf0fvVhU+SSZpGW4Qe9t--j_DmXdX3it5JcdB8FF2EsA@mail.gmail.com
---
 src/backend/access/nbtree/README      |  64 ++-
 src/backend/access/nbtree/nbtinsert.c | 599 +++++++++++++++++++++++---
 src/backend/access/nbtree/nbtutils.c  |  78 ++++
 src/include/access/nbtree.h           |   8 +-
 4 files changed, 682 insertions(+), 67 deletions(-)

diff --git a/src/backend/access/nbtree/README b/src/backend/access/nbtree/README
index 48864910b4..8bb3bf5de8 100644
--- a/src/backend/access/nbtree/README
+++ b/src/backend/access/nbtree/README
@@ -159,9 +159,9 @@ Lehman and Yao assume fixed-size keys, but we must deal with
 variable-size keys.  Therefore there is not a fixed maximum number of
 keys per page; we just stuff in as many as will fit.  When we split a
 page, we try to equalize the number of bytes, not items, assigned to
-each of the resulting pages.  Note we must include the incoming item in
-this calculation, otherwise it is possible to find that the incoming
-item doesn't fit on the split page where it needs to go!
+pages (though suffix truncation is also considered).  Note we must include
+the incoming item in this calculation, otherwise it is possible to find
+that the incoming item doesn't fit on the split page where it needs to go!
 
 The Deletion Algorithm
 ----------------------
@@ -670,6 +670,64 @@ variable-length types, such as text.  An opclass support function could
 manufacture the shortest possible key value that still correctly separates
 each half of a leaf page split.
 
+There is sophisticated criteria for choosing a leaf page split point.  The
+general idea is to make suffix truncation effective without unduly
+influencing the balance of space for each half of the page split.  The
+choice of split point can be thought of as a choice among points "between"
+items on the page to be split, at least if you pretend that the incoming
+tuple was placed on the page already, without provoking a split.  The split
+point between two index tuples with differences that appear as early as
+possible allows us to truncate away as many attributes as possible.
+Obviously suffix truncation is valuable because it makes pivot tuples
+smaller, which delays splits of internal pages, but that isn't the only
+reason why it's effective.  There are cases where suffix truncation can
+leave a B-Tree significantly smaller in size than it would have otherwise
+been, without actually making any pivot tuple smaller because of
+restrictions to preserve alignment.  Truncation prevents internal nodes
+from being more restrictive than truly necessary in how they describe which
+values belong on which leaf pages.  Average space utilization on the leaf
+level can be improved.  The number of internal pages is still reduced, but
+only as an indirect consequence.
+
+While it's not possible to correctly perform suffix truncation during
+internal page splits, it's still useful to be discriminating when splitting
+an internal page.  The split point that implies a downlink be inserted in
+the parent that's the smallest one available within an acceptable range of
+the optimal fillfactor-wise split point is chosen.  This idea also comes
+from the Prefix B-Tree paper.  This process has much in common with to what
+happens at the leaf level to make suffix truncation effective.  The overall
+effect is that suffix truncation tends to produce smaller and less
+discriminating pivot tuples, especially early in the lifetime of the index,
+while biasing internal page splits makes the earlier, less discriminating
+pivot tuples end up in the root page, delaying root page splits.
+
+Note that suffix truncation occasionally makes a pivot tuple larger than
+the leaf tuple that it's based on, since a heap TID must be appended when
+nothing else would distinguish each side of a leaf split.  This is
+represented differently in pivot tuples, for historic reasons.  Every tuple
+at the leaf level must be individually locatable by an insertion scankey
+that's fully filled-out by _bt_mkscankey(), so there is no way to
+completely avoid this.  As of Postgres v12, heap TID is treated as a
+tie-breaker key attribute to make this work. Adding a heap TID attribute
+during a leaf page split should only occur when there is an entire page
+full of duplicates, though, since the logic for selecting a split point
+will do all it can to avoid this outcome.
+
+Avoiding appending a heap TID to a pivot tuple is about much more than just
+making saving a single MAXALIGN() quantum.  It's worth going out of our way
+to avoid having a single value (or composition of key values) span two leaf
+pages when that isn't truly necessary, since if that's allowed to happen
+every point index scan will have to visit both pages.  It also makes it
+less likely that VACUUM will be able to perform page deletion on either
+page.  This is a kind of "false sharing".  Many duplicates mode is the
+mechanism that goes to great lengths to avoiding appending a heap TID.  It
+may lead to slightly inferior space utilization in extreme cases, when
+values are spaced apart at fixed intervals, even on levels above the leaf
+level.  This is considered an acceptable price to avoid groups of
+duplicates that straddle two leaf pages. A point lookup will only visit one
+leaf page, not two, since the high key on the first page will indicate that
+all duplicate values must be located on the first leaf page.
+
 Notes About Data Representation
 -------------------------------
 
diff --git a/src/backend/access/nbtree/nbtinsert.c b/src/backend/access/nbtree/nbtinsert.c
index 825be932b5..a8332db7de 100644
--- a/src/backend/access/nbtree/nbtinsert.c
+++ b/src/backend/access/nbtree/nbtinsert.c
@@ -28,25 +28,40 @@
 
 /* Minimum tree height for application of fastpath optimization */
 #define BTREE_FASTPATH_MIN_LEVEL	2
+#define STACK_SPLIT_POINTS			15
+
+typedef enum
+{
+	/* strategy to use for a call to FindSplitData */
+	SPLIT_DEFAULT,				/* give some weight to truncation */
+	SPLIT_MANY_DUPLICATES,		/* find minimally distinguishing point */
+	SPLIT_SINGLE_VALUE			/* leave left page almost empty */
+} SplitMode;
+
+typedef struct
+{
+	/* FindSplitData candidate split */
+	int			delta;			/* size delta */
+	bool		newitemonleft;	/* new item on left or right of split */
+	OffsetNumber firstright;	/* split point */
+} SplitPoint;
 
 typedef struct
 {
 	/* context data for _bt_checksplitloc */
 	Size		newitemsz;		/* size of new item to be inserted */
-	int			fillfactor;		/* needed when splitting rightmost page */
+	int			fillfactor;		/* needed for weighted splits */
 	bool		is_leaf;		/* T if splitting a leaf page */
-	bool		is_rightmost;	/* T if splitting a rightmost page */
+	bool		is_weighted;	/* T if weighted (e.g. rightmost) split */
 	OffsetNumber newitemoff;	/* where the new item is to be inserted */
+	bool		hikeyheaptid;	/* T if high key will likely get heap TID */
 	int			leftspace;		/* space available for items on left page */
 	int			rightspace;		/* space available for items on right page */
 	int			olddataitemstotal;	/* space taken by old items */
 
-	bool		have_split;		/* found a valid split? */
-
-	/* these fields valid only if have_split is true */
-	bool		newitemonleft;	/* new item on left or right of best split */
-	OffsetNumber firstright;	/* best split point */
-	int			best_delta;		/* best size delta so far */
+	int			maxsplits;		/* Maximum number of splits */
+	int			nsplits;		/* Current number of splits */
+	SplitPoint *splits;			/* Sorted by delta */
 } FindSplitData;
 
 
@@ -74,12 +89,22 @@ static Buffer _bt_split(Relation rel, Buffer buf, Buffer cbuf,
 static void _bt_insert_parent(Relation rel, Buffer buf, Buffer rbuf,
 				  BTStack stack, bool is_root, bool is_only);
 static OffsetNumber _bt_findsplitloc(Relation rel, Page page,
-				 OffsetNumber newitemoff,
-				 Size newitemsz,
-				 bool *newitemonleft);
-static void _bt_checksplitloc(FindSplitData *state,
+				 SplitMode mode, OffsetNumber newitemoff,
+				 Size newitemsz, IndexTuple newitem, bool *newitemonleft);
+static int _bt_checksplitloc(FindSplitData *state,
 				  OffsetNumber firstoldonright, bool newitemonleft,
 				  int dataitemstoleft, Size firstoldonrightsz);
+static OffsetNumber _bt_bestsplitloc(Relation rel, Page page,
+				 FindSplitData *state,
+				 int perfectpenalty,
+				 OffsetNumber newitemoff,
+				 IndexTuple newitem, bool *newitemonleft);
+static int _bt_perfect_penalty(Relation rel, Page page, bool is_leaf,
+					SplitMode mode, OffsetNumber newitemoff,
+					IndexTuple newitem, int nsplits,
+					SplitPoint *splits, SplitMode *secondmode);
+static int _bt_split_penalty(Relation rel, Page page, OffsetNumber newitemoff,
+				  IndexTuple newitem, SplitPoint *split, bool is_leaf);
 static bool _bt_pgaddtup(Page page, Size itemsize, IndexTuple itup,
 			 OffsetNumber itup_off);
 static bool _bt_isequal(TupleDesc itupdesc, BTScanInsert itup_scankey,
@@ -852,8 +877,8 @@ _bt_insertonpg(Relation rel,
 				 BlockNumberIsValid(RelationGetTargetBlock(rel))));
 
 		/* Choose the split point */
-		firstright = _bt_findsplitloc(rel, page,
-									  newitemoff, itemsz,
+		firstright = _bt_findsplitloc(rel, page, SPLIT_DEFAULT,
+									  newitemoff, itemsz, itup,
 									  &newitemonleft);
 
 		/* split the buffer into left and right halves */
@@ -1527,6 +1552,25 @@ _bt_split(Relation rel, Buffer buf, Buffer cbuf, OffsetNumber firstright,
  * for it, we might find ourselves with too little room on the page that
  * it needs to go into!)
  *
+ * We also give some weight to suffix truncation in deciding a split point
+ * on leaf pages.  We try to select a point where a distinguishing attribute
+ * appears earlier in the new high key for the left side of the split, in
+ * order to maximize the number of trailing attributes that can be truncated
+ * away.  Generally speaking, only candidate split points that fall within
+ * an acceptable space utilization range are considered.  This is even
+ * useful with pages that only have a single (non-TID) attribute, since it's
+ * helpful to avoid appending an explicit heap TID attribute to the new pivot
+ * tuple (high key/downlink) when it cannot actually be truncated.
+ *
+ * We do all we can to avoid having to append a heap TID in the new high
+ * key.  We may have to call ourselves recursively in many duplicates mode.
+ * This happens when a heap TID would otherwise be appended, but the page
+ * isn't completely full of logical duplicates (there may be a few as two
+ * distinct values).  Many duplicates mode has no hard requirements for
+ * space utilization, though it still keeps the use of space balanced as a
+ * non-binding secondary goal.  This significantly improves fan-out in
+ * practice, at least with most affected workloads.
+ *
  * If the page is the rightmost page on its level, we instead try to arrange
  * to leave the left split page fillfactor% full.  In this way, when we are
  * inserting successively increasing keys (consider sequences, timestamps,
@@ -1535,6 +1579,18 @@ _bt_split(Relation rel, Buffer buf, Buffer cbuf, OffsetNumber firstright,
  * This is the same as nbtsort.c produces for a newly-created tree.  Note
  * that leaf and nonleaf pages use different fillfactors.
  *
+ * If called recursively in single value mode, we also try to arrange to
+ * leave the left split page fillfactor% full, though we arrange to use a
+ * fillfactor that leaves the left page mostly empty and the right page
+ * mostly full, rather than the other way around.  This greatly helps with
+ * space management in cases where tuples with the same attribute values
+ * span multiple pages.  Newly inserted duplicates will tend to have higher
+ * heap TID values, so we'll end up splitting the same page again and again
+ * as even more duplicates are inserted.  (The heap TID attribute has
+ * descending sort order, so ascending heap TID values continually split the
+ * same low page).  See nbtree/README for more information about suffix
+ * truncation, and how a split point is chosen.
+ *
  * We are passed the intended insert position of the new tuple, expressed as
  * the offsetnumber of the tuple it must go in front of.  (This could be
  * maxoff+1 if the tuple is to go at the end.)
@@ -1547,8 +1603,10 @@ _bt_split(Relation rel, Buffer buf, Buffer cbuf, OffsetNumber firstright,
 static OffsetNumber
 _bt_findsplitloc(Relation rel,
 				 Page page,
+				 SplitMode mode,
 				 OffsetNumber newitemoff,
 				 Size newitemsz,
+				 IndexTuple newitem,
 				 bool *newitemonleft)
 {
 	BTPageOpaque opaque;
@@ -1560,13 +1618,15 @@ _bt_findsplitloc(Relation rel,
 				rightspace,
 				goodenough,
 				olddataitemstotal,
-				olddataitemstoleft;
+				olddataitemstoleft,
+				perfectpenalty;
 	bool		goodenoughfound;
+	SplitPoint	splits[STACK_SPLIT_POINTS];
+	SplitMode	secondmode;
+	OffsetNumber finalfirstright;
 
 	opaque = (BTPageOpaque) PageGetSpecialPointer(page);
-
-	/* Passed-in newitemsz is MAXALIGNED but does not include line pointer */
-	newitemsz += sizeof(ItemIdData);
+	maxoff = PageGetMaxOffsetNumber(page);
 
 	/* Total free space available on a btree page, after fixed overhead */
 	leftspace = rightspace =
@@ -1584,18 +1644,44 @@ _bt_findsplitloc(Relation rel,
 	/* Count up total space in data items without actually scanning 'em */
 	olddataitemstotal = rightspace - (int) PageGetExactFreeSpace(page);
 
-	state.newitemsz = newitemsz;
+	/* Passed-in newitemsz is MAXALIGNED but does not include line pointer */
+	state.newitemsz = newitemsz + sizeof(ItemIdData);
+	state.hikeyheaptid = (mode == SPLIT_SINGLE_VALUE);
 	state.is_leaf = P_ISLEAF(opaque);
-	state.is_rightmost = P_RIGHTMOST(opaque);
-	state.have_split = false;
+	state.is_weighted = P_RIGHTMOST(opaque);
 	if (state.is_leaf)
-		state.fillfactor = RelationGetFillFactor(rel,
-												 BTREE_DEFAULT_FILLFACTOR);
+	{
+		if (mode != SPLIT_SINGLE_VALUE)
+		{
+			/* Only used on rightmost page */
+			state.fillfactor = RelationGetFillFactor(rel,
+													 BTREE_DEFAULT_FILLFACTOR);
+		}
+		else
+		{
+			state.fillfactor = BTREE_SINGLEVAL_FILLFACTOR;
+			state.is_weighted = true;
+		}
+	}
 	else
+	{
+		Assert(mode == SPLIT_DEFAULT);
+		/* Only used on rightmost page */
 		state.fillfactor = BTREE_NONLEAF_FILLFACTOR;
-	state.newitemonleft = false;	/* these just to keep compiler quiet */
-	state.firstright = 0;
-	state.best_delta = 0;
+	}
+
+	if (mode == SPLIT_DEFAULT)
+		state.maxsplits = Min(Max(1, maxoff / 16), STACK_SPLIT_POINTS);
+	else if (mode == SPLIT_MANY_DUPLICATES)
+		state.maxsplits = maxoff + 2;
+	else
+		state.maxsplits = 1;
+	state.nsplits = 0;
+	if (mode != SPLIT_MANY_DUPLICATES)
+		state.splits = splits;
+	else
+		state.splits = palloc(sizeof(SplitPoint) * state.maxsplits);
+
 	state.leftspace = leftspace;
 	state.rightspace = rightspace;
 	state.olddataitemstotal = olddataitemstotal;
@@ -1604,11 +1690,13 @@ _bt_findsplitloc(Relation rel,
 	/*
 	 * Finding the best possible split would require checking all the possible
 	 * split points, because of the high-key and left-key special cases.
-	 * That's probably more work than it's worth; instead, stop as soon as we
-	 * find a "good-enough" split, where good-enough is defined as an
-	 * imbalance in free space of no more than pagesize/16 (arbitrary...) This
-	 * should let us stop near the middle on most pages, instead of plowing to
-	 * the end.
+	 * That's probably more work than it's worth in default mode; instead,
+	 * stop as soon as we find all "good-enough" splits, where good-enough is
+	 * defined as an imbalance in free space of no more than pagesize/16
+	 * (arbitrary...) This should let us stop near the middle on most pages,
+	 * instead of plowing to the end.  Many duplicates mode does consider all
+	 * choices, while single value mode gives up as soon as it finds a good
+	 * enough split point.
 	 */
 	goodenough = leftspace / 16;
 
@@ -1618,13 +1706,13 @@ _bt_findsplitloc(Relation rel,
 	 */
 	olddataitemstoleft = 0;
 	goodenoughfound = false;
-	maxoff = PageGetMaxOffsetNumber(page);
 
 	for (offnum = P_FIRSTDATAKEY(opaque);
 		 offnum <= maxoff;
 		 offnum = OffsetNumberNext(offnum))
 	{
 		Size		itemsz;
+		int			delta;
 
 		itemid = PageGetItemId(page, offnum);
 		itemsz = MAXALIGN(ItemIdGetLength(itemid)) + sizeof(ItemIdData);
@@ -1633,27 +1721,41 @@ _bt_findsplitloc(Relation rel,
 		 * Will the new item go to left or right of split?
 		 */
 		if (offnum > newitemoff)
-			_bt_checksplitloc(&state, offnum, true,
-							  olddataitemstoleft, itemsz);
+			delta = _bt_checksplitloc(&state, offnum, true,
+									  olddataitemstoleft, itemsz);
 
 		else if (offnum < newitemoff)
-			_bt_checksplitloc(&state, offnum, false,
-							  olddataitemstoleft, itemsz);
+			delta = _bt_checksplitloc(&state, offnum, false,
+									  olddataitemstoleft, itemsz);
 		else
 		{
 			/* need to try it both ways! */
-			_bt_checksplitloc(&state, offnum, true,
-							  olddataitemstoleft, itemsz);
+			(void) _bt_checksplitloc(&state, offnum, true,
+									 olddataitemstoleft, itemsz);
 
-			_bt_checksplitloc(&state, offnum, false,
-							  olddataitemstoleft, itemsz);
+			delta = _bt_checksplitloc(&state, offnum, false,
+									  olddataitemstoleft, itemsz);
 		}
 
-		/* Abort scan once we find a good-enough choice */
-		if (state.have_split && state.best_delta <= goodenough)
-		{
+		/* Record when good-enough choice found */
+		if (state.nsplits > 0 && state.splits[0].delta <= goodenough)
 			goodenoughfound = true;
-			break;
+
+		/*
+		 * Abort default mode scan once we've found a good-enough choice, and
+		 * reach the point where we stop finding new good-enough choices.
+		 * Might as well abort as soon as good-enough split point is found in
+		 * single value mode, where we won't discriminate between a selection
+		 * of split points based on their penalty.
+		 */
+		if (goodenoughfound)
+		{
+			if (mode == SPLIT_DEFAULT && delta > goodenough)
+				break;
+			else if (mode == SPLIT_SINGLE_VALUE)
+				break;
+
+			/* Many duplicates mode must be exhaustive */
 		}
 
 		olddataitemstoleft += itemsz;
@@ -1664,19 +1766,52 @@ _bt_findsplitloc(Relation rel,
 	 * the old items go to the left page and the new item goes to the right
 	 * page.
 	 */
-	if (newitemoff > maxoff && !goodenoughfound)
+	if (newitemoff > maxoff &&
+		(!goodenoughfound || mode == SPLIT_MANY_DUPLICATES))
 		_bt_checksplitloc(&state, newitemoff, false, olddataitemstotal, 0);
 
 	/*
 	 * I believe it is not possible to fail to find a feasible split, but just
 	 * in case ...
 	 */
-	if (!state.have_split)
+	if (state.nsplits == 0)
 		elog(ERROR, "could not find a feasible split point for index \"%s\"",
 			 RelationGetRelationName(rel));
 
-	*newitemonleft = state.newitemonleft;
-	return state.firstright;
+	/*
+	 * Search among acceptable split points for the entry with the lowest
+	 * penalty.  See _bt_split_penalty() for the definition of penalty.  The
+	 * goal here is to increase fan-out, by choosing a split point which is
+	 * amenable to being made smaller by suffix truncation, or is already
+	 * small.
+	 *
+	 * First find lowest possible penalty among acceptable split points -- the
+	 * "perfect" penalty.  This will be passed to _bt_bestsplitloc() if it
+	 * determines that candidate split points are good enough to finish
+	 * default mode split.  Perfect penalty saves _bt_bestsplitloc()
+	 * additional work around calculating penalties.
+	 */
+	perfectpenalty = _bt_perfect_penalty(rel, page, state.is_leaf, mode,
+										 newitemoff, newitem,
+										 state.nsplits, state.splits,
+										 &secondmode);
+
+	/* Start second pass over page if _bt_perfect_penalty() told us to */
+	if (secondmode != SPLIT_DEFAULT)
+		return _bt_findsplitloc(rel, page, secondmode, newitemoff, newitemsz,
+								newitem, newitemonleft);
+
+	/*
+	 * Search among acceptable split points for the entry that has the lowest
+	 * penalty, and thus maximizes fan-out.
+	 */
+	finalfirstright = _bt_bestsplitloc(rel, page, &state, perfectpenalty,
+									   newitemoff, newitem, newitemonleft);
+	/* Be tidy */
+	if (state.splits != splits)
+		pfree(state.splits);
+
+	return finalfirstright;
 }
 
 /*
@@ -1691,8 +1826,11 @@ _bt_findsplitloc(Relation rel,
  *
  * olddataitemstoleft is the total size of all old items to the left of
  * firstoldonright.
+ *
+ * Returns delta between space that will be left free on left and right side
+ * of split.
  */
-static void
+static int
 _bt_checksplitloc(FindSplitData *state,
 				  OffsetNumber firstoldonright,
 				  bool newitemonleft,
@@ -1700,7 +1838,8 @@ _bt_checksplitloc(FindSplitData *state,
 				  Size firstoldonrightsz)
 {
 	int			leftfree,
-				rightfree;
+				rightfree,
+				leftleafheaptidsz;
 	Size		firstrightitemsz;
 	bool		newitemisfirstonright;
 
@@ -1720,15 +1859,38 @@ _bt_checksplitloc(FindSplitData *state,
 
 	/*
 	 * The first item on the right page becomes the high key of the left page;
-	 * therefore it counts against left space as well as right space. When
+	 * therefore it counts against left space as well as right space (we
+	 * cannot assume that suffix truncation will make it any smaller).  When
 	 * index has included attributes, then those attributes of left page high
 	 * key will be truncated leaving that page with slightly more free space.
 	 * However, that shouldn't affect our ability to find valid split
-	 * location, because anyway split location should exists even without high
-	 * key truncation.
+	 * location, since we err in the direction of being pessimistic about free
+	 * space on the left half.  Besides, even when suffix truncation of
+	 * non-TID attributes occurs, there often won't be an entire MAXALIGN()
+	 * quantum in pivot space savings.
 	 */
 	leftfree -= firstrightitemsz;
 
+	/*
+	 * Assume that suffix truncation cannot avoid adding a heap TID to the
+	 * left half's new high key when splitting at the leaf level.  Don't let
+	 * this impact the balance of free space in the common case where adding a
+	 * heap TID is considered very unlikely, though, since there is no reason
+	 * to accept a likely-suboptimal split.
+	 *
+	 * When adding a heap TID seems likely, then actually factor that in to
+	 * delta calculation, rather than just having it as a constraint on
+	 * whether or not a split is acceptable.
+	 */
+	leftleafheaptidsz = 0;
+	if (state->is_leaf)
+	{
+		if (!state->hikeyheaptid)
+			leftleafheaptidsz = sizeof(ItemPointerData);
+		else
+			leftfree -= (int) sizeof(ItemPointerData);
+	}
+
 	/* account for the new item */
 	if (newitemonleft)
 		leftfree -= (int) state->newitemsz;
@@ -1744,17 +1906,20 @@ _bt_checksplitloc(FindSplitData *state,
 			(int) (MAXALIGN(sizeof(IndexTupleData)) + sizeof(ItemIdData));
 
 	/*
-	 * If feasible split point, remember best delta.
+	 * If feasible split point with lower delta than that of most marginal
+	 * spit point so far, or we haven't run out of space for split points,
+	 * remember it.
 	 */
-	if (leftfree >= 0 && rightfree >= 0)
+	if (leftfree - leftleafheaptidsz >= 0 && rightfree >= 0)
 	{
 		int			delta;
 
-		if (state->is_rightmost)
+		if (state->is_weighted)
 		{
 			/*
-			 * If splitting a rightmost page, try to put (100-fillfactor)% of
-			 * free space on left page. See comments for _bt_findsplitloc.
+			 * If splitting a rightmost page, or in single value mode, try to
+			 * put (100-fillfactor)% of free space on left page. See comments
+			 * for _bt_findsplitloc.
 			 */
 			delta = (state->fillfactor * leftfree)
 				- ((100 - state->fillfactor) * rightfree);
@@ -1767,14 +1932,322 @@ _bt_checksplitloc(FindSplitData *state,
 
 		if (delta < 0)
 			delta = -delta;
-		if (!state->have_split || delta < state->best_delta)
+		if (state->nsplits < state->maxsplits ||
+			delta < state->splits[state->nsplits - 1].delta)
 		{
-			state->have_split = true;
-			state->newitemonleft = newitemonleft;
-			state->firstright = firstoldonright;
-			state->best_delta = delta;
+			SplitPoint	newsplit;
+			int			j;
+
+			newsplit.delta = delta;
+			newsplit.newitemonleft = newitemonleft;
+			newsplit.firstright = firstoldonright;
+
+			/*
+			 * Make space at the end of the state array for new candidate
+			 * split point if we haven't already reached the maximum number of
+			 * split points.
+			 */
+			if (state->nsplits < state->maxsplits)
+				state->nsplits++;
+
+			/*
+			 * Replace the final item in the nsplits-wise array.  The final
+			 * item is either a garbage still-uninitialized entry, or the most
+			 * marginal real entry when we already have as many split points
+			 * as we're willing to consider.
+			 */
+			for (j = state->nsplits - 1;
+				 j > 0 && state->splits[j - 1].delta > newsplit.delta;
+				 j--)
+			{
+				state->splits[j] = state->splits[j - 1];
+			}
+			state->splits[j] = newsplit;
+		}
+
+		return delta;
+	}
+
+	return INT_MAX;
+}
+
+/*
+ * Subroutine to find the "best" split point among an array of acceptable
+ * candidate split points that split without there being an excessively high
+ * delta between the space left free on the left and right halves.  The "best"
+ * split point is the split point with the lowest penalty, which is an
+ * abstract idea whose definition varies depending on whether we're splitting
+ * at the leaf level, or an internal level.  See _bt_split_penalty() for the
+ * definition.
+ *
+ * "perfectpenalty" is assumed to be the lowest possible penalty among
+ * candidate split points.  This allows us to return early without wasting
+ * cycles on calculating the first differing attribute for all candidate
+ * splits when that clearly cannot improve our choice.  This optimization is
+ * important for several common cases, including insertion into a primary key
+ * index on an auto-incremented or monotonically increasing integer column.
+ *
+ * We return the index of the first existing tuple that should go on the
+ * righthand page, plus a boolean indicating if new item is on left of split
+ * point.
+ */
+static OffsetNumber
+_bt_bestsplitloc(Relation rel,
+				 Page page,
+				 FindSplitData *state,
+				 int perfectpenalty,
+				 OffsetNumber newitemoff,
+				 IndexTuple newitem,
+				 bool *newitemonleft)
+{
+	int			bestpenalty,
+				lowsplit;
+
+	/* No point calculating penalties in trivial cases */
+	if (perfectpenalty == INT_MAX || state->nsplits == 1)
+	{
+		*newitemonleft = state->splits[0].newitemonleft;
+		return state->splits[0].firstright;
+	}
+
+	/*
+	 * Now actually search among acceptable split points for the entry that
+	 * allows suffix truncation to truncate away the maximum possible number
+	 * of attributes.
+	 */
+	bestpenalty = INT_MAX;
+	lowsplit = 0;
+	for (int i = 0; i < state->nsplits; i++)
+	{
+		int			penalty;
+
+		penalty = _bt_split_penalty(rel, page, newitemoff, newitem,
+									state->splits + i, state->is_leaf);
+
+		if (penalty <= perfectpenalty)
+		{
+			bestpenalty = penalty;
+			lowsplit = i;
+			break;
+		}
+
+		if (penalty < bestpenalty)
+		{
+			bestpenalty = penalty;
+			lowsplit = i;
 		}
 	}
+
+	*newitemonleft = state->splits[lowsplit].newitemonleft;
+	return state->splits[lowsplit].firstright;
+}
+
+/*
+ * Subroutine to find the lowest possible penalty for any acceptable candidate
+ * split point.  This may be lower than any real penalty for any of the
+ * candidate split points, in which case the optimization is ineffective.
+ * Split penalties are generally discrete rather than continuous, so an
+ * actually-obtainable penalty is the common case.
+ *
+ * This is also a convenient point to decide to either finish splitting
+ * the page using the default strategy, or, alternatively, to do a second pass
+ * over page using a different strategy.
+ */
+static int
+_bt_perfect_penalty(Relation rel, Page page, bool is_leaf, SplitMode mode,
+					OffsetNumber newitemoff, IndexTuple newitem,
+					int nsplits, SplitPoint *splits, SplitMode *secondmode)
+{
+	ItemId		itemid;
+	OffsetNumber center;
+	IndexTuple	leftmost,
+				rightmost;
+	int			perfectpenalty;
+
+	/* Assume that a second pass over page won't be required for now */
+	*secondmode = SPLIT_DEFAULT;
+
+	/*
+	 * During a many duplicates pass over page, we settle for a "perfect"
+	 * split point that merely avoids appending a heap TID in new pivot.
+	 * Appending a heap TID is harmful enough to fan-out that it's worth
+	 * avoiding at all costs, but it doesn't make sense to go to those lengths
+	 * to also be able to truncate an extra, earlier attribute.
+	 */
+	if (!is_leaf)
+		return MAXALIGN(sizeof(IndexTupleData) + 1);
+	else if (mode == SPLIT_MANY_DUPLICATES)
+		return IndexRelationGetNumberOfKeyAttributes(rel);
+	else if (mode == SPLIT_SINGLE_VALUE)
+		return INT_MAX;
+
+	/*
+	 * Complicated though common case -- leaf page default mode split.
+	 *
+	 * Iterate from the end of split array to the start, in search of the
+	 * firstright-wise leftmost and rightmost entries among acceptable split
+	 * points.  The split point with the lowest delta is at the start of the
+	 * array.  It is deemed to be the split point whose firstright offset is
+	 * at the center.  Split points with firstright offsets at both the left
+	 * and right extremes among acceptable split points will be found at the
+	 * end of caller's array.
+	 */
+	leftmost = NULL;
+	rightmost = NULL;
+	center = splits[0].firstright;
+
+	/*
+	 * Split points can be thought of as points _between_ tuples on the
+	 * original unsplit page image, at least if you pretend that the incoming
+	 * tuple is already on the page to be split (imagine that the original
+	 * unsplit page actually had enough space to fit the incoming tuple).  The
+	 * rightmost tuple is the tuple that is immediately to the right of a
+	 * split point that is itself rightmost.  Likewise, the leftmost tuple is
+	 * the tuple to the left of the leftmost split point.  This is slightly
+	 * arbitrary.
+	 *
+	 * When there are very few candidates, no sensible comparison can be made
+	 * here, resulting in caller selecting lowest delta/the center split point
+	 * by default.  No great care is taken around boundary cases where the
+	 * center split point has the same firstright offset as either the
+	 * leftmost or rightmost split points (i.e. only newitemonleft differs).
+	 * We expect to find leftmost and rightmost tuples almost immediately.
+	 */
+	perfectpenalty = IndexRelationGetNumberOfKeyAttributes(rel);
+	for (int j = nsplits - 1; j > 1; j--)
+	{
+		SplitPoint *split = splits + j;
+
+		if (!leftmost && split->firstright <= center)
+		{
+			if (split->newitemonleft && newitemoff == split->firstright)
+				leftmost = newitem;
+			else
+			{
+				itemid = PageGetItemId(page,
+									   OffsetNumberPrev(split->firstright));
+				leftmost = (IndexTuple) PageGetItem(page, itemid);
+			}
+		}
+
+		if (!rightmost && split->firstright >= center)
+		{
+			if (!split->newitemonleft && newitemoff == split->firstright)
+				rightmost = newitem;
+			else
+			{
+				itemid = PageGetItemId(page, split->firstright);
+				rightmost = (IndexTuple) PageGetItem(page, itemid);
+			}
+		}
+
+		if (leftmost && rightmost)
+		{
+			Assert(leftmost != rightmost);
+			perfectpenalty = _bt_leave_natts_fast(rel, leftmost, rightmost);
+			break;
+		}
+	}
+
+	/* Work out which type of second pass will be performed, if any */
+	if (perfectpenalty > IndexRelationGetNumberOfKeyAttributes(rel))
+	{
+		BTPageOpaque opaque;
+		OffsetNumber maxoff;
+		int			outerpenalty;
+
+		opaque = (BTPageOpaque) PageGetSpecialPointer(page);
+		maxoff = PageGetMaxOffsetNumber(page);
+
+		if (P_FIRSTDATAKEY(opaque) == newitemoff)
+			leftmost = newitem;
+		else
+		{
+			itemid = PageGetItemId(page, P_FIRSTDATAKEY(opaque));
+			leftmost = (IndexTuple) PageGetItem(page, itemid);
+		}
+
+		if (newitemoff > maxoff)
+			rightmost = newitem;
+		else
+		{
+			itemid = PageGetItemId(page, maxoff);
+			rightmost = (IndexTuple) PageGetItem(page, itemid);
+		}
+
+		Assert(leftmost != rightmost);
+		outerpenalty = _bt_leave_natts_fast(rel, leftmost, rightmost);
+
+		/*
+		 * If page has many duplicates but is not entirely full of duplicates,
+		 * a many duplicates mode pass will be performed.  If page is entirely
+		 * full of duplicates, a single value mode pass will be performed.
+		 *
+		 * Caller should avoid a single value mode pass when incoming tuple
+		 * doesn't sort lowest among items on the page, though.  Instead, we
+		 * instruct caller to continue with original default mode split, since
+		 * an out-of-order new item suggests that newer tuples have come from
+		 * (non-HOT) updates, not inserts.  Evenly sharing space among each
+		 * half of the split avoids pathological performance.
+		 */
+		if (outerpenalty > IndexRelationGetNumberOfKeyAttributes(rel))
+		{
+			if (P_FIRSTDATAKEY(opaque) == newitemoff)
+				*secondmode = SPLIT_SINGLE_VALUE;
+			else
+			{
+				perfectpenalty = INT_MAX;
+				*secondmode = SPLIT_DEFAULT;
+			}
+		}
+		else
+			*secondmode = SPLIT_MANY_DUPLICATES;
+	}
+
+	return perfectpenalty;
+}
+
+/*
+ * Subroutine to find penalty for caller's candidate split point.
+ *
+ * On leaf pages, penalty is the attribute number that distinguishes each side
+ * of a split.  It's the last attribute that needs to be included in new high
+ * key for left page.  It can be greater than the number of key attributes in
+ * cases where a heap TID needs to be appending during truncation.
+ *
+ * On internal pages, penalty is simply the size of the first item on the
+ * right half of the split (excluding ItemId overhead) which becomes the new
+ * high key for the left page.  Internal page splits always use default mode.
+ */
+static int
+_bt_split_penalty(Relation rel, Page page, OffsetNumber newitemoff,
+				  IndexTuple newitem, SplitPoint *split, bool is_leaf)
+{
+	ItemId		itemid;
+	IndexTuple	lastleft;
+	IndexTuple	firstright;
+
+	if (split->newitemonleft && newitemoff == split->firstright)
+		lastleft = newitem;
+	else
+	{
+		itemid = PageGetItemId(page, OffsetNumberPrev(split->firstright));
+		lastleft = (IndexTuple) PageGetItem(page, itemid);
+	}
+
+	if (!split->newitemonleft && newitemoff == split->firstright)
+		firstright = newitem;
+	else
+	{
+		itemid = PageGetItemId(page, split->firstright);
+		firstright = (IndexTuple) PageGetItem(page, itemid);
+	}
+
+	if (!is_leaf)
+		return IndexTupleSize(firstright);
+
+	Assert(lastleft != firstright);
+	return _bt_leave_natts_fast(rel, lastleft, firstright);
 }
 
 /*
diff --git a/src/backend/access/nbtree/nbtutils.c b/src/backend/access/nbtree/nbtutils.c
index fabc676ba3..aeca964716 100644
--- a/src/backend/access/nbtree/nbtutils.c
+++ b/src/backend/access/nbtree/nbtutils.c
@@ -22,6 +22,7 @@
 #include "access/relscan.h"
 #include "miscadmin.h"
 #include "utils/array.h"
+#include "utils/datum.h"
 #include "utils/lsyscache.h"
 #include "utils/memutils.h"
 #include "utils/rel.h"
@@ -2297,6 +2298,83 @@ _bt_leave_natts(Relation rel, IndexTuple lastleft, IndexTuple firstright)
 	return leavenatts;
 }
 
+/*
+ * _bt_leave_natts_fast - fast, approximate variant of _bt_leave_natts.
+ *
+ * This is exported so that a candidate split point can have its effect on
+ * suffix truncation inexpensively evaluated ahead of time when finding a
+ * split location.  A naive bitwise approach to datum comparisons is used to
+ * save cycles.  This is inherently approximate, but usually provides the same
+ * answer as the authoritative approach that _bt_leave_natts takes, since the
+ * vast majority of types in Postgres cannot be equal according to any
+ * available opclass unless they're bitwise equal.
+ *
+ * Testing has shown that an approach involving treating the tuple as a
+ * decomposed binary string would work almost as well as the approach taken
+ * here.  It would also be faster.  It might actually be necessary to go that
+ * way in the future, if suffix truncation is made sophisticated enough to
+ * truncate at a finer granularity (i.e. truncate within an attribute, rather
+ * than just truncating away whole attributes).  The current approach isn't
+ * markedly slower, since it works particularly well with the "perfect
+ * penalty" optimization (there are fewer, more expensive calls here).  It
+ * also works with INCLUDE indexes (indexes with non-key attributes) without
+ * any special effort.
+ *
+ * This can return a number of attributes that is one greater than the
+ * number of key attributes for the index relation.  This indicates that the
+ * caller must use a heap TID as a unique-ifier in new pivot tuple.
+ */
+int
+_bt_leave_natts_fast(Relation rel, IndexTuple lastleft, IndexTuple firstright)
+{
+	TupleDesc	itupdesc = RelationGetDescr(rel);
+	int			keysz = IndexRelationGetNumberOfKeyAttributes(rel);
+	int			result;
+
+	/*
+	 * Using authoritative comparisons makes no difference in almost all
+	 * cases. However, there are a small number of shipped opclasses where
+	 * there might occasionally be an inconsistency between the answers given
+	 * by this function and _bt_leave_natts().  This includes numeric_ops,
+	 * since display scale might vary among logically equal datums.
+	 * Case-insensitive collations may also be interesting.
+	 *
+	 * This is assumed to be okay, since there is no risk that inequality will
+	 * look like equality.  Suffix truncation may be less effective than it
+	 * could be in these narrow cases, but it should be impossible for caller
+	 * to spuriously perform a second pass to find a split location, where
+	 * evenly splitting the page is given secondary importance.
+	 */
+#ifdef AUTHORITATIVE_COMPARE_TEST
+	return _bt_leave_natts(rel, lastleft, firstright);
+#endif
+
+	result = 1;
+	for (int attnum = 1; attnum <= keysz; attnum++)
+	{
+		Datum		datum1,
+					datum2;
+		bool		isNull1,
+					isNull2;
+		Form_pg_attribute att;
+
+		datum1 = index_getattr(lastleft, attnum, itupdesc, &isNull1);
+		datum2 = index_getattr(firstright, attnum, itupdesc, &isNull2);
+		att = TupleDescAttr(itupdesc, attnum - 1);
+
+		if (isNull1 != isNull2)
+			break;
+
+		if (!isNull1 &&
+			!datumIsEqual(datum1, datum2, att->attbyval, att->attlen))
+			break;
+
+		result++;
+	}
+
+	return result;
+}
+
 /*
  *  _bt_check_natts() -- Verify tuple has expected number of attributes.
  *
diff --git a/src/include/access/nbtree.h b/src/include/access/nbtree.h
index b9d82f6dfc..1e9869b30e 100644
--- a/src/include/access/nbtree.h
+++ b/src/include/access/nbtree.h
@@ -143,11 +143,15 @@ typedef struct BTMetaPageData
  * For pages above the leaf level, we use a fixed 70% fillfactor.
  * The fillfactor is applied during index build and when splitting
  * a rightmost page; when splitting non-rightmost pages we try to
- * divide the data equally.
+ * divide the data equally.  When splitting a page that's entirely
+ * filled with a single value (duplicates), the leaf-page
+ * fillfactor is overridden, and is applied regardless of whether
+ * the page is a rightmost page.
  */
 #define BTREE_MIN_FILLFACTOR		10
 #define BTREE_DEFAULT_FILLFACTOR	90
 #define BTREE_NONLEAF_FILLFACTOR	70
+#define BTREE_SINGLEVAL_FILLFACTOR	1
 
 /*
  *	In general, the btree code tries to localize its knowledge about
@@ -693,6 +697,8 @@ extern bool btproperty(Oid index_oid, int attno,
 		   bool *res, bool *isnull);
 extern IndexTuple _bt_suffix_truncate(Relation rel, IndexTuple lastleft,
 					IndexTuple firstright);
+extern int _bt_leave_natts_fast(Relation rel, IndexTuple lastleft,
+					 IndexTuple firstright);
 extern bool _bt_check_natts(Relation rel, Page page, OffsetNumber offnum);
 extern void _bt_check_third_page(Relation rel, Relation heap, Page page,
 					 Size itemsz);
-- 
2.17.1

