Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index. - Mailing list pgsql-hackers

On Thu, Aug 29, 2019 at 5:07 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I agree that v9 might be ever so slightly more space efficient than v5
> was, on balance.

I see some Valgrind errors on v9, all of which look like the following
two sample errors I go into below.

First one:

==11193== VALGRINDERROR-BEGIN
==11193== Unaddressable byte(s) found during client check request
==11193==    at 0x4C0E03: PageAddItemExtended (bufpage.c:332)
==11193==    by 0x20F6C3: _bt_split (nbtinsert.c:1643)
==11193==    by 0x20F6C3: _bt_insertonpg (nbtinsert.c:1206)
==11193==    by 0x21239B: _bt_doinsert (nbtinsert.c:306)
==11193==    by 0x2150EE: btinsert (nbtree.c:207)
==11193==    by 0x20D63A: index_insert (indexam.c:186)
==11193==    by 0x36B7F2: ExecInsertIndexTuples (execIndexing.c:393)
==11193==    by 0x391793: ExecInsert (nodeModifyTable.c:593)
==11193==    by 0x3924DC: ExecModifyTable (nodeModifyTable.c:2219)
==11193==    by 0x37306D: ExecProcNodeFirst (execProcnode.c:445)
==11193==    by 0x36C738: ExecProcNode (executor.h:240)
==11193==    by 0x36C738: ExecutePlan (execMain.c:1648)
==11193==    by 0x36C738: standard_ExecutorRun (execMain.c:365)
==11193==    by 0x36C7DD: ExecutorRun (execMain.c:309)
==11193==    by 0x4CC41A: ProcessQuery (pquery.c:161)
==11193==    by 0x4CC5EB: PortalRunMulti (pquery.c:1283)
==11193==    by 0x4CD31C: PortalRun (pquery.c:796)
==11193==    by 0x4C8EFC: exec_simple_query (postgres.c:1231)
==11193==    by 0x4C9EE0: PostgresMain (postgres.c:4256)
==11193==    by 0x453650: BackendRun (postmaster.c:4446)
==11193==    by 0x453650: BackendStartup (postmaster.c:4137)
==11193==    by 0x453650: ServerLoop (postmaster.c:1704)
==11193==    by 0x454CAC: PostmasterMain (postmaster.c:1377)
==11193==    by 0x3B85A1: main (main.c:210)
==11193==  Address 0x9c11350 is 0 bytes after a recently re-allocated
block of size 8,192 alloc'd
==11193==    at 0x4C2FB0F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11193==    by 0x61085A: AllocSetAlloc (aset.c:914)
==11193==    by 0x617AD8: palloc (mcxt.c:938)
==11193==    by 0x21A829: _bt_mkscankey (nbtutils.c:107)
==11193==    by 0x2118F3: _bt_doinsert (nbtinsert.c:93)
==11193==    by 0x2150EE: btinsert (nbtree.c:207)
==11193==    by 0x20D63A: index_insert (indexam.c:186)
==11193==    by 0x36B7F2: ExecInsertIndexTuples (execIndexing.c:393)
==11193==    by 0x391793: ExecInsert (nodeModifyTable.c:593)
==11193==    by 0x3924DC: ExecModifyTable (nodeModifyTable.c:2219)
==11193==    by 0x37306D: ExecProcNodeFirst (execProcnode.c:445)
==11193==    by 0x36C738: ExecProcNode (executor.h:240)
==11193==    by 0x36C738: ExecutePlan (execMain.c:1648)
==11193==    by 0x36C738: standard_ExecutorRun (execMain.c:365)
==11193==    by 0x36C7DD: ExecutorRun (execMain.c:309)
==11193==    by 0x4CC41A: ProcessQuery (pquery.c:161)
==11193==    by 0x4CC5EB: PortalRunMulti (pquery.c:1283)
==11193==    by 0x4CD31C: PortalRun (pquery.c:796)
==11193==    by 0x4C8EFC: exec_simple_query (postgres.c:1231)
==11193==    by 0x4C9EE0: PostgresMain (postgres.c:4256)
==11193==    by 0x453650: BackendRun (postmaster.c:4446)
==11193==    by 0x453650: BackendStartup (postmaster.c:4137)
==11193==    by 0x453650: ServerLoop (postmaster.c:1704)
==11193==    by 0x454CAC: PostmasterMain (postmaster.c:1377)
==11193==
==11193== VALGRINDERROR-END
{
   <insert_a_suppression_name_here>
   Memcheck:User
   fun:PageAddItemExtended
   fun:_bt_split
   fun:_bt_insertonpg
   fun:_bt_doinsert
   fun:btinsert
   fun:index_insert
   fun:ExecInsertIndexTuples
   fun:ExecInsert
   fun:ExecModifyTable
   fun:ExecProcNodeFirst
   fun:ExecProcNode
   fun:ExecutePlan
   fun:standard_ExecutorRun
   fun:ExecutorRun
   fun:ProcessQuery
   fun:PortalRunMulti
   fun:PortalRun
   fun:exec_simple_query
   fun:PostgresMain
   fun:BackendRun
   fun:BackendStartup
   fun:ServerLoop
   fun:PostmasterMain
   fun:main
}

nbtinsert.c:1643 is the first PageAddItem() in _bt_split() -- the
lefthikey call.

Second one:

==11193== VALGRINDERROR-BEGIN
==11193== Invalid read of size 2
==11193==    at 0x20FDF5: _bt_insertonpg (nbtinsert.c:1126)
==11193==    by 0x21239B: _bt_doinsert (nbtinsert.c:306)
==11193==    by 0x2150EE: btinsert (nbtree.c:207)
==11193==    by 0x20D63A: index_insert (indexam.c:186)
==11193==    by 0x36B7F2: ExecInsertIndexTuples (execIndexing.c:393)
==11193==    by 0x391793: ExecInsert (nodeModifyTable.c:593)
==11193==    by 0x3924DC: ExecModifyTable (nodeModifyTable.c:2219)
==11193==    by 0x37306D: ExecProcNodeFirst (execProcnode.c:445)
==11193==    by 0x36C738: ExecProcNode (executor.h:240)
==11193==    by 0x36C738: ExecutePlan (execMain.c:1648)
==11193==    by 0x36C738: standard_ExecutorRun (execMain.c:365)
==11193==    by 0x36C7DD: ExecutorRun (execMain.c:309)
==11193==    by 0x4CC41A: ProcessQuery (pquery.c:161)
==11193==    by 0x4CC5EB: PortalRunMulti (pquery.c:1283)
==11193==    by 0x4CD31C: PortalRun (pquery.c:796)
==11193==    by 0x4C8EFC: exec_simple_query (postgres.c:1231)
==11193==    by 0x4C9EE0: PostgresMain (postgres.c:4256)
==11193==    by 0x453650: BackendRun (postmaster.c:4446)
==11193==    by 0x453650: BackendStartup (postmaster.c:4137)
==11193==    by 0x453650: ServerLoop (postmaster.c:1704)
==11193==    by 0x454CAC: PostmasterMain (postmaster.c:1377)
==11193==    by 0x3B85A1: main (main.c:210)
==11193==  Address 0x9905b90 is 11,088 bytes inside a recently
re-allocated block of size 524,288 alloc'd
==11193==    at 0x4C2FB0F: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==11193==    by 0x61085A: AllocSetAlloc (aset.c:914)
==11193==    by 0x617AD8: palloc (mcxt.c:938)
==11193==    by 0x1C5677: CopyIndexTuple (indextuple.c:508)
==11193==    by 0x20E887: _bt_compress_one_page (nbtinsert.c:2751)
==11193==    by 0x21241E: _bt_findinsertloc (nbtinsert.c:773)
==11193==    by 0x21241E: _bt_doinsert (nbtinsert.c:303)
==11193==    by 0x2150EE: btinsert (nbtree.c:207)
==11193==    by 0x20D63A: index_insert (indexam.c:186)
==11193==    by 0x36B7F2: ExecInsertIndexTuples (execIndexing.c:393)
==11193==    by 0x391793: ExecInsert (nodeModifyTable.c:593)
==11193==    by 0x3924DC: ExecModifyTable (nodeModifyTable.c:2219)
==11193==    by 0x37306D: ExecProcNodeFirst (execProcnode.c:445)
==11193==    by 0x36C738: ExecProcNode (executor.h:240)
==11193==    by 0x36C738: ExecutePlan (execMain.c:1648)
==11193==    by 0x36C738: standard_ExecutorRun (execMain.c:365)
==11193==    by 0x36C7DD: ExecutorRun (execMain.c:309)
==11193==    by 0x4CC41A: ProcessQuery (pquery.c:161)
==11193==    by 0x4CC5EB: PortalRunMulti (pquery.c:1283)
==11193==    by 0x4CD31C: PortalRun (pquery.c:796)
==11193==    by 0x4C8EFC: exec_simple_query (postgres.c:1231)
==11193==    by 0x4C9EE0: PostgresMain (postgres.c:4256)
==11193==    by 0x453650: BackendRun (postmaster.c:4446)
==11193==    by 0x453650: BackendStartup (postmaster.c:4137)
==11193==    by 0x453650: ServerLoop (postmaster.c:1704)
==11193==
==11193== VALGRINDERROR-END
{
   <insert_a_suppression_name_here>
   Memcheck:Addr2
   fun:_bt_insertonpg
   fun:_bt_doinsert
   fun:btinsert
   fun:index_insert
   fun:ExecInsertIndexTuples
   fun:ExecInsert
   fun:ExecModifyTable
   fun:ExecProcNodeFirst
   fun:ExecProcNode
   fun:ExecutePlan
   fun:standard_ExecutorRun
   fun:ExecutorRun
   fun:ProcessQuery
   fun:PortalRunMulti
   fun:PortalRun
   fun:exec_simple_query
   fun:PostgresMain
   fun:BackendRun
   fun:BackendStartup
   fun:ServerLoop
   fun:PostmasterMain
   fun:main
}

nbtinsert.c:1126 is this code from _bt_insertonpg():

            elog(DEBUG4, "dest before (%u,%u)",
                 ItemPointerGetBlockNumberNoCheck((ItemPointer) dest),
                 ItemPointerGetOffsetNumberNoCheck((ItemPointer) dest));

This is probably harmless, but it needs to be fixed.

-- 
Peter Geoghegan