Home > mailing lists

Re: Initializing LWLock Array from Server Code - Mailing list pgsql-hackers

From	Souvik Bhattacherjee
Subject	Re: Initializing LWLock Array from Server Code
Date	May 1, 2019 00:52:15
Msg-id	CAANrPSdEu-bkuxOypPvNcWD-7ASa4L-s-yMAeJ3KqfJGHJKOvw@mail.gmail.com Whole thread Raw
In response to	Initializing LWLock Array from Server Code (Souvik Bhattacherjee <kivuosb@gmail.com>)
Responses	Re: Initializing LWLock Array from Server Code
List	pgsql-hackers

Tree view

Hi Robert,

Thank you for your reply and sorry that I couldn't reply earlier.

Since I didn't get any response within a couple of days, I took the longer route -- changed the lwlock.h and lwlock.c

for accommodating the lw locks for the shared hash table.

I'll describe what I modified in the lwlock.h and lwlock.c and it seems to be working fine. Although I haven't got an opportunity

to test it extensively. If you could let me know if I missed out anything that might cause problems later that would be great.

In lwlock.h, the following modifications were made:

/* the number of partitions of the shared hash table */

#define NUM_FILTERHASH_PARTITIONS 64

/* Offsets for various chunks of preallocated lwlocks. */
#define BUFFER_MAPPING_LWLOCK_OFFSET    NUM_INDIVIDUAL_LWLOCKS
#define LOCK_MANAGER_LWLOCK_OFFSET        \
    (BUFFER_MAPPING_LWLOCK_OFFSET + NUM_BUFFER_PARTITIONS)
#define PREDICATELOCK_MANAGER_LWLOCK_OFFSET \
    (LOCK_MANAGER_LWLOCK_OFFSET + NUM_LOCK_PARTITIONS)

/* offset for shared filterhash lwlocks in the MainLWLockArray */

#define FILTERHASH_OFFSET \
(PREDICATELOCK_MANAGER_LWLOCK_OFFSET + NUM_PREDICATELOCK_PARTITIONS)

/* modified NUM_FIXED_LOCKS */

#define NUM_FIXED_LWLOCKS \
(FILTERHASH_OFFSET + NUM_FILTERHASH_PARTITIONS)

/* added LWTRANCHE_FILTERHASH in the BuiltinTrancheIds */

typedef enum BuiltinTrancheIds
{
    LWTRANCHE_CLOG_BUFFERS = NUM_INDIVIDUAL_LWLOCKS,
    LWTRANCHE_COMMITTS_BUFFERS,
    LWTRANCHE_SUBTRANS_BUFFERS,
    LWTRANCHE_MXACTOFFSET_BUFFERS,
    LWTRANCHE_MXACTMEMBER_BUFFERS,
    LWTRANCHE_ASYNC_BUFFERS,
    LWTRANCHE_OLDSERXID_BUFFERS,
    LWTRANCHE_WAL_INSERT,
    LWTRANCHE_BUFFER_CONTENT,
    LWTRANCHE_BUFFER_IO_IN_PROGRESS,
    LWTRANCHE_REPLICATION_ORIGIN,
    LWTRANCHE_REPLICATION_SLOT_IO_IN_PROGRESS,
    LWTRANCHE_PROC,
    LWTRANCHE_BUFFER_MAPPING,
    LWTRANCHE_LOCK_MANAGER,
    LWTRANCHE_PREDICATE_LOCK_MANAGER,
    LWTRANCHE_FILTERHASH,
    LWTRANCHE_PARALLEL_HASH_JOIN,
    LWTRANCHE_PARALLEL_QUERY_DSA,
    LWTRANCHE_SESSION_DSA,
    LWTRANCHE_SESSION_RECORD_TABLE,
    LWTRANCHE_SESSION_TYPMOD_TABLE,
    LWTRANCHE_SHARED_TUPLESTORE,
    LWTRANCHE_TBM,
    LWTRANCHE_PARALLEL_APPEND,
    LWTRANCHE_FIRST_USER_DEFINED
}            BuiltinTrancheIds;

In lwlock.c, the following modifications were made:

In function, static void InitializeLWLocks(void), the following lines were added:

/* Initialize filterhash LWLocks in main array */
    lock = MainLWLockArray + NUM_INDIVIDUAL_LWLOCKS +
        NUM_BUFFER_PARTITIONS + NUM_LOCK_PARTITIONS + NUM_FILTERHASH_PARTITIONS;
    for (id = 0; id < NUM_FILTERHASH_PARTITIONS; id++, lock++)
        LWLockInitialize(&lock->lock, LWTRANCHE_FILTERHASH);

In function void RegisterLWLockTranches(void), the following line was added:

LWLockRegisterTranche(LWTRANCHE_FILTERHASH, "filter_hash");

All of this was done after allocating the required of space in the shared memory.

Thanks,

-SB

On Mon, Apr 29, 2019 at 1:59 PM Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Apr 26, 2019 at 2:58 PM Souvik Bhattacherjee <kivuosb@gmail.com> wrote:
> I have created a shared hash table in partitioned mode inside the postgres server code. In order to guard the partitions, I'm trying to initialize an array of LWLocks. The code that I'm trying to use for that is
>
> void RequestNamedLWLockTranche(const char *tranche_name, int num_lwlocks);
> LWLockPadded *GetNamedLWLockTranche(const char *tranche_name);
>
> I'm not sure where exactly should this code be called from the server code. So I had placed it in
>
> void CreateSharedMemoryAndSemaphores(bool makePrivate, int port);
>
> within ipic.c. However, I'm getting the following error message when starting the database:
>
> FATAL: requested tranche is not registered
>
> So at this point, I'm a little confused as to where the methods should be called from inside the server code. Any pointers would be appreciated.

RequestNamedLWLockTranche() changes the behavior of LWLockShmemSize()
and CreateLWLocks(), so must be called before either of those.
GetNamedLWLockTranche() must be called after CreateLWLocks().

This machinery works for pg_stat_statements, so see that as an example
of how to make this work from an extension. If you're hacking the
core server code, then look at the places where the corresponding bits
of pg_stat_statements code get called. IIRC, _PG_init() gets called
from process_shared_preload_libraries(), so you might look at the
placement of the calls to that function.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

pgsql-hackers by date:

From: Tom Lane
Date: 01 May 2019, 00:36:55
Subject: Re: Calling PrepareTempTablespaces in BufFileCreateTemp

From: Andres Freund
Date: 01 May 2019, 01:10:45
Subject: Re: REINDEX INDEX results in a crash for an index of pg_class since9.6

Re: Initializing LWLock Array from Server Code - Mailing list pgsql-hackers

Previous

Next