[HACKERS] Help needed regarding DSA based in-memory index! - Mailing list pgsql-hackers

From Gaddam Sai Ram
Subject [HACKERS] Help needed regarding DSA based in-memory index!
Date
Msg-id 15f4ea99b34.e69a4e1b633.8937474039794097334@zohocorp.com
Whole thread Raw
List pgsql-hackers
Hi Munro,
      Thanks for cautioning us about possible memory leaks(during error cases) incase of long-lived DSA segements(have a look in below thread for more details).


      Actually we are following an approach to avoid this DSA memory leaks. Let me explain our implementation and please validate and correct us in-case we       miss anything.

      Implementation:
      
      Basically we have to put our index data into memory (Index Column Value Vs Ctid) which we get in aminsert callback function.
      
      Coming to the implementation, in aminsert Callback function, 
  • We Switch to CurTransactionContext 
  • Cache the DMLs of a transaction into dlist(global per process)
  • Even if different clients work parallel, it won't be a problem because every client gets one dlist in separate process and it'll have it's own CurTransactionContext
  • We have registered transaction callback (using RegisterXactCallback() function). And during event pre-commit(XACT_EVENT_PRE_COMMIT), we populate all the transaction specific DMLs (from dlist) into our in-memory index(DSA) obviously inside PG_TRY/PG_CATCH block.
  • In case we got some errors(because of dsa_allocate() or something else) while processing dlist(while populating in-memory index), we cleanup the DSA memory in PG_CATCH block that is allocated/used till that point.
  • During other error cases, typically transactions gets aborted and PRE_COMMIT event is not called and hence we don't touch DSA at that time. Hence no need to bother about leaks.
  • Even sub transaction case is handled with sub transaction callbacks.
  • CurTransactionContext(dlist basically) is automatically cleared after that particular transaction.

I want to know if this approach is good and works well in all cases. Kindly provide your feedback on this.

Regards
G. Sai Ram

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: [HACKERS] Remove secondary checkpoint
Next
From: Andres Freund
Date:
Subject: Re: [HACKERS] Remove secondary checkpoint