RE: Copy data to DSA area - Mailing list pgsql-hackers

From Ideriha, Takeshi
Subject RE: Copy data to DSA area
Date
Msg-id 4E72940DA2BF16479384A86D54D0988A7DB4FA55@G01JPEXMBKW04
Whole thread Raw
In response to RE: Copy data to DSA area  ("Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>)
Responses Re: Copy data to DSA area  (Thomas Munro <thomas.munro@gmail.com>)
List pgsql-hackers
Hi, 

>From: Ideriha, Takeshi [mailto:ideriha.takeshi@jp.fujitsu.com]
>Sent: Friday, April 26, 2019 11:50 PM
>Well, after developing PoC, I realized that this PoC doesn't solve the local process is
>crashed before the context becomes shared because local process keeps track of
>pointer to chunks.
>Maybe all of you have already noticed and pointed out this case :) So it needs another
>work but this poc is a good step for me to advance more.

I think the point to prevent memory leak is allocating memory and storing its 
address into a structure at the same time. This structure should be trackable from
other process. 

So I'm going to change the design. I'll allocate a buffer of pointers to 
dsa objects which are not permanent yet. This buffer is located on shared 
memory. For simplicity, the buffer is allocated per process. That is the 
number of backends equals to MaxBackends. 

Developer calls API to make objects permanent. If objects become permanent, 
corresponding pointers are deleted from the buffer. If they fail to become 
permanent, they are freed from shared memory by checking the buffer and 
corresponding pointers also deleted from the buffer. 

There are two cases of failure: one is transaction abort, the other is process
crash. If transaction aborts, that process itself has responsibility to free
the objects. The process free the objects. In case of process crash, another 
backend needs to take care of it. I'm assuming on_shmem_exit callback can be 
used to free objects by postmaster. 

Regarding MemoryContext, one Shared MemoryContext is on the shared memory
and each chunk has a back pointer to it to pfree and so on.

I'm going to make a new patch and send it later.

Regards,
Takeshi Ideriha





pgsql-hackers by date:

Previous
From: Thomas Munro
Date:
Subject: Re: We're leaking predicate locks in HEAD
Next
From: Thomas Munro
Date:
Subject: Re: Copy data to DSA area