Re: What is the posix_memalign() equivalent for the PostgreSQL? - Mailing list pgsql-hackers
| From | Anderson Carniel |
|---|---|
| Subject | Re: What is the posix_memalign() equivalent for the PostgreSQL? |
| Date | |
| Msg-id | CAAAXyN1+Viz1QYmGwjT=MADWeqH_9PM3Df953A6jPADEmp-1yA@mail.gmail.com Whole thread Raw |
| In response to | Re: What is the posix_memalign() equivalent for the PostgreSQL? (Craig Ringer <craig@2ndquadrant.com>) |
| Responses |
Re: What is the posix_memalign() equivalent for the PostgreSQL?
(Tom Lane <tgl@sss.pgh.pa.us>)
|
| List | pgsql-hackers |
The unexpected errors are problems to free the memory that was previously allocated with posix_memalign. After a number of memory allocation and free calls, the next free call crashes the system.
On the other hand, if I replace the posix_memalign and the free calls for the following malloc/free aligned implementations, it works.
void *malloc_aligned(size_t alignment, size_t bytes) {
void *mallocPtr; //Initial pointer returned from malloc
void *newMallocPtr; //New pointer after adjustment
void *alignedPtr;
size_t alignMask; //Need this to get the aligned address
size_t totalBytes = 0;
/* Make sure alignment is power of 2 and it is not zero
* because zero is not power of 2 */
if ( !(!(alignment & (alignment-1)) && alignment) )
return NULL;
/* We need to allocate extra memory to make sure the allocated
* memory will be aligned and need sizeof(size_t) bytes more for
* storing the value of the bytes we padded.
*/
totalBytes = bytes + alignment + sizeof(size_t);
mallocPtr = palloc(totalBytes);
if (NULL == mallocPtr)
return NULL;
newMallocPtr = (void*)((char*)mallocPtr + sizeof(size_t));
alignMask = ~(alignment - 1);
/* Value of alignedPtr should be multiple of alignment */
alignedPtr = (void *)(((size_t)newMallocPtr + alignment) & alignMask);
/* Store the extra bytes info right before alignedPtr */
*((size_t*)alignedPtr - 1) = (size_t)alignedPtr - (size_t)mallocPtr;
return alignedPtr;
}
void free_aligned(void *raw_data) {
void *mallocPtr; //Initial malloc pointer
size_t extraBytes;
if (NULL == raw_data)
return;
/* Retrieve the extra padded byte info */
extraBytes = *((size_t*)raw_data - 1);
/* Get initial malloc ptr */
mallocPtr = (void*) ((size_t)raw_data - extraBytes);
pfree(mallocPtr);
}
Please note that I am using in these functions, the palloc and pfree instead of malloc and free respectively. But the problem is that the free_aligned function is not indeed freeing the allocated memory. Thus, I would like to know if the PostgreSQL provides a memory function that allocates aligned memory. If not, according to your experience, is there a significance difference between the performance of the O_DIRECT or not?
Thank you,
Anderson
2016-09-02 7:24 GMT-03:00 Craig Ringer <craig@2ndquadrant.com>:
On 2 September 2016 at 01:12, Anderson Carniel <accarniel@gmail.com> wrote:
> Dear all,
>
> I am developing an extension for the PostgreSQL that write/read some
> external files from the PostgreSQL. In order to write/read, I am using the
> O_DIRECT flag and using the posix_memalign to allocate memory. I would like
> to know if the postgresql internal library provides an equivalent function
> for the posix_memalign since I am getting unexpected errors.
"unexpected errors". Details please?
If you're trying to allocate aligned memory, I believe PostgreSQL
typically uses the TYPEALIGN macros (see c.h) but I'm painfully
clueless in the area, so ... yeah. Don't trust me.
I was a bit surprised not to see a MemoryContextAlloc or palloc
variant that returns memory aligned to a given boundary.
> All my
> allocations are in the TopMemoryContext since I am working with several
> buffers that must be alive while the PostgreSQL Server is activated.
You can't posix_memalign into TopMemoryContext. Such memory is outside
the memory context system, like memory directly malloc()'d.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
pgsql-hackers by date: