Re: dynamically allocating chunks from shared memory - Mailing list pgsql-hackers

From Markus Wanner
Subject Re: dynamically allocating chunks from shared memory
Date
Msg-id 4C48977D.3080702@bluegap.ch
Whole thread Raw
In response to Re: dynamically allocating chunks from shared memory  (Alvaro Herrera <alvherre@commandprompt.com>)
Responses Re: dynamically allocating chunks from shared memory
List pgsql-hackers
Hi,

On 07/22/2010 08:31 PM, Alvaro Herrera wrote:
> FWIW I don't think you should be thinking in "replacing imessages with
> SLRU".  I rather think you should be thinking in how can you implement
> the imessages API on top of SLRU.

Well, I'm rather comparing SLRU with the dynamic allocator. So far I'm 
unconvinced that SLRU would be a better base for imessages than a 
dynamic allocator. (And I'm arguing that SLRU should use a dynamic 
allocator underneath).

> So as far as the coordinator and
> background worker are concerned, there wouldn't be any difference --
> they keep using the same API they are using today.

Agreed, the imessages API to the upper layer doesn't need to care about 
the underlying stuff.

> Also let me repeat my earlier comment about imessages being more similar
> to multixact than to notify.  The content of each multixact entry is
> just an arbitrary amount of bytes.  If imessages are numbered from a
> monotonically increasing sequence,

Well, there's absolutely no need to serialize imessages. So they don't 
currently carry any such number. And opposed to multixact entries, they 
are clearly directed at exactly one single consumer. Every consumer has 
its own receive queue. Sending messages concurrently to different 
recipients may happen completely parallelized, without any (b)locking in 
between.

The dynamic allocator is the only part of the chain which might need to 
do some locking to protect the shared resource (memory) against 
concurrent access. Note, however, that wamalloc (as any modern dynamic 
allocator) is parallelized to some extent, i.e. concurrent malloc/free 
calls don't necessarily need to block each other.

> it should be possible to use a very
> similar technique, and perhaps you should be able to reduce locking
> requirements as well (write messages with only a shared lock, after
> you've determined and reserved the area you're going to write).

Writing to the message is currently (i.e. imessages-on-dynshmem) done 
without *any* kind of lock held. So that would rather increase locking 
requirements and lower parallelism, I fear.

Regards

Markus


pgsql-hackers by date:

Previous
From: Robert Haas
Date:
Subject: Re: Copy path in Dynamic programming
Next
From: David Fetter
Date:
Subject: Re: patch: to_string, to_array functions