> It is not meant to be a full implementation of application level queuing
> system though but just the capture, persisting and distribution parts
>
> Using this as an "application level queue" needs a set of interface
> functions to extract the events and also to keep track of the processed
> events. As there is no general consensus what these shoul be (like if
> processing same event twice is allowed) this part is left for specific
> queue consumer implementations.
Well, but AFAICT, you've already prohibited features through your design
which are essential to application-level queues, and are implemented by,
for example, pgQ.
1. your design only allows the queue to be read on replicas, not on the
node where the item was inserted.
2. if you can't UPDATE or DELETE queue items -- or LOCK them -- how on
earth would a client know which items they have executed and which they
haven't?
3. Double-down on #2 in a multithreaded environment.
For an application-level queue, the base functionality is:
ADD ITEM
READ NEXT (#) ITEM(S)
LOCK ITEM
DELETE ITEM
More sophisticated an useful queues also allow:
READ NEXT UNLOCKED ITEM
LOCK NEXT UNLOCKED ITEM
UPDATE ITEM
READ NEXT (#) UNSEEN ITEM(S)
The design you describe seems to prohibit pretty much all of the above
operations after READ NEXT. This makes it completely useless as a
application-level queue.
And, for that matter, if your new queue only accepts INSERTs, why not
just improve LISTEN/NOTIFY so that it's readable on replicas? What does
this design buy you that that doesn't?
Quite possibly you have plans which answer all of the above, but they
aren't at all clear in your RFC.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
