Hi,
[ CC'ing to the postgres-r mailing list ]
Mark Mielke wrote:
> On 08/12/2009 12:04 PM, Suno Ano wrote:
>> can anybody tell me, is there a roadmap with regards to
>> http://www.postgres-r.org ?
I'm glad you're asking.
>> I would love to see it become production-ready asap.
Yes, me too. Do you have some spare cycles to spend? I'd be happy to
help you getting started. However, I have a 16 days old daughter at
home, so please don't expect response times under a few days ;-)
> Even a breakdown of what is left to do might be useful in case any of us
> want to pick at it. :-)
The TODO file from the patch is a good place to start from. For the sake
of simplicity, I've attached it.
I've written a series of posts covering various topics of Postgres-R
about a year ago. Here are the links, following the discussions
down-thread might be interesting as well.
Postgres-R: current state of development, Jul 15 2008:
http://archives.postgresql.org/pgsql-hackers/2008-07/msg00735.php
Postgres-R: primary key patches, Jul 16 2008
http://archives.postgresql.org/pgsql-hackers/2008-07/msg00777.php
Postgres-R: tuple serialization, Jul 22 2008
http://archives.postgresql.org/pgsql-hackers/2008-07/msg00969.php
Postgres-R: internal messaging, Jul 23 2008
http://archives.postgresql.org/pgsql-hackers/2008-07/msg01051.php
Some random updates to last year's "current state of development" that
come to mind:
* I've adjusted to the signaling to use the signal
multiplexer code that recently landed on HEAD.
* Work on the initialization and recovery stuff is
progressing slowly, but steadily.
* The tuple serialization code is being refactored ATM
to get a lot smaller and easier to understand and
debug.
That should get you an impression on the current state of development, I
think. Please feel free to ask more specific questions.
Regards
Markus Wanner
P.S.: Sanu, did you note the addition of the link to the Postgres-R
mailing list, which you pointed out was hard to find?
URGENT
======
* Implement parsing of the replication_gcs GUC for spread and ensemble.
* check for places there replication_enabled should be checked more
extensively.
complaint about select() not interrupted by signals:
http://archives.postgresql.org/pgsql-hackers/2008-12/msg00448.php
restartable signals 'n all that
http://archives.postgresql.org/pgsql-hackers/2007-07/msg00003.php
3.2.1 Internal Message Passing
==============================
* Maybe send IMSGT_READY after some other commands, not only after
IMSGT_CHANGESET. Remember that local transactions also have to send an
IMSGT_READY, so that their proc->coid gets reset.
* Make sure the coordinator copes with killed backends (local as
well as remote ones).
* Check if we can use pselect to avoid race conditions with IMessage stuff
within the coordinator's main loop.
* Check error conditions such as out of memory and out of disk space. Those
could prevent a single node from applying a remote transaction. What to
do in such cases? A similar one is "limit of queued remote transactions
reached".
3.2.2 Communication with the Postmaster
=======================================
* Get rid of the SIGHUP signal (was IMSGT_SYSTEM_READY) for the
coordinator and instead only start the coordinator as soon as the
postmaster is ready to fork helper backends. Should simplify things and
make them more similar to the current Postgres code, i.e. for the
autovacuum launcher.
* Handle restarts of the coordinator due to a crashed backend. The
postmaster already sends a signal to terminate an existing
coordinator process and it tries to restart one. But the coordinator
should then start recovery and only allow other backends after that.
Keep in mind that this recovery process is costly and we should somehow
prevent nodes which fail repeatedly from endlessly consuming resources
of the complete cluster.
* The backends need to report errors from remote *and* local transactions
to the coordinator. Worker backends erroring out while waitin for
changesets are critical. Erroring out due to serialization failure is fine,
we can simply ignore the changeset, once it arrives late on. But other
errors are probably pretty bad at that stage. Upon crashes, the postmaster
restarts all backends and the coordinator anyway, so the backend
process itself can take care of informing the coordinator via
imessages.
* Think about a newly requested helper backend crashing before it
registers with the coordinator. That would prevent requesting any further
helper backend.
3.2.3 Group Communication System Issues
=======================================
* Drop the static receive buffers of the GCS interfaces in favor of a
dynamic one. It's much easier to handle.
* Hot swapping of the underlying GCS of a replicated database is currently
not supported. It would involve waiting for all nodes of the group to
have joined the new group, then swap.
If we enforce the GCS group name to equal the database name, that's
needed for renaming a replicated database. Might be a good reason
against that rule.
* Better error reporting to the client in case of GCS errors. There are
three phases: connecting, initialization and joining the group. All those
can potentionally fail. Currently, an ALTER DATABASE waits until it gets
a DB_STATE_CHANGE, for which it waits forever, if something fails.
3.3.1 Group Communication Services
==================================
* Prevent EGCS from sending an initial view which does not include the
local node.
* Complete support for Spread
* Support for Appia
3.3.2 Global Object Identifiers
===============================
* Use a naming service translating local OIDs to global ids, so that we
don't have to send the full schema and table name every time.
3.3.3 Global Transaction Identifiers
====================================
* Drop COIDs in favor of GIDs
3.4 Collection of Transactional Data Changes
============================================
* Make sure we correctly serialize transactions, which modify tuples that
are referenced by a foreign key. An insert or update to a tuple with a
reference to somewhere must make sure the referenced tuple didn't change.
(The other way around should be covered automatically by the changeset,
because it also catches changes by the ON UPDATE or ON DELETE hooks of
the affected foreign key.)
Write tests for that behaviour.
* Think about removing these additional members of the EState:
es_allLocksGranted, es_tupleChangeApplied and es_loopCounter. Those can
certainly be simplified.
* Take care of a correct READ COMMITTED mode, which requires changes of a
committed transaction to be visible immediately to all other concurrently
running transactions. This might be very similar to a fully synchronous,
lock based replication mode. This certainly introduces higher commit
latency.
* Add the schema name to the changeset and seq_increment messages to fully
support namespaces.
* Support for savepoints requires communicating additional sub-transaction
states.
3.6 Application of Change Sets
==============================
* Possibly use heap_{insert,update,delete} directly, instead of going
through ExecInsert, ExecUpdate and ExecDelete? That could save us some
conditionals, but we would probably need to re-add other stuff.
* Possibly limit ExecOpenIndices() to open only the primary key index for
CMD_DELETE?
* Check if ExecInsertIndexTuples() could break due to out of sync replica
with UNIQUE constraint violations.
* Make sure the statement_timeout does not affect hepler backends.
* Prevent possible deadlocks which might occur by re-ordered (optimistic)
application of change sets from remote transactions. Just make sure the
next transaction according to the decided ordering always has a spare
helper backend available to get executed on and is not blocked by other
remote transactions which must wait for it (and thus cause deadlocking).
3.8.2 Data Definition Changes
=============================
* check which messages the coordinator must ignore, because
they could originate from backends which were running concurrently
to a STOP REPLICATION command. Such backends could possibly send
changesets and other replication requests.
* Add proper handling of CREATE / ALTER / DROP TABLE and make sure those
don't interfere with normal, parallel changeset application.
3.9 Initialization and Recovery
===============================
* helper processes connected to template databases should exit immediately
after having performed their job, so CREATE DATABASE from such a
template database works again.
3.9.1 Initialization and Recovery: Data Transfer
================================================
3.9.2 Initialization and Recovery: Schema Adaption
==================================================
* Implement schema adaption
* Make sure triggers and constraints either do only contain functions
which are available on every node _or_ execute the triggers and check the
constraints only on the machines having them (remote execution?)
3.9.5 Initialization and Recovery: Full Cluster Shutdown and Restart
====================================================================
* After a full crash (no majority running, thus stopped cluster wide
operation), we need to be able to recover from the distributed,
permanent storage into a consistent state. This requires nodes
communicating their recently committed transactions, which didn't
make it to the other nodes before the crash.
Cleanup
=======
* merge repl_database_info::state into the group::nodes->state, and the
main_state into main_group::nodes->state. Add a simpler routine to
retrieve the local node.
* Cleanup the "node_id_self_ref" mess. The GCS should not be able to send
a viewchange to the coordinator, which does not include the local node
itself. In that sense, maybe "nodes" doesn't need to include the local
node?
* Reduce the amount of elog(DEBUG...) to a usefull level. Currently mainly
DEBUG3 is used, sometimes DEBUG5. Maybe also rethink the precompiler
flags which enable or disable this verbose debugging.
* At the moment, exec_simple_query is exported to the replication code,
where in stock Postgres, that call is static.
* Same applies for ExecInsert, which is no longer static, but also used
in the recovery code. However, that should be mixed into
ExecProcessCollection() to reduce code duplication anyway.
* Consistently name the backends 'worker' and 'helper' backends?
* Never call cset_process() from worker backends! Fix the comment above
that function.
* The recovery subscriber currently issues a CREATE DATABASE from within
a transaction block. That's unclean.
* The database encoding is transferred as a number, not string. Not sure
if that matters.
