Thread: Scaling with lazy index updates

Pg Performers,

This might be a out of the ordinary question, or perhaps I have been out
of the loop for a while but does PostgreSQL (or any other database) have
support for lazy index updates.  What I mean by lazy index updates is
index updating which occur at a regular interval rather than per
transaction.

I have found that inserts and updates tend to slow down when the database
gets really big.  I think it is likely an effect of updating indexes when
the insert or update occurs.

Looking forward to feedback and possibly direction on my lazy index update
question.

TIA,

Fred

Howdy Fred,

> This might be a out of the ordinary question, or perhaps I have been out
> of the loop for a while but does PostgreSQL (or any other database) have
> support for lazy index updates.  What I mean by lazy index updates is
> index updating which occur at a regular interval rather than per
> transaction.

In a word: No.

The issue with "asynchronous index updates" (which is what you asked about) is
that they don't work with the way PostgreSQL uses indexes.   If the index
hasn't been updated, then when a query uses an index scan the row simply
wouldn't show up.    If that's acceptable behavior for you, then perhaps you
could consider asynchronous *table* updates, done at the application layer,
which would be much easier to implement.

We do as much as we can by offloading b-tree "cleanup" for indexes until
VACUUM/REINDEX, which is called manually.

Hmmm.   Can you think of an example of an RDBMS which does *not* update
indexes immediately (and transactionally)?  I can't.

--
-Josh Berkus
 Aglio Database Solutions
 San Francisco