Re: large dataset with write vs read clients - Mailing list pgsql-performance

From Chris Browne
Subject Re: large dataset with write vs read clients
Date
Msg-id 87aamj9xea.fsf@cbbrowne.afilias-int.info
Whole thread Raw
In response to large dataset with write vs read clients  (Aaron Turner <synfinatic@gmail.com>)
List pgsql-performance
mladen.gogala@vmsinfo.com (Mladen Gogala) writes:
> I have a logical problem with asynchronous commit. The "commit"
> command should instruct the database to make the outcome of the
> transaction permanent. The application should wait to see whether the
> commit was successful or not. Asynchronous behavior in the commit
> statement breaks the ACID rules and should not be used in a RDBMS
> system. If you don't need ACID, you may not need RDBMS at all. You may
> try with MongoDB. MongoDB is web scale:
> http://www.youtube.com/watch?v=b2F-DItXtZs

The "client" always has the option of connecting to a set of databases,
and stowing parts of the data hither and thither.  That often leads to
the relaxation called "BASE."  (And IBM has been selling that relaxation
as MQ-Series since the early '90s!)

There often *ARE* cases where it is acceptable for some of the data to
not be as durable, because that data is readily reconstructed.  This is
particularly common for calculated/cached/aggregated data.

Many things can get relaxed for a "data warehouse" data store, where the
database is not authoritative, but rather aggregates data drawn from
other authoritative sources.  In such applications, neither the A, C, I,
nor the D are pointedly crucial, in the DW data store.

- We don't put the original foreign key constraints into the DW
  database; they don't need to be enforced a second time.  Ditto for
  constraints of all sorts.

- Batching of the loading of updates is likely to break several of the
  letters.  And I find it *quite* acceptable to lose "D" if the data may
  be safely reloaded into the DW database.

I don't think this is either cavalier nor that it points to "MongoDB is
web scale."
--
"cbbrowne","@","gmail.com"
Rules  of the  Evil Overlord  #181.  "I  will decree  that all  hay be
shipped in tightly-packed bales. Any wagonload of loose hay attempting
to pass through a checkpoint will be set on fire."

pgsql-performance by date:

Previous
From: Chris Browne
Date:
Subject: Re: large dataset with write vs read clients
Next
From: Dan Harris
Date:
Subject: Re: Slow count(*) again...