Re: Again, sorry, caching. - Mailing list pgsql-hackers

On a side note, is it possible that we could add the "iscachable" which
for now, would give cache bias?  That is, allow for a mechanism to
indicate that the pages that are required for this query will be
frequently needed.  I've not looked at the buffer cache implementation.
Is it possible to somehow weight the corresponding pages in the cache so
that it works something like this?  For each query that does not use the
biased pages ages the biased pages.  Once the age threshold has been
hit, allow the pages to be flushed per normal page replacement
strategy.  Every time the biased pages get a hit, renew the bias on the
pages.

I'm not sure this holds water but I'm thinking it would at least help
insure that the pages in question are quickly available without having
to constantly re-read them from disk.

What ya think?  Cache already work like this?  Doable?


On Tue, 2002-03-19 at 07:17, Jeff Davis wrote:
> > Yes...I was thinking that a generic library interface with a nice design
> > pattern might meet this need rather well.  Done properly, I think we can
> > make it where all that, more or less, would be needed is application
> > hooks which accept the result set to be cached and a mechanism to signal
> > invalidation of the current cache....obviously that's not an exhaustive
> > list... :)
>
> A library implies that the application is running long enough to actually
> hear the notofication. Web apps start up, read from the database, and before
> any cache is needed they're done and the next one starts up, reading again
> from the database. Only currently open connections receive the notification.

I think you misunderstood me.  My intension was the creation of a
pattern library, whereby, creation of your "layer", without regard for
the implementation requirements, can more easily be implemented.  In
other words, every time you need to implement this "layer" for various
applications which address various problem domains, the library would
serve as the heart of it reducing the amount of common code that would
otherwise ave to be put in place in one form or another.  Thus my
reference to a design pattern.

Should also be noted that some fast and slick web applications
architecture often support some form of context shared persistence which
would allow for caching to be implementing even in web application
space.

>
> I think that you do need an entire layer... but that's not a bad thing
> necessarily. Have a daemon that stays connected for a long time and when a
> notification arrives, rewrite the cache (or mark it dirty). Other
> applications can read data from static files or shared memory, or even
> another communication socket with your daemon.

Exactly...all of which, I'm thinking, can be encompassed within a design
pattern, greatly reducing the effort required for a new "layer"
application requirement.

>
> There may be some way around running a daemon, so if you have a better
> solution please let me know.

I hadn't spent enough time thinking out it.  My initial thought was to
provide the support functionality and let the coder determine his own
route to achieve his goal.  Does he need a new daemon or can it be built
into his application?  This is why a library is appealing.

>
> I think I am in favor of client caching in general, but "mlw" (sorry, I can't
> find your real name in the emails at hand) makes some good points. The most
> important one is that we don't want to change application architectures on
> everyone. It's easy if you just have to add "iscachable" to a query, it's
> hard if you have to start writing against a different set of routines (to
> grab from your client cache rather than a database).

Yes.  I completely agree with that.

>
> However, I am perfectly happy writing a client-side cache or using temp
> tables to store a result set. I also don't care that much if someone chooses
> PostgreSQL for their website (unless I'm responsible for it's success in some
> way :) ). That's me personally, if you want to attract more users from mysql,
> "iscachable" is very likely an attractive feature.

And/or provide language bindings to a caching library which greatly
helps facilitate this.  Granted, "iscachable" concept is certainly a
powerful concept.

Greg