Re: Question about caching on full table scans - Mailing list pgsql-performance

From Markus Innerebner
Subject Re: Question about caching on full table scans
Date
Msg-id 7FB562B8-75CB-446A-B69F-6D999D171710@inf.unibz.it
Whole thread Raw
In response to Re: Question about caching on full table scans  ("Albe Laurenz" <laurenz.albe@wien.gv.at>)
Responses Re: Question about caching on full table scans  (Jeff Janes <jeff.janes@gmail.com>)
Re: Question about caching on full table scans  (Scott Marlowe <scott.marlowe@gmail.com>)
List pgsql-performance
Hi Laurenz,


In your approach 1 to 3, what do you mean with "load into main memory"?


I forgot to say: I use Java and connect with JDBC.

in approach 1 I do an initial loading of the entire relation, by executing 1 SQL query to load all edges in main memory, where I create my main memory structure
as an adjacency list.

Do you
a) make sure that the data you talk about are in the PostgreSQL buffer
cache
or

b) retrieve the data from PostgreSQL and store it somewhere in your
application?

In approach 1 I do that, as described before.

But after each experiment I restart a new java process.



To clear PostgreSQL's cache, restart the server.
That should be a fast operation.
Since version 8.3, PostgreSQL is smart enough not to evict the
whole cache for a large sequential scan.



To flush the filesystem cache (from Linux 2.6.16 on), use
sync; echo 3 > /proc/sys/vm/drop_caches

I started to do that , and 
yes, this solves my problem!!

I assume that deleting file system cache implies that also postgres cache is deleted, isn't it ?

so i will invoke after each experiment this command

thanks a lot!!

Markus

pgsql-performance by date:

Previous
From: "Albe Laurenz"
Date:
Subject: Re: Question about caching on full table scans
Next
From: Jeff Janes
Date:
Subject: Re: Question about caching on full table scans