Re: how to (temporarily) disable/minimize benefits of disk block cache or postgresql shared buffer - Mailing list pgsql-performance

From Rajesh Kumar Mallah
Subject Re: how to (temporarily) disable/minimize benefits of disk block cache or postgresql shared buffer
Date
Msg-id AANLkTikGhiN3aeWfF3EVlLu_WHFtAzQgZp1c9lhZYUQ_@mail.gmail.com
Whole thread Raw
In response to Re: how to (temporarily) disable/minimize benefits of disk block cache or postgresql shared buffer  (Craig Ringer <craig@postnewspapers.com.au>)
Responses Re: how to (temporarily) disable/minimize benefits of disk block cache or postgresql shared buffer
List pgsql-performance
Dear Criag,

Thanks for thinking about it.I do not understand why u feel OpenVz is weird.
at the most its not very popular. But lets not get into that debate as its not
the proper forum. From your reply i understand that there is not a easy and
clean way of doing it. Since performance related profiling requires multiple
iterations it is not feasible to reboot the machine. I think i will try to profile
my code using new and unique input parameters in each iteration, this shall
roughly serve my purpose. 

On Fri, Jul 2, 2010 at 8:30 AM, Craig Ringer <craig@postnewspapers.com.au> wrote:
On 02/07/10 01:59, Rajesh Kumar Mallah wrote:

> I had set it to 128kb
> it does not really work , i even tried your next suggestion. I am in
> virtualized
> environment particularly OpenVz. where echo 3 > /proc/sys/vm/drop_caches
> does not work inside the virtual container, i did it in the hardware node
> but still does not give desired result.

Yeah, if you're in a weird virtualized environment like that you're
likely to have problems, because caching can be done at multiple levels.
In the case of OpenVZ, it's hard to know what the "guest" and what the
"host" even is sometimes, and I wouldn't trust it to respect things like
the Linux VM cache management.

You might have to fall back on the classic method: a program that tries
to allocate as much RAM as it can. On Linux this is EXTREMELY unsafe
unless you ensure you have vm overcommit disabled (see the postgresql
docs) because by default Linux systems will never fail a memory
allocation - instead they'll go looking for a big process to kill to
free some memory. In theory this should be your memory gobbler program,
but in reality the OOM killer isn't so predictable.

So: try turning vm overcommit off, then writing (or finding) a simple
program that keeps on malloc()ing memory until an allocation call fails.
That should force any caches out, freeing you for another cold run.

Note that this method won't just force out the obvious caches like
postgresql data files. It also forces out things like caches of running
binaries. Things will grind to an absolute crawl for a minute or two
before resuming normal speed, because *everything* has to come back from
disk at once. The same is true of using /proc/sys/vm/drop_caches to drop
all caches.

I guess, in the end, nothing really subtitutes for a good reboot.

--
Craig Ringer

Tech-related writing: http://soapyfrogs.blogspot.com/

#avg_ls_inline_popup { position:absolute; z-index:9999; padding: 0px 0px; margin-left: 0px; margin-top: 0px; width: 240px; overflow: hidden; word-wrap: break-word; color: black; font-size: 10px; text-align: left; line-height: 13px;}

pgsql-performance by date:

Previous
From: Robert Haas
Date:
Subject: Re: Low perfomance SUM and Group by large databse
Next
From: Rajesh Kumar Mallah
Date:
Subject: Re: Extremely high CPU usage when building tables