Re: Linux kernel impact on PostgreSQL performance - Mailing list pgsql-hackers

From Jim Nasby
Subject Re: Linux kernel impact on PostgreSQL performance
Date
Msg-id 52D5D4DC.2010902@nasby.net
Whole thread Raw
In response to Re: Linux kernel impact on PostgreSQL performance  (Jeff Janes <jeff.janes@gmail.com>)
Responses Re: Linux kernel impact on PostgreSQL performance
List pgsql-hackers
On 1/14/14, 11:30 AM, Jeff Janes wrote:
> I think the "reclaim this page if you need memory but leave it resident if there is no memory pressure" hint would be
moreuseful for temporary working files than for what was being discussed above (shared buffers).  When I do work that
needslarge temporary files, I often see physical write IO spike but physical read IO does not.  I interpret that to
meanthat the temporary data is being written to disk to satisfy either dirty_expire_centisecs or dirty_*bytes, but the
dataremains in the FS cache and so disk reads are not needed to satisfy it.  So a hint that says "this file will never
befsynced so please ignore dirty_*bytes and dirty_expire_centisecs.  I will need it again relatively soon (but not
aftera reboot), but will do so mostly sequentially, so please don't evict this without need, but if you do need to then
itis a good candidate" would be good.
 

I also frequently see this, and it has an even larger impact if pgsql_tmp is on the same filesystem as WAL. Which
*theoretically*shouldn't matter with a BBU controller, except that when the kernel suddenly decides your *temporary*
dataneeds to hit the media you're screwed.
 

Though, it also occurs to me... perhaps it would be better for us to simply map temp objects to memory and let the
kernelswap them out if needed...
 
-- 
Jim C. Nasby, Data Architect                       jim@nasby.net
512.569.9461 (cell)                         http://jim.nasby.net



pgsql-hackers by date:

Previous
From: Andres Freund
Date:
Subject: Changeset Extraction v7.0 (was logical changeset generation)
Next
From: Marko Tiikkaja
Date:
Subject: plpgsql.warn_shadow