Thread: FreeBSD config
Hi, We have postgres running on freebsd 4.9 with 2 Gigs of memory. As per repeated advice on the mailing lists we configured effective_cache_size = 25520 which you get by doing `sysctl -n vfs.hibufspace` / 8192 Which results in using 200Megs for disk caching. Is there a reason not to increase the hibufspace beyond the 200 megs and provide a bigger cache to postgres? I looked both on the postgres and freebsd mailing lists and couldn't find a good answer to this. If yes, any suggestions on what would be a good size on a 2 Gig machine? Regards, Dror -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
On Thu, 26 Feb 2004, Dror Matalon wrote: > Hi, > > We have postgres running on freebsd 4.9 with 2 Gigs of memory. As per > repeated advice on the mailing lists we configured effective_cache_size > = 25520 which you get by doing `sysctl -n vfs.hibufspace` / 8192 > > Which results in using 200Megs for disk caching. > > Is there a reason not to increase the hibufspace beyond the 200 megs and > provide a bigger cache to postgres? I looked both on the postgres and > freebsd mailing lists and couldn't find a good answer to this. Actually, I think you're confusing effective_cache_size with shared_buffers. effective_cache_size changes no cache settings for postgresql, it simply acts as a hint to the planner on about how much of the dataset your OS / Kernel / Disk cache can hold. Making it bigger only tells the query planny it's more likely the data it's looking for will be in cache. shared_buffers, OTOH, sets the amount of cache that postgresql uses. It's generall considered that 256 Megs or 1/4 of memory, whichever is LESS, is a good setting for production database servers.
On Thu, Feb 26, 2004 at 11:55:31AM -0700, scott.marlowe wrote: > On Thu, 26 Feb 2004, Dror Matalon wrote: > > > Hi, > > > > We have postgres running on freebsd 4.9 with 2 Gigs of memory. As per > > repeated advice on the mailing lists we configured effective_cache_size > > = 25520 which you get by doing `sysctl -n vfs.hibufspace` / 8192 > > > > Which results in using 200Megs for disk caching. > > > > Is there a reason not to increase the hibufspace beyond the 200 megs and > > provide a bigger cache to postgres? I looked both on the postgres and > > freebsd mailing lists and couldn't find a good answer to this. > > Actually, I think you're confusing effective_cache_size with > shared_buffers. No, I'm not. > > effective_cache_size changes no cache settings for postgresql, it simply > acts as a hint to the planner on about how much of the dataset your OS / > Kernel / Disk cache can hold. I understand that. The question is why have the OS, in this case FreeBsd use only 200 Megs for disk cache and not more. Why not double the vfs.hibufspace to 418119680 and double the effective_cache_size to 51040. > > Making it bigger only tells the query planny it's more likely the data > it's looking for will be in cache. > > shared_buffers, OTOH, sets the amount of cache that postgresql uses. It's > generall considered that 256 Megs or 1/4 of memory, whichever is LESS, is > a good setting for production database servers. > Actually last I looked, I thought that the recommended max shared buffers was 10,000, 80MB, even on machines with large amounts of memory. Regards, Dror -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
On Thu, 26 Feb 2004, Dror Matalon wrote: > On Thu, Feb 26, 2004 at 11:55:31AM -0700, scott.marlowe wrote: > > On Thu, 26 Feb 2004, Dror Matalon wrote: > > > > > Hi, > > > > > > We have postgres running on freebsd 4.9 with 2 Gigs of memory. As per > > > repeated advice on the mailing lists we configured effective_cache_size > > > = 25520 which you get by doing `sysctl -n vfs.hibufspace` / 8192 > > > > > > Which results in using 200Megs for disk caching. > > > > > > Is there a reason not to increase the hibufspace beyond the 200 megs and > > > provide a bigger cache to postgres? I looked both on the postgres and > > > freebsd mailing lists and couldn't find a good answer to this. > > > > Actually, I think you're confusing effective_cache_size with > > shared_buffers. > > No, I'm not. OK, sorry, I wasn't sure which you meant. > > effective_cache_size changes no cache settings for postgresql, it simply > > acts as a hint to the planner on about how much of the dataset your OS / > > Kernel / Disk cache can hold. > > I understand that. The question is why have the OS, in this case FreeBsd > use only 200 Megs for disk cache and not more. Why not double the > vfs.hibufspace to 418119680 and double the effective_cache_size to 51040. Doesn't the kernel just use the spare memory to buffer anyway? I'd say if you got 2 megs memory and nothing else on the box, give a big chunk (1 gig or so) to the kernel to manage. Unless large kernel caches cause some issues in FreeBSD. > > Making it bigger only tells the query planny it's more likely the data > > it's looking for will be in cache. > > > > shared_buffers, OTOH, sets the amount of cache that postgresql uses. It's > > generall considered that 256 Megs or 1/4 of memory, whichever is LESS, is > > a good setting for production database servers. > > > > Actually last I looked, I thought that the recommended max shared > buffers was 10,000, 80MB, even on machines with large amounts of memory. It really depends on what you're doing. For loads involving very large data sets, up to 256 Megs has resulted in improvements, but anything after that has only had advantages in very limited types of applications.
> We have postgres running on freebsd 4.9 with 2 Gigs of memory. As per > repeated advice on the mailing lists we configured effective_cache_size > = 25520 which you get by doing `sysctl -n vfs.hibufspace` / 8192 > > Which results in using 200Megs for disk caching. effective_cache_size does nothing of the sort. CHeck your shared_buffers value... > Is there a reason not to increase the hibufspace beyond the 200 megs and > provide a bigger cache to postgres? I looked both on the postgres and > freebsd mailing lists and couldn't find a good answer to this. Well, maybe butnot necessarily. It's better to leave the OS to look after most of your RAM. Chris
On Fri, Feb 27, 2004 at 05:47:47AM +0800, Christopher Kings-Lynne wrote: > > We have postgres running on freebsd 4.9 with 2 Gigs of memory. As per > > repeated advice on the mailing lists we configured effective_cache_size > > = 25520 which you get by doing `sysctl -n vfs.hibufspace` / 8192 > > > > Which results in using 200Megs for disk caching. > > effective_cache_size does nothing of the sort. CHeck your > shared_buffers value... Sigh. http://www.varlena.com/varlena/GeneralBits/Tidbits/annotated_conf_e.html effective_cache_size Sets the optimizer's assumption about the effective size of the disk cache (that is, the portion of the kernel's disk cache that will be used for PostgreSQL data files). This is measured in disk pages, which are normally 8 kB each. http://archives.postgresql.org/pgsql-performance/2003-07/msg00159.php talks about how to programmatically determine the right setting for effective_cache_size: case `uname` in "FreeBSD") echo "effective_cache_size = $((`sysctl -n vfs.hibufspace` / 8192))" ;; *) echo "Unable to automatically determine the effective cache size" >> /dev/stderr ;; esac which brings me back to my question why not make Freebsd use more of its memory for disk caching and then tell postgres about it. > > > Is there a reason not to increase the hibufspace beyond the 200 megs and > > provide a bigger cache to postgres? I looked both on the postgres and > > freebsd mailing lists and couldn't find a good answer to this. > > Well, maybe butnot necessarily. It's better to leave the OS to look after > most of your RAM. > > Chris > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
On 26 Feb 2004 at 13:58, Dror Matalon wrote:
>
> which brings me back to my question why not make Freebsd use more of its
> memory for disk caching and then tell postgres about it.
>
I think there is some confusion about maxbufsize and hibufspace. I looking at a comment in the FreeBSB source 4.9 that explains this. I think you will want to increase effective_cache to match maxbufsize not hibufspace but I could be wrong.
$FreeBSD: src/sys/kern/vfs_bio.c,v 1.242.2.21 line 363
Thanks for the pointer. So maxbufspace = nbuf * BKVASIZE; Which is confirmed in http://unix.derkeiler.com/Mailing-Lists/FreeBSD/performance/2003-09/0045.html and it looks like there's a patch by Sean Chittenden at http://people.freebsd.org/~seanc/patches/patch-HEAD-kern.nbuf that does what I was asking. Seems a little on the bleeding edge. Has anyone tried this? On Thu, Feb 26, 2004 at 04:36:01PM -0600, Kevin Barnard wrote: > On 26 Feb 2004 at 13:58, Dror Matalon wrote: > > > > > which brings me back to my question why not make Freebsd use more of its > > memory for disk caching and then tell postgres about it. > > > > I think there is some confusion about maxbufsize and hibufspace. I looking at a > comment in the FreeBSB source 4.9 that explains this. I think you will want to > increase effective_cache to match maxbufsize not hibufspace but I could be wrong. > > $FreeBSD: src/sys/kern/vfs_bio.c,v 1.242.2.21 line 363 > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
On Thu, Feb 26, 2004 at 06:06:06PM -0500, Christopher Weimann wrote: > On 02/26/2004-11:16AM, Dror Matalon wrote: > > > > > > effective_cache_size changes no cache settings for postgresql, it simply > > > acts as a hint to the planner on about how much of the dataset your OS / > > > Kernel / Disk cache can hold. > > > > I understand that. The question is why have the OS, in this case FreeBsd > > use only 200 Megs for disk cache and not more. Why not double the > > vfs.hibufspace to 418119680 and double the effective_cache_size to 51040. > > > > FreeBSD uses ALL ram that isn't being used for something else as > its disk cache. The "effective_cache_size" in the PostGreSQL config > has no effect on how the OS chooses to use memory, it is just hint > to the PostGreSQL planner so it can guess the the likelyhood of > what it is looking for being in the cache. Let me try and say it again. I know that setting effective_cache_size doesn't affect the OS' cache. I know it just gives Postgres the *idea* of how much cache the OS is using. I know that. I also know that a correct hint helps performance. I've read Matt Dillon's discussion about the freebsd VM at http://www.daemonnews.org/200001/freebsd_vm.html and I didn't see him saying that Freebsd uses all the free RAM for disk cache. Would you care to provide a URL pointing to that? Assuming you are correct, why has the ongoing recommendation been to use hibufspace/8192 as the effective_cache_size? Seems like it would be quite a bit more on machines with lots of RAM. Regards, Dror -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
Dror Matalon wrote: > Let me try and say it again. I know that setting effective_cache_size > doesn't affect the OS' cache. I know it just gives Postgres the *idea* > of how much cache the OS is using. I know that. I also know that a > correct hint helps performance. > > I've read Matt Dillon's discussion about the freebsd VM at > http://www.daemonnews.org/200001/freebsd_vm.html and I didn't see him > saying that Freebsd uses all the free RAM for disk cache. Would you care > to provide a URL pointing to that? I don't believe freeBSD yses everything available unlike linux. It is actually a good thing. If you have 1GB RAM and kernel buffers set at 600MB, you are guaranteed to have some mmory in crunch situations. As far you original questions, I think you can increase the kernel buffer sizes for VFS safely. However remembet that more to dedicate to kernel buffers, less space you have in case of crunch for whatever reasons. FreeBSD gives you a control which linux does not. Use it to best of your advantage.. Shridhar
On Fri, 27 Feb 2004, Shridhar Daithankar wrote: > Dror Matalon wrote: > > > Let me try and say it again. I know that setting effective_cache_size > > doesn't affect the OS' cache. I know it just gives Postgres the *idea* > > of how much cache the OS is using. I know that. I also know that a > > correct hint helps performance. > > > > I've read Matt Dillon's discussion about the freebsd VM at > > http://www.daemonnews.org/200001/freebsd_vm.html and I didn't see him > > saying that Freebsd uses all the free RAM for disk cache. Would you care > > to provide a URL pointing to that? > > I don't believe freeBSD yses everything available unlike linux. It is actually a > good thing. If you have 1GB RAM and kernel buffers set at 600MB, you are > guaranteed to have some mmory in crunch situations. Linux doesn't work with a pre-assigned size for kernel cache. It just grabs whatever's free, minus a few megs for easily launching new programs or allocating more memory for programs, and uses that for the cache. then, when a request comes in for more memory than is free, it dumps some of the least used buffers and gives them back. It would seem to work very well underneath a mixed load server like an LAPP box.
I guess the thing to do is to move this topic over to a freebsd list where we can get more definitive answers on how disk caching is handled. I asked here since I know that FreeBsd is often recommended, http://www.varlena.com/varlena/GeneralBits/Tidbits/perf.html# as a good platform for postgres, and with Modern machines often having Gigabytes of memory the issue of, possibly, having a disk cache of 200MB would be one often asked. On Fri, Feb 27, 2004 at 12:46:08PM +0530, Shridhar Daithankar wrote: > Dror Matalon wrote: > > >Let me try and say it again. I know that setting effective_cache_size > >doesn't affect the OS' cache. I know it just gives Postgres the *idea* > >of how much cache the OS is using. I know that. I also know that a > >correct hint helps performance. > > > >I've read Matt Dillon's discussion about the freebsd VM at > >http://www.daemonnews.org/200001/freebsd_vm.html and I didn't see him > >saying that Freebsd uses all the free RAM for disk cache. Would you care > >to provide a URL pointing to that? > > I don't believe freeBSD yses everything available unlike linux. It is > actually a good thing. If you have 1GB RAM and kernel buffers set at 600MB, > you are guaranteed to have some mmory in crunch situations. > > As far you original questions, I think you can increase the kernel buffer > sizes for VFS safely. However remembet that more to dedicate to kernel > buffers, less space you have in case of crunch for whatever reasons. > > FreeBSD gives you a control which linux does not. Use it to best of your > advantage.. > > Shridhar -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
>>>>> "DM" == Dror Matalon <dror@zapatec.com> writes: DM> which brings me back to my question why not make Freebsd use more of its DM> memory for disk caching and then tell postgres about it. Because this is a painfully hard thing to do ;-( It involves hacking a system header file and recompiling the kernel. It is not a simple tunable. It has side effects regarding some other sizing parameter as well, but I don't recall the details. Details have been posted to this list at least once before by Sean Chittenden. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D. Khera Communications, Inc. Internet: khera@kciLink.com Rockville, MD +1-301-869-4449 x806 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
On Friday 27 February 2004 21:03, scott.marlowe wrote: > Linux doesn't work with a pre-assigned size for kernel cache. > It just grabs whatever's free, minus a few megs for easily launching new > programs or allocating more memory for programs, and uses that for the > cache. then, when a request comes in for more memory than is free, it > dumps some of the least used buffers and gives them back. > > It would seem to work very well underneath a mixed load server like an > LAPP box. I was just pointing out that freeBSD is different than linux nd for one thing it is good because if there is a bug in freeSD VM, it won't run rampant because you can explicitly limit kernel cache and other parameter. OTOH, freeBSD VM anyways works. And running unlimited kernel cache allowed linux to iron out some of corner cases bugs. Not a concern anymore I believe but having choice is always great.. Shridhar
On 02/26/2004-01:58PM, Dror Matalon wrote: > > Sigh. > Sigh, right back at you. > which brings me back to my question why not make Freebsd use more of its > memory for disk caching and then tell postgres about it. > Because you can't. It already uses ALL RAM that isn't in use for something else.
On 02/26/2004-11:16AM, Dror Matalon wrote: > > > > effective_cache_size changes no cache settings for postgresql, it simply > > acts as a hint to the planner on about how much of the dataset your OS / > > Kernel / Disk cache can hold. > > I understand that. The question is why have the OS, in this case FreeBsd > use only 200 Megs for disk cache and not more. Why not double the > vfs.hibufspace to 418119680 and double the effective_cache_size to 51040. > FreeBSD uses ALL ram that isn't being used for something else as its disk cache. The "effective_cache_size" in the PostGreSQL config has no effect on how the OS chooses to use memory, it is just hint to the PostGreSQL planner so it can guess the the likelyhood of what it is looking for being in the cache.
Shridhar Daithankar wrote: > Dror Matalon wrote: > >> I've read Matt Dillon's discussion about the freebsd VM at >> http://www.daemonnews.org/200001/freebsd_vm.html and I didn't see him >> saying that Freebsd uses all the free RAM for disk cache. Would you care >> to provide a URL pointing to that? > > > Quoting from http://www.daemonnews.org/200001/freebsd_vm.html : <snip>* When To Free a Page* Since the VM system uses all available memory for disk caching, there are usually very few truly-free pages... </snip> Got to say - a very interesting discussion you have all being having, I am now quite confused about what those vfs.*buf* variables actually do... Please feed back any clarfications from the FreeBSD experts to this list! regards Mark
On Mon, Mar 01, 2004 at 08:30:58PM +1300, Mark Kirkwood wrote: > > > Shridhar Daithankar wrote: > > >Dror Matalon wrote: > > > >>I've read Matt Dillon's discussion about the freebsd VM at > >>http://www.daemonnews.org/200001/freebsd_vm.html and I didn't see him > >>saying that Freebsd uses all the free RAM for disk cache. Would you care > >>to provide a URL pointing to that? > > > > > > I noticed this passage too, but ... > Quoting from http://www.daemonnews.org/200001/freebsd_vm.html : > > <snip>* > When To Free a Page* > > Since the VM system uses all available memory for disk caching, there ^^^^^^^^^^^^^ The VM system, as you can see from the article, is focused on paging and caching the programs and program data. Is the cache for disk reads and writes thrown into the mix as well? > are usually very few truly-free pages... > </snip> > > Got to say - a very interesting discussion you have all being having, I > am now quite confused about what those vfs.*buf* variables actually do... Same here. > > Please feed back any clarfications from the FreeBSD experts to this list! > > regards > > Mark > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com
>I noticed this passage too, but ... > > >>Quoting from http://www.daemonnews.org/200001/freebsd_vm.html : >> >><snip>* >>When To Free a Page* >> >>Since the VM system uses all available memory for disk caching, there >> >> > ^^^^^^^^^^^^^ > >The VM system, as you can see from the article, is focused on paging and >caching the programs and program data. Is the cache for disk reads and >writes thrown into the mix as well? > > > Yes - that is the real question. The following link : http://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/vm-cache.html and the few "next" pages afterward talk about a unified buffer cache i.e file buffer cache is part of the KVM system which is part of the VM system - but this does not seem to preclude those vfs.*buf* variables limiting the size of the file buffer cache... hmm ... so no real decrease of confusion at this end.. :-) Mark
>>>>> "CW" == Christopher Weimann <cweimann@k12hq.com> writes: >> which brings me back to my question why not make Freebsd use more of its >> memory for disk caching and then tell postgres about it. >> CW> Because you can't. It already uses ALL RAM that isn't in use for CW> something else. No, it does not. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D. Khera Communications, Inc. Internet: khera@kciLink.com Rockville, MD +1-301-869-4449 x806 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/