Re: Page replacement algorithm in buffer cache - Mailing list pgsql-hackers
From | Atri Sharma |
---|---|
Subject | Re: Page replacement algorithm in buffer cache |
Date | |
Msg-id | CAOeZVicR8pQEERYmF2nxwJ8E1smHPkJKmd0y0wz-+=VVJC3=_g@mail.gmail.com Whole thread Raw |
In response to | Re: Page replacement algorithm in buffer cache (Jeff Janes <jeff.janes@gmail.com>) |
List | pgsql-hackers |
> Perhaps this isn't the help you were looking for, but I spent a long time > looking into this a few years ago. Then I stopped and decided to work on > other things. I would recommend you do so too. Agreed. It seems that my concerns were not valid, and since you have already done some testing here, it further closes the matter. > 4) If most, but not quite all, of the highly-used data fits shared_buffers > and shared_buffers takes most of RAM (or at least, most of RAM not already > needed for other things like work_mem and executables), then the replacement > policy matters a lot. But different policies suit different work-loads, and > there is little reason to think we can come up with a way to choose between > them. (Also, in these conditions, performance is very chaotic. You can run > the same algorithm for a long time, and it can suddenly switch from good to > bad or the other way around, and then stay in that new mode for a long > time). Also, even if you come up with a good algorithm, if you make the > data set 20% smaller or 20% larger, it is no longer a good algorithm. Does that mean that an ideal, high performance postgres setup should *never* set the shared_buffers to a large percentage of the system's RAM? If so, have we ever encountered issues with low specs systems? > 5) Having buffers enter with usage_count=0 rather than 1 would probably be > slightly better most of the time under conditions described in 4, but there > is no way get enough evidence of this over enough conditions to justify > making a change. And besides, how often do people run with shared_buffers > being most of RAM, and the hot data just barely not fitting in it? Agreed. > 1) If all data fits in RAM but not shared_buffers, and you have a very large > number of CPUs and a read-only or read-mostly workload, then BufFreelistLock > can be a major bottle neck. (But, on a Amazon high-CPU instance, I did not > see this very much. I suspect the degree of problem depends a lot on > whether you have a lot of sockets with a few CPUs each, versus one chip with > many CPUs). This is very easy to come up with model cases for, pgbench -S > -c8 -j8, for example, can often show it. I will try that, thanks. > 2) A major reason that people run with shared_buffers much lower than RAM is > that performance seems to suffer with shared_buffers > 8GB under write-heavy > workloads, even with spread-out checkpoints. This is frequently reported as > a real world problem, but as far as I know has never been reduced to a > simple reproducible test case. (Although there was a recent thread, maybe > "High CPU usage / load average after upgrading to Ubuntu 12.04", that I > thought might be relevant to this. I haven't had the time to seriously > study the thread, or the hardware to investigate it myself) > This seems interesting.Do we have some indications as to what the problems could be? Regards, Atri -- Regards, Atri l'apprenant
pgsql-hackers by date: