On Mon, 21 Mar 2011 10:24:22 +0000, Greg Stark wrote:
> On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus <josh@agliodbs.com>
> wrote:
>>> To take the opposite approach... has anyone looked at having the OS
>>> just manage all caching for us? Something like MMAPed shared buffers?
>>> Even if we find the issue with large shared buffers, we still can't
>>> dedicate serious amounts of memory to them because of work_mem
>>> issues. Granted, that's something else on the TODO list, but it
>>> really seems like we're re-inventing the wheels that the OS has
>>> already created here...
>
> A lot of people have talked about it. You can find references to mmap
> going at least as far back as 2001 or so. The problem is that it
> would
> depend on the OS implementing things in a certain way and
> guaranteeing
> things we don't think can be portably assumed. We would need to mlock
> large amounts of address space which most OS's don't allow, and we
> would need to at least mlock and munlock lots of small bits of memory
> all over the place which would create lots and lots of mappings which
> the kernel and hardware implementations would generally not
> appreciate.Actually, just from curious, I done test with mmap, and I got 2% boost on data reading, maybe because of
skippingmemcpy in fread. I really curious how fast, if even, it will be if I add some good and needed stuff and how
e.g.vacuum will work.
<snip>
>> 2-level caches work well for a variety of applications.
>
> I think 2-level caches with simple heuristics like "pin all the
> indexes" is unlikely to be helpful. At least it won't optimize the
> average case and I think that's been proven. It might be helpful for
> optimizing the worst-case which would reduce the standard deviation.
> Perhaps we're at the point now where that matters.
>Actually, 2nd level caches do not pin index buffer. It's just, in simple words, some set of reserved buffers' ids to
beused for index pages, all logic with pining, etc. it's same, the difference is that default level operation will not
touch2nd level. I post some reports from my simple tests. When I was experimenting with 2nd level caches I saw that
someoperations may swap out system tables buffers, too.
<snip>
Regards,Radek
