Check /proc/meminfo for a better breakdown of how the memory is being used. This should work:
grep -A1 Active /proc/meminfo
I suspect your inactive file cache is larger than the active set, suggesting an overly aggressive memory manager.
$ grep -A1 Active /proc/meminfo
Active: 34393512 kB
Inactive: 20765832 kB
Active(anon): 13761028 kB
Inactive(anon): 890688 kB
Active(file): 20632484 kB
Inactive(file): 19875144 kB
The inactive set isn't larger than the active set, they're about even, but I'm still reading that as the memory manager being aggressive in marking pages as inactive, is that what it says to you too?
Interestingly, I just looked at the memory graph for our standby backup database, and while it *normally* uses all the available RAM as the page cache, which is what I'd expect to see, when it was the active database for a time in April and May, the page cache size was reduced by about the same margin. So it's the act of running an active postgres instance that causes the phenomenon.