On Wed, Mar 07, 2018 at 10:55:29AM +0100, Tomas Vondra wrote: > On 03/07/2018 09:55 AM, Arthur Zakirov wrote: > > Hello Andres, > > > > On Thu, Mar 01, 2018 at 08:31:49PM -0800, Andres Freund wrote: > >> Is there any chance we can instead can convert dictionaries into a form > >> we can just mmap() into memory? That'd scale a lot higher and more > >> dynamicallly? > > > > To avoid misunderstanding can you please elaborate on using mmap()? The > > DSM approach looks like more simple and requires less code. Also DSM may > > use mmap() if I'm not mistaken. > > > > I think the mmap() idea is that you preprocess the dictionary, store the > result in a file, and then mmap it when needed, without the expensive > preprocessing.
Understand. I'm not againts the mmap() approach, just I have lack of understanding mmap() benefits... Current shared Ispell approach requires preprocessing after server restarting, and the main advantage of mmap() here is that mmap() doesn't require preprocessing after restarting.
Speaking about the implementation.
It seems that the most appropriate place to store preprocessed files is 'pg_dynshmem' folder. File prefix could be 'ts_dict.', otherwise dsm_cleanup_for_mmap() will remove them.
I'm not sure about reusing dsm_impl_mmap() and dsm_impl_windows(). But maybe it's worth to reuse them.
I don't think so serialization to file (mmap) has not too sense. But the shared dictionary should loaded every time, and should be released every time if it is possible.Maybe there can be some background worker, that holds dictionary in memory.
