Re: insert and query performance on big string table with pg_trgm - Mailing list pgsql-performance

From Matthew Hall
Subject Re: insert and query performance on big string table with pg_trgm
Date
Msg-id 8C8D9D8C-F0C6-48D3-AE26-B733EFAED4DB@mhcomputing.net
Whole thread Raw
In response to Re: insert and query performance on big string table with pg_trgm  (Sergei Kornilov <sk@zsrv.org>)
List pgsql-performance
> On Dec 5, 2017, at 11:23 PM, Sergei Kornilov <sk@zsrv.org> wrote:
> You has very slow (or busy) disks, not postgresql issue. Reading 6760 * 8KB in 70 seconds is very bad result.
>
> For better performance you need better disks, at least raid10 (not raid5). Much more memory in shared_buffers can
helpwith read performance and so reduce disk utilization, but write operations still will be slow. 
>
> Sergei

Sergei,

Thanks so much for confirming, this really helps a lot to know what to do. I thought the disk could be some of my
issue,but I wanted to make sure I did all of the obvious tuning first. I have learned some very valuable things which
I'llbe able to use on future challenges like this which I didn't learn previously. 

Based on this advice from everyone, I'm setting up a box with more RAM, lots of SSDs, and RAID 10. I'll write back in a
fewmore days after I've completed it. 

I can also confirm that the previous advice about using a hash / digest based unique index seemed to make the loading
processslower for me, not faster, which is an interesting result to consider for future users following this thread (if
any).I don't yet have specific data how much slower, because it's actually still going! 

Sincerely,
Matthew.

pgsql-performance by date:

Previous
From: Vitaliy Garnashevich
Date:
Subject: Re: Bitmap scan is undercosted?
Next
From: Jeff Janes
Date:
Subject: Re: Bitmap scan is undercosted?