Re: corruption in indexes under heavy load - Mailing list pgsql-general

From John R Pierce
Subject Re: corruption in indexes under heavy load
Date
Msg-id 07fe0e26-3f6e-52c5-c6b3-2409a31ce25d@hogranch.com
Whole thread Raw
In response to corruption in indexes under heavy load  (Russell Keane <Russell.Keane@inps.co.uk>)
List pgsql-general
On 8/25/2016 1:50 AM, Russell Keane wrote:

We’re writing a large amount of data to a number of tables in PG 9.3 on Windows Server 2012 R2 and then, immediately after, creating a number of indexes (there are no indexes during the initial data write). The data we’re writing exists in files on the same drive as PG’s data.

During the index creation we’re seeing write latency on the storage up to 1500ms (during the data write it may go up to 200ms, possibly due to reading the data from the same storage). The write latency is being investigated separately as there’s no way it should ever get that high.


assuming this server has lots of ram, you might try setting maintenance_work_mem to 1gb and trying those index creations again.   this hugely speeds up the sorting process that create index has to do (it also speeds up vacuuming large tables).

-- 
john r pierce, recycling bits in santa cruz

pgsql-general by date:

Previous
From: Simon Riggs
Date:
Subject: Re: corruption in indexes under heavy load
Next
From: Sylvain Marechal
Date:
Subject: BDR: Recover from "FATAL: mismatch in worker state" without restarting postgres