corruption in indexes under heavy load - Mailing list pgsql-general

From Russell Keane
Subject corruption in indexes under heavy load
Date
Msg-id 8D0E5D045E36124A8F1DDDB463D54855BFD24F36A7@mxsvr1.is.inps.co.uk
Whole thread Raw
Responses Re: corruption in indexes under heavy load  (Simon Riggs <simon@2ndquadrant.com>)
Re: corruption in indexes under heavy load  (John R Pierce <pierce@hogranch.com>)
List pgsql-general

We’re writing a large amount of data to a number of tables in PG 9.3 on Windows Server 2012 R2 and then, immediately after, creating a number of indexes (there are no indexes during the initial data write). The data we’re writing exists in files on the same drive as PG’s data.

During the index creation we’re seeing write latency on the storage up to 1500ms (during the data write it may go up to 200ms, possibly due to reading the data from the same storage). The write latency is being investigated separately as there’s no way it should ever get that high.

 

There is no indication in the PG log that anything is amiss while creating the indexes and there are no clients accessing or updating the data at this time, although there are a number of instances of “pgstat wait timeout” around the time of the high write latency. Therefore we must assume that the indexes were created successfully.

 

We’re fairly convinced the issue lies with the actual storage but I was wondering if there is anything within PG that would be affected by the high latency and result in corrupt indexes.

 

Regards,

Russell



Registered name: In Practice Systems Ltd.
Registered address: The Bread Factory, 1a Broughton Street, London, SW8 3QJ
Registered Number: 1788577
Registered in England
The information in this internet email is confidential and is intended solely for the addressee. Access, copying or re-use of information in it by anyone else is not authorised. Any views or opinions presented are solely those of the author and do not necessarily represent those of INPS or any of its affiliates. If you are not the intended recipient please contact is.helpdesk@inps.co.uk

pgsql-general by date:

Previous
From: Francisco Olarte
Date:
Subject: Re: ON CONFLICT does not support deferrable unique constraints
Next
From: Simon Riggs
Date:
Subject: Re: corruption in indexes under heavy load