make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested. - Mailing list pgsql-hackers

From Masahiro Ikeda
Subject make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.
Date
Msg-id f9a3654d70c2e73da4efbdb0925b2739@oss.nttdata.com
Whole thread Raw
List pgsql-hackers
Hi,

This thread came from another thread about collecting the WAL 
stats([1]).

Is it better to make the stats collector shutdown without writing the 
stats files
if the immediate shutdown is requested?

There was a related discussion([2]) although it's stopped on 12/1/2016.
IIUC, the thread's consensus are

(1) It's useful to make the stats collector shutdown quickly without 
writing the stats files
     if the immediate shutdown is requested in some cases because there 
is a possibility
     that it takes a long time until the failover happens. The possible 
reasons are that
     disk write speed is slow, the stats files are big, and so on. And 
there is no negative impact
     on the users because all stats files are removed in a crash recovery 
phase now.

(2) As another aspect, it needs to change the behavior removing all 
stats files because autovacuum
     uses the stats. There are some ideas, for example writing the stats 
files every X minutes
     (using wal or another mechanism) and even if a crash occurs, the 
startup process can restore
     the stats with slightly low freshness. But, it needs to find a way 
how to handle the stats files
     when deleting on PITR rewind or stats collector crash happens.

I agree that the above consensus and I think we can make separate two 
patches.
The first one is for (1) and the second one is for (2).

Why don't you apply the patch for (1) first?
I attached the patch based on tsunakawa-san's patch([2]).
(v1-0001-pgstat_avoid_writing_on_sigquit.patch)

At this time, there are no cons points for the users because
the stats files are removed in a crash recovery phase as pointed in the 
discussion.

[1] 
https://www.postgresql.org/message-id/c616cf24bf4ecd818f7cab0900a40842%40oss.nttdata.com
[2] 
https://www.postgresql.org/message-id/flat/0A3221C70F24FB45833433255569204D1F5EF25A%40G01JPEXMBYT05

Regards,
-- 
Masahiro Ikeda
NTT DATA CORPORATION
Attachment

pgsql-hackers by date:

Previous
From: Andreas Karlsson
Date:
Subject: Re: [HACKERS] GSoC 2017: Foreign Key Arrays
Next
From: Peter Geoghegan
Date:
Subject: Re: New IndexAM API controlling index vacuum strategies