Re: Recovery performance of standby for multiple concurrent truncateson large tables - Mailing list pgsql-hackers

From Ants Aasma
Subject Re: Recovery performance of standby for multiple concurrent truncateson large tables
Date
Msg-id CA+CSw_v4nvzGLzQoVae7Azs1=v-j_ziJi-c3_U9O0x+_xXYm-w@mail.gmail.com
Whole thread Raw
In response to Recovery performance of standby for multiple concurrent truncateson large tables  ("Jamison, Kirk" <k.jamison@jp.fujitsu.com>)
List pgsql-hackers
On Tue, Jul 10, 2018 at 10:05 AM Jamison, Kirk <k.jamison@jp.fujitsu.com> wrote:

Since in the current implementation, the replay of each TRUNCATE/DROP TABLE scans the whole shared buffer.

One approach (though idea is not really developed yet) is to improve the recovery by delaying the shared buffer scan and invalidation (DropRelFileNodeBuffers) and to put it after the next checkpoint (after failover completion). The replay of TRUNCATE/DROP TABLE just make the checkpointer process remember what relations should be invalidated in the shared buffer during subsequent checkpoint. The checkpointer then scans the shared buffer only once to invalidate the buffers of relations that was dropped and truncated.


How about using the background writer for this? It seems to me that the main reason to invalidate buffers would be to free them up for buffer allocation, which is precisely the task of background writer. When adding a filenode to be invalidated, take note of bgwriter position and add it to a queue. When bgwriter is advancing, check each buffer tag against a hash table of filenodes being invalidated. When background writer has completed a loop it can remove the invalidated filenode. When bgwriter falls behind the clock sweep and there are filenodes to invalidate it should run the invalidation scan instead of skipping ahead. If there are already too many filenodes being invalidated, then whoever is trying to add a new one gets to run the invalidation scan until something can be evicted.

--
Ants Aasma
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: https://www.cybertec-postgresql.com/

pgsql-hackers by date:

Previous
From: Dave Page
Date:
Subject: Re: How can we submit code patches that implement our (pending) patents?
Next
From: Emre Hasegeli
Date:
Subject: Re: [PATCH] Improve geometric types