Re: Protect syscache from bloating with negative cache entries - Mailing list pgsql-hackers
| From | Kyotaro HORIGUCHI |
|---|---|
| Subject | Re: Protect syscache from bloating with negative cache entries |
| Date | |
| Msg-id | 20190118.173330.139175539.horiguchi.kyotaro@lab.ntt.co.jp Whole thread Raw |
| In response to | Re: Protect syscache from bloating with negative cache entries (Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>) |
| Responses |
RE: Protect syscache from bloating with negative cache entries
("Tsunakawa, Takayuki" <tsunakawa.takay@jp.fujitsu.com>)
|
| List | pgsql-hackers |
At Fri, 18 Jan 2019 16:39:29 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in
<20190118.163929.229869562.horiguchi.kyotaro@lab.ntt.co.jp>
> Hello.
>
> At Fri, 18 Jan 2019 11:46:03 +1300, Gavin Flower <GavinFlower@archidevsys.co.nz> wrote in
<4e62e6b7-0ffb-54ae-3757-5583fcca38c0@archidevsys.co.nz>
> > On 18/01/2019 08:48, Bruce Momjian wrote:
> > > Unfortunately, because we have not found something we are happy with,
> > > we
> > > have done nothing. I agree LRU can be expensive. What if we do some
> > > kind of clock sweep and expiration like we do for shared buffers? I
>
> So, it doesn't use LRU but a kind of clock-sweep method. If it
> finds the size is about to exceed the threshold by
> resiz(doubl)ing when the current hash is filled up, it tries to
> trim away the entries that are left for a duration corresponding
> to usage count. This is not a hard limit but seems to be a good
> compromise.
>
> > > think the trick is figuring how frequently to do the sweep. What if
> > > we
> > > mark entries as unused every 10 queries, mark them as used on first
> > > use,
> > > and delete cache entries that have not be used in the past 10 queries.
>
> As above, it tires pruning at every resizing time. So this adds
> complexity to the frequent paths only by setting last accessed
> time and incrementing access counter. It scans the whole hash at
> resize time but it doesn't add much comparing to resizing itself.
>
> > If you take that approach, then this number should be configurable.
> > What if I had 12 common queries I used in rotation?
>
> This basically has two knobs. The minimum hash size to do the
> pruning and idle time before reaping unused entries, per
> catcache.
This is the rebased version.
0001: catcache pruning
syscache_memory_target controls per-cache basis minimum size
where this starts pruning.
syscache_prune_min_time controls minimum idle duration until an
catcache entry is removed.
0002: catcache statistics view
track_syscache_usage_interval is the interval statitics of
catcache is collected.
pg_stat_syscache is the view that shows the statistics.
0003: Remote GUC setting
It is independent from the above two, and heavily arguable.
pg_set_backend_config(pid, name, value) changes the GUC <name> on
the backend with <pid> to <value>.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
From 7071de30e79507f55d8021dc9c8b6801a292745c Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
Date: Tue, 16 Oct 2018 13:04:30 +0900
Subject: [PATCH 1/3] Remove entries that haven't been used for a certain time
Catcache entries can be left alone for several reasons. It is not
desirable that they eat up memory. With this patch, This adds
consideration of removal of entries that haven't been used for a
certain time before enlarging the hash array.
---
doc/src/sgml/config.sgml | 38 ++++++
src/backend/access/transam/xact.c | 5 +
src/backend/utils/cache/catcache.c | 166 ++++++++++++++++++++++++--
src/backend/utils/misc/guc.c | 23 ++++
src/backend/utils/misc/postgresql.conf.sample | 2 +
src/include/utils/catcache.h | 28 ++++-
6 files changed, 254 insertions(+), 8 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index b6f5822b84..af3c52b868 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1662,6 +1662,44 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-syscache-memory-target" xreflabel="syscache_memory_target">
+ <term><varname>syscache_memory_target</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>syscache_memory_target</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the maximum amount of memory to which syscache is expanded
+ without pruning. The value defaults to 0, indicating that pruning is
+ always considered. After exceeding this size, syscache pruning is
+ considered according to
+ <xref linkend="guc-syscache-prune-min-age"/>. If you need to keep
+ certain amount of syscache entries with intermittent usage, try
+ increase this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry id="guc-syscache-prune-min-age" xreflabel="syscache_prune_min_age">
+ <term><varname>syscache_prune_min_age</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>syscache_prune_min_age</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the minimum amount of unused time in seconds at which a
+ syscache entry is considered to be removed. -1 indicates that syscache
+ pruning is disabled at all. The value defaults to 600 seconds
+ (<literal>10 minutes</literal>). The syscache entries that are not
+ used for the duration can be removed to prevent syscache bloat. This
+ behavior is suppressed until the size of syscache exceeds
+ <xref linkend="guc-syscache-memory-target"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-max-stack-depth" xreflabel="max_stack_depth">
<term><varname>max_stack_depth</varname> (<type>integer</type>)
<indexterm>
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 18467d96d2..dbffec8067 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -733,7 +733,12 @@ void
SetCurrentStatementStartTimestamp(void)
{
if (!IsParallelWorker())
+ {
stmtStartTimestamp = GetCurrentTimestamp();
+
+ /* Set this timestamp as aproximated current time */
+ SetCatCacheClock(stmtStartTimestamp);
+ }
else
Assert(stmtStartTimestamp != 0);
}
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index 8152f7e21e..ee40093553 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -72,9 +72,24 @@
#define CACHE6_elog(a,b,c,d,e,f,g)
#endif
+/*
+ * GUC variable to define the minimum size of hash to cosider entry eviction.
+ * This variable is shared among various cache mechanisms.
+ */
+int cache_memory_target = 0;
+
+/* GUC variable to define the minimum age of entries that will be cosidered to
+ * be evicted in seconds. This variable is shared among various cache
+ * mechanisms.
+ */
+int cache_prune_min_age = 600;
+
/* Cache management header --- pointer is NULL until created */
static CatCacheHeader *CacheHdr = NULL;
+/* Timestamp used for any operation on caches. */
+TimestampTz catcacheclock = 0;
+
static inline HeapTuple SearchCatCacheInternal(CatCache *cache,
int nkeys,
Datum v1, Datum v2,
@@ -491,6 +506,7 @@ CatCacheRemoveCTup(CatCache *cache, CatCTup *ct)
CatCacheFreeKeys(cache->cc_tupdesc, cache->cc_nkeys,
cache->cc_keyno, ct->keys);
+ cache->cc_tupsize -= ct->size;
pfree(ct);
--cache->cc_ntup;
@@ -842,6 +858,7 @@ InitCatCache(int id,
cp->cc_nkeys = nkeys;
for (i = 0; i < nkeys; ++i)
cp->cc_keyno[i] = key[i];
+ cp->cc_tupsize = 0;
/*
* new cache is initialized as far as we can go for now. print some
@@ -859,9 +876,129 @@ InitCatCache(int id,
*/
MemoryContextSwitchTo(oldcxt);
+ /* initilize catcache reference clock if haven't done yet */
+ if (catcacheclock == 0)
+ catcacheclock = GetCurrentTimestamp();
+
return cp;
}
+/*
+ * CatCacheCleanupOldEntries - Remove infrequently-used entries
+ *
+ * Catcache entries can be left alone for several reasons. We remove them if
+ * they are not accessed for a certain time to prevent catcache from
+ * bloating. The eviction is performed with the similar algorithm with buffer
+ * eviction using access counter. Entries that are accessed several times can
+ * live longer than those that have had no access in the same duration.
+ */
+static bool
+CatCacheCleanupOldEntries(CatCache *cp)
+{
+ int i;
+ int nremoved = 0;
+ size_t hash_size;
+#ifdef CATCACHE_STATS
+ /* These variables are only for debugging purpose */
+ int ntotal = 0;
+ /*
+ * nth element in nentries stores the number of cache entries that have
+ * lived unaccessed for corresponding multiple in ageclass of
+ * cache_prune_min_age. The index of nremoved_entry is the value of the
+ * clock-sweep counter, which takes from 0 up to 2.
+ */
+ double ageclass[] = {0.05, 0.1, 1.0, 2.0, 3.0, 0.0};
+ int nentries[] = {0, 0, 0, 0, 0, 0};
+ int nremoved_entry[3] = {0, 0, 0};
+ int j;
+#endif
+
+ /* Return immediately if no pruning is wanted */
+ if (cache_prune_min_age < 0)
+ return false;
+
+ /*
+ * Return without pruning if the size of the hash is below the target.
+ */
+ hash_size = cp->cc_nbuckets * sizeof(dlist_head);
+ if (hash_size + cp->cc_tupsize < (Size) cache_memory_target * 1024L)
+ return false;
+
+ /* Search the whole hash for entries to remove */
+ for (i = 0; i < cp->cc_nbuckets; i++)
+ {
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &cp->cc_bucket[i])
+ {
+ CatCTup *ct = dlist_container(CatCTup, cache_elem, iter.cur);
+ long entry_age;
+ int us;
+
+
+ /*
+ * Calculate the duration from the time of the last access to the
+ * "current" time. Since catcacheclock is not advanced within a
+ * transaction, the entries that are accessed within the current
+ * transaction won't be pruned.
+ */
+ TimestampDifference(ct->lastaccess, catcacheclock, &entry_age, &us);
+
+#ifdef CATCACHE_STATS
+ /* count catcache entries for each age class */
+ ntotal++;
+ for (j = 0 ;
+ ageclass[j] != 0.0 &&
+ entry_age > cache_prune_min_age * ageclass[j] ;
+ j++);
+ if (ageclass[j] == 0.0) j--;
+ nentries[j]++;
+#endif
+
+ /*
+ * Try to remove entries older than cache_prune_min_age seconds.
+ * Entries that are not accessed after last pruning are removed in
+ * that seconds, and that has been accessed several times are
+ * removed after leaving alone for up to three times of the
+ * duration. We don't try shrink buckets since pruning effectively
+ * caps catcache expansion in the long term.
+ */
+ if (entry_age > cache_prune_min_age)
+ {
+#ifdef CATCACHE_STATS
+ Assert (ct->naccess >= 0 && ct->naccess <= 2);
+ nremoved_entry[ct->naccess]++;
+#endif
+ if (ct->naccess > 0)
+ ct->naccess--;
+ else
+ {
+ if (!ct->c_list || ct->c_list->refcount == 0)
+ {
+ CatCacheRemoveCTup(cp, ct);
+ nremoved++;
+ }
+ }
+ }
+ }
+ }
+
+#ifdef CATCACHE_STATS
+ ereport(DEBUG1,
+ (errmsg ("removed %d/%d, age(-%.0fs:%d, -%.0fs:%d, *-%.0fs:%d, -%.0fs:%d, -%.0fs:%d) naccessed(0:%d, 1:%d,
2:%d)",
+ nremoved, ntotal,
+ ageclass[0] * cache_prune_min_age, nentries[0],
+ ageclass[1] * cache_prune_min_age, nentries[1],
+ ageclass[2] * cache_prune_min_age, nentries[2],
+ ageclass[3] * cache_prune_min_age, nentries[3],
+ ageclass[4] * cache_prune_min_age, nentries[4],
+ nremoved_entry[0], nremoved_entry[1], nremoved_entry[2]),
+ errhidestmt(true)));
+#endif
+
+ return nremoved > 0;
+}
+
/*
* Enlarge a catcache, doubling the number of buckets.
*/
@@ -1275,6 +1412,11 @@ SearchCatCacheInternal(CatCache *cache,
*/
dlist_move_head(bucket, &ct->cache_elem);
+ /* Update access information for pruning */
+ if (ct->naccess < 2)
+ ct->naccess++;
+ ct->lastaccess = catcacheclock;
+
/*
* If it's a positive entry, bump its refcount and return it. If it's
* negative, we can report failure to the caller.
@@ -1820,11 +1962,13 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
CatCTup *ct;
HeapTuple dtp;
MemoryContext oldcxt;
+ int tupsize = 0;
/* negative entries have no tuple associated */
if (ntp)
{
int i;
+ int tupsize;
Assert(!negative);
@@ -1843,13 +1987,14 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
/* Allocate memory for CatCTup and the cached tuple in one go */
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
- ct = (CatCTup *) palloc(sizeof(CatCTup) +
- MAXIMUM_ALIGNOF + dtp->t_len);
+ tupsize = sizeof(CatCTup) + MAXIMUM_ALIGNOF + dtp->t_len;
+ ct = (CatCTup *) palloc(tupsize);
ct->tuple.t_len = dtp->t_len;
ct->tuple.t_self = dtp->t_self;
ct->tuple.t_tableOid = dtp->t_tableOid;
ct->tuple.t_data = (HeapTupleHeader)
MAXALIGN(((char *) ct) + sizeof(CatCTup));
+ ct->size = tupsize;
/* copy tuple contents */
memcpy((char *) ct->tuple.t_data,
(const char *) dtp->t_data,
@@ -1877,8 +2022,8 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
{
Assert(negative);
oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
- ct = (CatCTup *) palloc(sizeof(CatCTup));
-
+ tupsize = sizeof(CatCTup);
+ ct = (CatCTup *) palloc(tupsize);
/*
* Store keys - they'll point into separately allocated memory if not
* by-value.
@@ -1899,17 +2044,24 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp, Datum *arguments,
ct->dead = false;
ct->negative = negative;
ct->hash_value = hashValue;
+ ct->naccess = 0;
+ ct->lastaccess = catcacheclock;
+ ct->size = tupsize;
dlist_push_head(&cache->cc_bucket[hashIndex], &ct->cache_elem);
cache->cc_ntup++;
CacheHdr->ch_ntup++;
+ cache->cc_tupsize += tupsize;
/*
- * If the hash table has become too full, enlarge the buckets array. Quite
- * arbitrarily, we enlarge when fill factor > 2.
+ * If the hash table has become too full, try cleanup by removing
+ * infrequently used entries to make a room for the new entry. If it
+ * failed, enlarge the bucket array instead. Quite arbitrarily, we try
+ * this when fill factor > 2.
*/
- if (cache->cc_ntup > cache->cc_nbuckets * 2)
+ if (cache->cc_ntup > cache->cc_nbuckets * 2 &&
+ !CatCacheCleanupOldEntries(cache))
RehashCatCache(cache);
return ct;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index c216ed0922..134c357bf3 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -80,6 +80,7 @@
#include "tsearch/ts_cache.h"
#include "utils/builtins.h"
#include "utils/bytea.h"
+#include "utils/catcache.h"
#include "utils/guc_tables.h"
#include "utils/float.h"
#include "utils/memutils.h"
@@ -2190,6 +2191,28 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"cache_memory_target", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the minimum syscache size to keep."),
+ gettext_noop("Cache is not pruned before exceeding this size."),
+ GUC_UNIT_KB
+ },
+ &cache_memory_target,
+ 0, 0, MAX_KILOBYTES,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"cache_prune_min_age", PGC_USERSET, RESOURCES_MEM,
+ gettext_noop("Sets the minimum unused duration of cache entries before removal."),
+ gettext_noop("Cache entries that live unused for longer than this seconds are considered to be
removed."),
+ GUC_UNIT_S
+ },
+ &cache_prune_min_age,
+ 600, -1, INT_MAX,
+ NULL, NULL, NULL
+ },
+
/*
* We use the hopefully-safely-small value of 100kB as the compiled-in
* default for max_stack_depth. InitializeGUCOptions will increase it if
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index a21865a77f..d82af3bd6c 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -128,6 +128,8 @@
#work_mem = 4MB # min 64kB
#maintenance_work_mem = 64MB # min 1MB
#autovacuum_work_mem = -1 # min 1MB, or -1 to use maintenance_work_mem
+#cache_memory_target = 0kB # in kB
+#cache_prune_min_age = 600s # -1 disables pruning
#max_stack_depth = 2MB # min 100kB
#dynamic_shared_memory_type = posix # the default is the first option
# supported by the operating system:
diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h
index 65d816a583..5d24809900 100644
--- a/src/include/utils/catcache.h
+++ b/src/include/utils/catcache.h
@@ -22,6 +22,7 @@
#include "access/htup.h"
#include "access/skey.h"
+#include "datatype/timestamp.h"
#include "lib/ilist.h"
#include "utils/relcache.h"
@@ -61,6 +62,7 @@ typedef struct catcache
slist_node cc_next; /* list link */
ScanKeyData cc_skey[CATCACHE_MAXKEYS]; /* precomputed key info for heap
* scans */
+ int cc_tupsize; /* total amount of catcache tuples */
/*
* Keep these at the end, so that compiling catcache.c with CATCACHE_STATS
@@ -119,7 +121,9 @@ typedef struct catctup
bool dead; /* dead but not yet removed? */
bool negative; /* negative cache entry? */
HeapTupleData tuple; /* tuple management header */
-
+ int naccess; /* # of access to this entry, up to 2 */
+ TimestampTz lastaccess; /* approx. timestamp of the last usage */
+ int size; /* palloc'ed size off this tuple */
/*
* The tuple may also be a member of at most one CatCList. (If a single
* catcache is list-searched with varying numbers of keys, we may have to
@@ -189,6 +193,28 @@ typedef struct catcacheheader
/* this extern duplicates utils/memutils.h... */
extern PGDLLIMPORT MemoryContext CacheMemoryContext;
+/* for guc.c, not PGDLLPMPORT'ed */
+extern int cache_prune_min_age;
+extern int cache_memory_target;
+
+/* to use as access timestamp of catcache entries */
+extern TimestampTz catcacheclock;
+
+/*
+ * SetCatCacheClock - set timestamp for catcache access record
+ */
+static inline void
+SetCatCacheClock(TimestampTz ts)
+{
+ catcacheclock = ts;
+}
+
+static inline TimestampTz
+GetCatCacheClock(void)
+{
+ return catcacheclock;
+}
+
extern void CreateCacheMemoryContext(void);
extern CatCache *InitCatCache(int id, Oid reloid, Oid indexoid,
--
2.16.3
From 7cc50a1bf62290c704d90cd9b5b740d68cd8f646 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
Date: Tue, 16 Oct 2018 15:48:28 +0900
Subject: [PATCH 2/3] Syscache usage tracking feature.
Collects syscache usage statictics and show it using the view
pg_stat_syscache. The feature is controlled by the GUC variable
track_syscache_usage_interval.
---
doc/src/sgml/config.sgml | 15 ++
src/backend/catalog/system_views.sql | 17 +++
src/backend/postmaster/pgstat.c | 206 ++++++++++++++++++++++++--
src/backend/tcop/postgres.c | 23 +++
src/backend/utils/adt/pgstatfuncs.c | 134 +++++++++++++++++
src/backend/utils/cache/catcache.c | 115 ++++++++++----
src/backend/utils/cache/syscache.c | 24 +++
src/backend/utils/init/globals.c | 1 +
src/backend/utils/init/postinit.c | 11 ++
src/backend/utils/misc/guc.c | 10 ++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/catalog/pg_proc.dat | 9 ++
src/include/miscadmin.h | 1 +
src/include/pgstat.h | 7 +-
src/include/utils/catcache.h | 9 +-
src/include/utils/syscache.h | 19 +++
src/include/utils/timeout.h | 1 +
src/test/regress/expected/rules.out | 24 ++-
18 files changed, 582 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index af3c52b868..6dd024340b 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -6662,6 +6662,21 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-track-syscache-usage-interval" xreflabel="track_syscache_usage_interval">
+ <term><varname>track_syscache_usage_interval</varname> (<type>integer</type>)
+ <indexterm>
+ <primary><varname>track_syscache_usage_interval</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Specifies the interval to collect system cache usage statistics in
+ milliseconds. This parameter is 0 by default, which means disabled.
+ Only superusers can change this setting.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-track-io-timing" xreflabel="track_io_timing">
<term><varname>track_io_timing</varname> (<type>boolean</type>)
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index f4d9e9daf7..30e2da935a 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -904,6 +904,22 @@ CREATE VIEW pg_stat_progress_vacuum AS
FROM pg_stat_get_progress_info('VACUUM') AS S
LEFT JOIN pg_database D ON S.datid = D.oid;
+CREATE VIEW pg_stat_syscache AS
+ SELECT
+ S.pid AS pid,
+ S.relid::regclass AS relname,
+ S.indid::regclass AS cache_name,
+ S.size AS size,
+ S.ntup AS ntuples,
+ S.searches AS searches,
+ S.hits AS hits,
+ S.neg_hits AS neg_hits,
+ S.ageclass AS ageclass,
+ S.last_update AS last_update
+ FROM pg_stat_activity A
+ JOIN LATERAL (SELECT A.pid, * FROM pg_get_syscache_stats(A.pid)) S
+ ON (A.pid = S.pid);
+
CREATE VIEW pg_user_mappings AS
SELECT
U.oid AS umid,
@@ -1183,6 +1199,7 @@ GRANT EXECUTE ON FUNCTION pg_ls_waldir() TO pg_monitor;
GRANT EXECUTE ON FUNCTION pg_ls_archive_statusdir() TO pg_monitor;
GRANT EXECUTE ON FUNCTION pg_ls_tmpdir() TO pg_monitor;
GRANT EXECUTE ON FUNCTION pg_ls_tmpdir(oid) TO pg_monitor;
+GRANT EXECUTE ON FUNCTION pg_get_syscache_stats(int) TO pg_monitor;
GRANT pg_read_all_settings TO pg_monitor;
GRANT pg_read_all_stats TO pg_monitor;
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 13da412c59..2c0c6b343e 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -66,6 +66,7 @@
#include "utils/ps_status.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/timestamp.h"
#include "utils/tqual.h"
@@ -125,6 +126,7 @@
bool pgstat_track_activities = false;
bool pgstat_track_counts = false;
int pgstat_track_functions = TRACK_FUNC_OFF;
+int pgstat_track_syscache_usage_interval = 0;
int pgstat_track_activity_query_size = 1024;
/* ----------
@@ -237,6 +239,11 @@ typedef struct TwoPhasePgStatRecord
bool t_truncated; /* was the relation truncated? */
} TwoPhasePgStatRecord;
+/* bitmap symbols to specify target file types remove */
+#define PGSTAT_REMFILE_DBSTAT 1 /* remove only databsae stats files */
+#define PGSTAT_REMFILE_SYSCACHE 2 /* remove only syscache stats files */
+#define PGSTAT_REMFILE_ALL 3 /* remove both type of files */
+
/*
* Info about current "snapshot" of stats file
*/
@@ -631,10 +638,13 @@ startup_failed:
}
/*
- * subroutine for pgstat_reset_all
+ * remove stats files
+ *
+ * clean up stats files in specified directory. target is one of
+ * PGSTAT_REFILE_DBSTAT/SYSCACHE/ALL and restricts files to remove.
*/
static void
-pgstat_reset_remove_files(const char *directory)
+pgstat_reset_remove_files(const char *directory, int target)
{
DIR *dir;
struct dirent *entry;
@@ -645,25 +655,39 @@ pgstat_reset_remove_files(const char *directory)
{
int nchars;
Oid tmp_oid;
+ int filetype = 0;
/*
* Skip directory entries that don't match the file names we write.
* See get_dbstat_filename for the database-specific pattern.
*/
if (strncmp(entry->d_name, "global.", 7) == 0)
+ {
+ filetype = PGSTAT_REMFILE_DBSTAT;
nchars = 7;
+ }
else
{
+ char head[2];
+
nchars = 0;
- (void) sscanf(entry->d_name, "db_%u.%n",
- &tmp_oid, &nchars);
- if (nchars <= 0)
- continue;
+ (void) sscanf(entry->d_name, "%c%c_%u.%n",
+ head, head + 1, &tmp_oid, &nchars);
+
/* %u allows leading whitespace, so reject that */
- if (strchr("0123456789", entry->d_name[3]) == NULL)
+ if (nchars < 3 || !isdigit(entry->d_name[3]))
continue;
+
+ if (strncmp(head, "db", 2) == 0)
+ filetype = PGSTAT_REMFILE_DBSTAT;
+ else if (strncmp(head, "cc", 2) == 0)
+ filetype = PGSTAT_REMFILE_SYSCACHE;
}
+ /* skip if this is not a target */
+ if ((filetype & target) == 0)
+ continue;
+
if (strcmp(entry->d_name + nchars, "tmp") != 0 &&
strcmp(entry->d_name + nchars, "stat") != 0)
continue;
@@ -684,8 +708,9 @@ pgstat_reset_remove_files(const char *directory)
void
pgstat_reset_all(void)
{
- pgstat_reset_remove_files(pgstat_stat_directory);
- pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY);
+ pgstat_reset_remove_files(pgstat_stat_directory, PGSTAT_REMFILE_ALL);
+ pgstat_reset_remove_files(PGSTAT_STAT_PERMANENT_DIRECTORY,
+ PGSTAT_REMFILE_ALL);
}
#ifdef EXEC_BACKEND
@@ -4286,6 +4311,9 @@ PgstatCollectorMain(int argc, char *argv[])
pgStatRunningInCollector = true;
pgStatDBHash = pgstat_read_statsfiles(InvalidOid, true, true);
+ /* Remove left-over syscache stats files */
+ pgstat_reset_remove_files(pgstat_stat_directory, PGSTAT_REMFILE_SYSCACHE);
+
/*
* Loop to process messages until we get SIGQUIT or detect ungraceful
* death of our parent postmaster.
@@ -6376,3 +6404,163 @@ pgstat_clip_activity(const char *raw_activity)
return activity;
}
+
+/*
+ * return the filename for a syscache stat file; filename is the output
+ * buffer, of length len.
+ */
+void
+pgstat_get_syscachestat_filename(bool permanent, bool tempname, int backendid,
+ char *filename, int len)
+{
+ int printed;
+
+ /* NB -- pgstat_reset_remove_files knows about the pattern this uses */
+ printed = snprintf(filename, len, "%s/cc_%u.%s",
+ permanent ? PGSTAT_STAT_PERMANENT_DIRECTORY :
+ pgstat_stat_directory,
+ backendid,
+ tempname ? "tmp" : "stat");
+ if (printed >= len)
+ elog(ERROR, "overlength pgstat path");
+}
+
+/*
+ * pgstat_write_syscache_stats() -
+ * Write the syscache statistics files.
+ *
+ * If 'force' is false, this function skips writing a file and resturns the
+ * time remaining in the current interval in milliseconds. If'force' is true,
+ * writes a file regardless of the remaining time and reset the interval.
+ */
+long
+pgstat_write_syscache_stats(bool force)
+{
+ static TimestampTz last_report = 0;
+ TimestampTz now;
+ long elapsed;
+ long secs;
+ int usecs;
+ int cacheId;
+ FILE *fpout;
+ char statfile[MAXPGPATH];
+ char tmpfile[MAXPGPATH];
+
+ /* Return if we don't want it */
+ if (!force && pgstat_track_syscache_usage_interval <= 0)
+ return 0;
+
+
+ /* Check aginst the in*/
+ now = GetCurrentTransactionStopTimestamp();
+ TimestampDifference(last_report, now, &secs, &usecs);
+ elapsed = secs * 1000 + usecs / 1000;
+
+ if (!force && elapsed < pgstat_track_syscache_usage_interval)
+ {
+ /* not yet the time, inform the remaining time to the caller */
+ return pgstat_track_syscache_usage_interval - elapsed;
+ }
+
+ /* now write the file */
+ last_report = now;
+
+ pgstat_get_syscachestat_filename(false, true,
+ MyBackendId, tmpfile, MAXPGPATH);
+ pgstat_get_syscachestat_filename(false, false,
+ MyBackendId, statfile, MAXPGPATH);
+
+ /*
+ * This function can be called from ProcessInterrupts(). Inhibit recursive
+ * interrupts to avoid recursive entry.
+ */
+ HOLD_INTERRUPTS();
+
+ fpout = AllocateFile(tmpfile, PG_BINARY_W);
+ if (fpout == NULL)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not open temporary statistics file \"%s\": %m",
+ tmpfile)));
+ /*
+ * Failure writing this file is not critical. Just skip this time and
+ * tell caller to wait for the next interval.
+ */
+ RESUME_INTERRUPTS();
+ return pgstat_track_syscache_usage_interval;
+ }
+
+ /* write out every catcache stats */
+ for (cacheId = 0 ; cacheId < SysCacheSize ; cacheId++)
+ {
+ SysCacheStats *stats;
+
+ stats = SysCacheGetStats(cacheId);
+ Assert (stats);
+
+ /* write error is checked later using ferror() */
+ fputc('T', fpout);
+ (void)fwrite(&cacheId, sizeof(int), 1, fpout);
+ (void)fwrite(&last_report, sizeof(TimestampTz), 1, fpout);
+ (void)fwrite(stats, sizeof(*stats), 1, fpout);
+ }
+ fputc('E', fpout);
+
+ if (ferror(fpout))
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not write syscache statistics file \"%s\": %m",
+ tmpfile)));
+ FreeFile(fpout);
+ unlink(tmpfile);
+ }
+ else if (FreeFile(fpout) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not close syscache statistics file \"%s\": %m",
+ tmpfile)));
+ unlink(tmpfile);
+ }
+ else if (rename(tmpfile, statfile) < 0)
+ {
+ ereport(LOG,
+ (errcode_for_file_access(),
+ errmsg("could not rename syscache statistics file \"%s\" to \"%s\": %m",
+ tmpfile, statfile)));
+ unlink(tmpfile);
+ }
+
+ RESUME_INTERRUPTS();
+ return 0;
+}
+
+/*
+ * GUC assignment callback for track_syscache_usage_interval.
+ *
+ * Make a statistics file immedately when syscache statistics is turned
+ * on. Remove it as soon as turned off as well.
+ */
+void
+pgstat_track_syscache_assign_hook(int newval, void *extra)
+{
+ if (newval > 0)
+ {
+ /*
+ * Immediately create a stats file. It's safe since we're not midst
+ * accessing syscache.
+ */
+ pgstat_write_syscache_stats(true);
+ }
+ else
+ {
+ /* Turned off, immediately remove the statsfile */
+ char fname[MAXPGPATH];
+
+ pgstat_get_syscachestat_filename(false, false, MyBackendId,
+ fname, MAXPGPATH);
+ unlink(fname); /* don't care of the result */
+ }
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0c0891b33e..e7972e645f 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3157,6 +3157,12 @@ ProcessInterrupts(void)
}
+ if (IdleSyscacheStatsUpdateTimeoutPending)
+ {
+ IdleSyscacheStatsUpdateTimeoutPending = false;
+ pgstat_write_syscache_stats(true);
+ }
+
if (ParallelMessagePending)
HandleParallelMessages();
}
@@ -3733,6 +3739,7 @@ PostgresMain(int argc, char *argv[],
sigjmp_buf local_sigjmp_buf;
volatile bool send_ready_for_query = true;
bool disable_idle_in_transaction_timeout = false;
+ bool disable_idle_catcache_update_timeout = false;
/* Initialize startup process environment if necessary. */
if (!IsUnderPostmaster)
@@ -4173,9 +4180,19 @@ PostgresMain(int argc, char *argv[],
}
else
{
+ long timeout;
+
ProcessCompletedNotifies();
pgstat_report_stat(false);
+ timeout = pgstat_write_syscache_stats(false);
+
+ if (timeout > 0)
+ {
+ disable_idle_catcache_update_timeout = true;
+ enable_timeout_after(IDLE_CATCACHE_UPDATE_TIMEOUT,
+ timeout);
+ }
set_ps_display("idle", false);
pgstat_report_activity(STATE_IDLE, NULL);
}
@@ -4218,6 +4235,12 @@ PostgresMain(int argc, char *argv[],
disable_idle_in_transaction_timeout = false;
}
+ if (disable_idle_catcache_update_timeout)
+ {
+ disable_timeout(IDLE_CATCACHE_UPDATE_TIMEOUT, false);
+ disable_idle_catcache_update_timeout = false;
+ }
+
/*
* (6) check for any other interesting events that happened while we
* slept.
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 053bb73863..0d32bf8daa 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -14,6 +14,8 @@
*/
#include "postgres.h"
+#include <sys/stat.h>
+
#include "access/htup_details.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_type.h"
@@ -28,6 +30,7 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/inet.h"
+#include "utils/syscache.h"
#include "utils/timestamp.h"
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -1882,3 +1885,134 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
PG_RETURN_DATUM(HeapTupleGetDatum(
heap_form_tuple(tupdesc, values, nulls)));
}
+
+Datum
+pgstat_get_syscache_stats(PG_FUNCTION_ARGS)
+{
+#define PG_GET_SYSCACHE_SIZE 9
+ int pid = PG_GETARG_INT32(0);
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ TupleDesc tupdesc;
+ Tuplestorestate *tupstore;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ PgBackendStatus *beentry;
+ int beid;
+ char fname[MAXPGPATH];
+ FILE *fpin;
+ char c;
+
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ if (!(rsinfo->allowedModes & SFRM_Materialize))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("materialize mode required, but it is not " \
+ "allowed in this context")));
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ rsinfo->returnMode = SFRM_Materialize;
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ /* find beentry for given pid*/
+ beentry = NULL;
+ for (beid = 1;
+ (beentry = pgstat_fetch_stat_beentry(beid)) &&
+ beentry->st_procpid != pid ;
+ beid++);
+
+ /*
+ * we silently return empty result on failure or insufficient privileges
+ */
+ if (!beentry ||
+ (!has_privs_of_role(GetUserId(), beentry->st_userid) &&
+ !is_member_of_role(GetUserId(), DEFAULT_ROLE_READ_ALL_STATS)))
+ goto no_data;
+
+ pgstat_get_syscachestat_filename(false, false, beid, fname, MAXPGPATH);
+
+ if ((fpin = AllocateFile(fname, PG_BINARY_R)) == NULL)
+ {
+ if (errno != ENOENT)
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("could not open statistics file \"%s\": %m",
+ fname)));
+ /* also return empty on no statistics file */
+ goto no_data;
+ }
+
+ /* read the statistics file into tuplestore */
+ while ((c = fgetc(fpin)) == 'T')
+ {
+ TimestampTz last_update;
+ SysCacheStats stats;
+ int cacheid;
+ Datum values[PG_GET_SYSCACHE_SIZE];
+ bool nulls[PG_GET_SYSCACHE_SIZE] = {0};
+ Datum datums[SYSCACHE_STATS_NAGECLASSES * 2];
+ bool arrnulls[SYSCACHE_STATS_NAGECLASSES * 2] = {0};
+ int dims[] = {SYSCACHE_STATS_NAGECLASSES, 2};
+ int lbs[] = {1, 1};
+ ArrayType *arr;
+ int i, j;
+
+ fread(&cacheid, sizeof(int), 1, fpin);
+ fread(&last_update, sizeof(TimestampTz), 1, fpin);
+ if (fread(&stats, 1, sizeof(stats), fpin) != sizeof(stats))
+ {
+ ereport(WARNING,
+ (errmsg("corrupted syscache statistics file \"%s\"",
+ fname)));
+ goto no_data;
+ }
+
+ i = 0;
+ values[i++] = ObjectIdGetDatum(stats.reloid);
+ values[i++] = ObjectIdGetDatum(stats.indoid);
+ values[i++] = Int64GetDatum(stats.size);
+ values[i++] = Int64GetDatum(stats.ntuples);
+ values[i++] = Int64GetDatum(stats.nsearches);
+ values[i++] = Int64GetDatum(stats.nhits);
+ values[i++] = Int64GetDatum(stats.nneg_hits);
+
+ for (j = 0 ; j < SYSCACHE_STATS_NAGECLASSES ; j++)
+ {
+ datums[j * 2] = Int32GetDatum((int32) stats.ageclasses[j]);
+ datums[j * 2 + 1] = Int32GetDatum((int32) stats.nclass_entries[j]);
+ }
+
+ arr = construct_md_array(datums, arrnulls, 2, dims, lbs,
+ INT4OID, sizeof(int32), true, 'i');
+ values[i++] = PointerGetDatum(arr);
+
+ values[i++] = TimestampTzGetDatum(last_update);
+
+ Assert (i == PG_GET_SYSCACHE_SIZE);
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+ }
+
+ /* check for the end of file. abandon the result if file is broken */
+ if (c != 'E' || fgetc(fpin) != EOF)
+ tuplestore_clear(tupstore);
+
+ FreeFile(fpin);
+
+no_data:
+ tuplestore_donestoring(tupstore);
+ return (Datum) 0;
+}
diff --git a/src/backend/utils/cache/catcache.c b/src/backend/utils/cache/catcache.c
index ee40093553..4a3b3094a0 100644
--- a/src/backend/utils/cache/catcache.c
+++ b/src/backend/utils/cache/catcache.c
@@ -90,6 +90,10 @@ static CatCacheHeader *CacheHdr = NULL;
/* Timestamp used for any operation on caches. */
TimestampTz catcacheclock = 0;
+/* age classes for pruning */
+static double ageclass[SYSCACHE_STATS_NAGECLASSES]
+ = {0.05, 0.1, 1.0, 2.0, 3.0, 0.0};
+
static inline HeapTuple SearchCatCacheInternal(CatCache *cache,
int nkeys,
Datum v1, Datum v2,
@@ -620,9 +624,7 @@ CatCacheInvalidate(CatCache *cache, uint32 hashValue)
else
CatCacheRemoveCTup(cache, ct);
CACHE1_elog(DEBUG2, "CatCacheInvalidate: invalidated");
-#ifdef CATCACHE_STATS
cache->cc_invals++;
-#endif
/* could be multiple matches, so keep looking! */
}
}
@@ -698,9 +700,7 @@ ResetCatalogCache(CatCache *cache)
}
else
CatCacheRemoveCTup(cache, ct);
-#ifdef CATCACHE_STATS
cache->cc_invals++;
-#endif
}
}
}
@@ -907,10 +907,11 @@ CatCacheCleanupOldEntries(CatCache *cp)
* cache_prune_min_age. The index of nremoved_entry is the value of the
* clock-sweep counter, which takes from 0 up to 2.
*/
- double ageclass[] = {0.05, 0.1, 1.0, 2.0, 3.0, 0.0};
- int nentries[] = {0, 0, 0, 0, 0, 0};
+ int nentries[SYSCACHE_STATS_NAGECLASSES] = {0, 0, 0, 0, 0, 0};
int nremoved_entry[3] = {0, 0, 0};
int j;
+
+ Assert(ageclass[SYSCACHE_STATS_NAGECLASSES - 1] == 0.0);
#endif
/* Return immediately if no pruning is wanted */
@@ -924,7 +925,11 @@ CatCacheCleanupOldEntries(CatCache *cp)
if (hash_size + cp->cc_tupsize < (Size) cache_memory_target * 1024L)
return false;
- /* Search the whole hash for entries to remove */
+ /*
+ * Search the whole hash for entries to remove. This is a quite time
+ * consuming task during catcache lookup, but accetable since now we are
+ * going to expand the hash table.
+ */
for (i = 0; i < cp->cc_nbuckets; i++)
{
dlist_mutable_iter iter;
@@ -937,21 +942,21 @@ CatCacheCleanupOldEntries(CatCache *cp)
/*
- * Calculate the duration from the time of the last access to the
- * "current" time. Since catcacheclock is not advanced within a
- * transaction, the entries that are accessed within the current
- * transaction won't be pruned.
+ * Calculate the duration from the time from the last access to
+ * the "current" time. Since catcacheclock is not advanced within
+ * a transaction, the entries that are accessed within the current
+ * transaction always get 0 as the result.
*/
TimestampDifference(ct->lastaccess, catcacheclock, &entry_age, &us);
#ifdef CATCACHE_STATS
/* count catcache entries for each age class */
ntotal++;
- for (j = 0 ;
- ageclass[j] != 0.0 &&
- entry_age > cache_prune_min_age * ageclass[j] ;
- j++);
- if (ageclass[j] == 0.0) j--;
+
+ j = 0;
+ while (j < SYSCACHE_STATS_NAGECLASSES - 1 &&
+ entry_age > cache_prune_min_age * ageclass[j])
+ j++;
nentries[j]++;
#endif
@@ -984,14 +989,17 @@ CatCacheCleanupOldEntries(CatCache *cp)
}
#ifdef CATCACHE_STATS
+ StaticAssertStmt(SYSCACHE_STATS_NAGECLASSES == 6,
+ "number of syscache age class must be 6");
ereport(DEBUG1,
- (errmsg ("removed %d/%d, age(-%.0fs:%d, -%.0fs:%d, *-%.0fs:%d, -%.0fs:%d, -%.0fs:%d) naccessed(0:%d, 1:%d,
2:%d)",
+ (errmsg ("removed %d/%d, age(-%.0fs:%d, -%.0fs:%d, *-%.0fs:%d, -%.0fs:%d, -%.0fs:%d, rest:%d)
naccessed(0:%d,1:%d, 2:%d)",
nremoved, ntotal,
ageclass[0] * cache_prune_min_age, nentries[0],
ageclass[1] * cache_prune_min_age, nentries[1],
ageclass[2] * cache_prune_min_age, nentries[2],
ageclass[3] * cache_prune_min_age, nentries[3],
ageclass[4] * cache_prune_min_age, nentries[4],
+ nentries[5],
nremoved_entry[0], nremoved_entry[1], nremoved_entry[2]),
errhidestmt(true)));
#endif
@@ -1368,9 +1376,7 @@ SearchCatCacheInternal(CatCache *cache,
if (unlikely(cache->cc_tupdesc == NULL))
CatalogCacheInitializeCache(cache);
-#ifdef CATCACHE_STATS
cache->cc_searches++;
-#endif
/* Initialize local parameter array */
arguments[0] = v1;
@@ -1430,9 +1436,7 @@ SearchCatCacheInternal(CatCache *cache,
CACHE3_elog(DEBUG2, "SearchCatCache(%s): found in bucket %d",
cache->cc_relname, hashIndex);
-#ifdef CATCACHE_STATS
cache->cc_hits++;
-#endif
return &ct->tuple;
}
@@ -1441,9 +1445,7 @@ SearchCatCacheInternal(CatCache *cache,
CACHE3_elog(DEBUG2, "SearchCatCache(%s): found neg entry in bucket %d",
cache->cc_relname, hashIndex);
-#ifdef CATCACHE_STATS
cache->cc_neg_hits++;
-#endif
return NULL;
}
@@ -1571,9 +1573,7 @@ SearchCatCacheMiss(CatCache *cache,
CACHE3_elog(DEBUG2, "SearchCatCache(%s): put in bucket %d",
cache->cc_relname, hashIndex);
-#ifdef CATCACHE_STATS
cache->cc_newloads++;
-#endif
return &ct->tuple;
}
@@ -1684,9 +1684,7 @@ SearchCatCacheList(CatCache *cache,
Assert(nkeys > 0 && nkeys < cache->cc_nkeys);
-#ifdef CATCACHE_STATS
cache->cc_lsearches++;
-#endif
/* Initialize local parameter array */
arguments[0] = v1;
@@ -1743,9 +1741,7 @@ SearchCatCacheList(CatCache *cache,
CACHE2_elog(DEBUG2, "SearchCatCacheList(%s): found list",
cache->cc_relname);
-#ifdef CATCACHE_STATS
cache->cc_lhits++;
-#endif
return cl;
}
@@ -2253,3 +2249,64 @@ PrintCatCacheListLeakWarning(CatCList *list)
list->my_cache->cc_relname, list->my_cache->id,
list, list->refcount);
}
+
+/*
+ * CatCacheGetStats - fill in SysCacheStats struct.
+ *
+ * This is a support routine for SysCacheGetStats, substantially fills in the
+ * result. The classification here is based on the same criteria to
+ * CatCacheCleanupOldEntries().
+ */
+void
+CatCacheGetStats(CatCache *cache, SysCacheStats *stats)
+{
+ int i, j;
+
+ Assert(ageclass[SYSCACHE_STATS_NAGECLASSES - 1] == 0.0);
+
+ /* fill in the stats struct */
+ stats->size = cache->cc_tupsize + cache->cc_nbuckets * sizeof(dlist_head);
+ stats->ntuples = cache->cc_ntup;
+ stats->nsearches = cache->cc_searches;
+ stats->nhits = cache->cc_hits;
+ stats->nneg_hits = cache->cc_neg_hits;
+
+ /* cache_prune_min_age can be changed on-session, fill it every time */
+ for (i = 0 ; i < SYSCACHE_STATS_NAGECLASSES ; i++)
+ stats->ageclasses[i] = (int) (cache_prune_min_age * ageclass[i]);
+
+ /*
+ * nth element in nclass_entries stores the number of cache entries that
+ * have lived unaccessed for corresponding multiple in ageclass of
+ * cache_prune_min_age.
+ */
+ memset(stats->nclass_entries, 0, sizeof(int) * SYSCACHE_STATS_NAGECLASSES);
+
+ /* Scan the whole hash */
+ for (i = 0; i < cache->cc_nbuckets; i++)
+ {
+ dlist_mutable_iter iter;
+
+ dlist_foreach_modify(iter, &cache->cc_bucket[i])
+ {
+ CatCTup *ct = dlist_container(CatCTup, cache_elem, iter.cur);
+ long entry_age;
+ int us;
+
+ /*
+ * Calculate the duration from the time from the last access to
+ * the "current" time. Since catcacheclock is not advanced within
+ * a transaction, the entries that are accessed within the current
+ * transaction won't be pruned.
+ */
+ TimestampDifference(ct->lastaccess, catcacheclock, &entry_age, &us);
+
+ j = 0;
+ while (j < SYSCACHE_STATS_NAGECLASSES - 1 &&
+ entry_age > stats->ageclasses[j])
+ j++;
+
+ stats->nclass_entries[j]++;
+ }
+ }
+}
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index ac98c19155..7b38a06708 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -20,6 +20,9 @@
*/
#include "postgres.h"
+#include <sys/stat.h>
+#include <unistd.h>
+
#include "access/htup_details.h"
#include "access/sysattr.h"
#include "catalog/indexing.h"
@@ -1534,6 +1537,27 @@ RelationSupportsSysCache(Oid relid)
return false;
}
+/*
+ * SysCacheGetStats - returns stats of specified syscache
+ *
+ * This routine returns the address of its local static memory.
+ */
+SysCacheStats *
+SysCacheGetStats(int cacheId)
+{
+ static SysCacheStats stats;
+
+ Assert(cacheId >=0 && cacheId < SysCacheSize);
+
+ memset(&stats, 0, sizeof(stats));
+
+ stats.reloid = cacheinfo[cacheId].reloid;
+ stats.indoid = cacheinfo[cacheId].indoid;
+
+ CatCacheGetStats(SysCache[cacheId], &stats);
+
+ return &stats;
+}
/*
* OID comparator for pg_qsort
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index fd51934aaf..f039ecd805 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -32,6 +32,7 @@ volatile sig_atomic_t QueryCancelPending = false;
volatile sig_atomic_t ProcDiePending = false;
volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
+volatile sig_atomic_t IdleSyscacheStatsUpdateTimeoutPending = false;
volatile sig_atomic_t ConfigReloadPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index 7415c4faab..6b0fdbbd87 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -73,6 +73,7 @@ static void ShutdownPostgres(int code, Datum arg);
static void StatementTimeoutHandler(void);
static void LockTimeoutHandler(void);
static void IdleInTransactionSessionTimeoutHandler(void);
+static void IdleSyscacheStatsUpdateTimeoutHandler(void);
static bool ThereIsAtLeastOneRole(void);
static void process_startup_options(Port *port, bool am_superuser);
static void process_settings(Oid databaseid, Oid roleid);
@@ -629,6 +630,8 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
RegisterTimeout(LOCK_TIMEOUT, LockTimeoutHandler);
RegisterTimeout(IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
IdleInTransactionSessionTimeoutHandler);
+ RegisterTimeout(IDLE_CATCACHE_UPDATE_TIMEOUT,
+ IdleSyscacheStatsUpdateTimeoutHandler);
}
/*
@@ -1240,6 +1243,14 @@ IdleInTransactionSessionTimeoutHandler(void)
SetLatch(MyLatch);
}
+static void
+IdleSyscacheStatsUpdateTimeoutHandler(void)
+{
+ IdleSyscacheStatsUpdateTimeoutPending = true;
+ InterruptPending = true;
+ SetLatch(MyLatch);
+}
+
/*
* Returns true if at least one role is defined in this database cluster.
*/
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 134c357bf3..e8d7b6998a 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -3154,6 +3154,16 @@ static struct config_int ConfigureNamesInt[] =
NULL, NULL, NULL
},
+ {
+ {"track_syscache_usage_interval", PGC_SUSET, STATS_COLLECTOR,
+ gettext_noop("Sets the interval between syscache usage collection, in milliseconds. Zero disables syscache
usagetracking."),
+ NULL
+ },
+ &pgstat_track_syscache_usage_interval,
+ 0, 0, INT_MAX / 2,
+ NULL, NULL, NULL
+ },
+
{
{"gin_pending_list_limit", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the maximum size of the pending list for GIN index."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index d82af3bd6c..4a6c9fceb5 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -554,6 +554,7 @@
#track_io_timing = off
#track_functions = none # none, pl, all
#track_activity_query_size = 1024 # (change requires restart)
+#track_syscache_usage_interval = 0 # zero disables tracking
#stats_temp_directory = 'pg_stat_tmp'
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3ecc2e12c3..11fc1f3075 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9669,6 +9669,15 @@
proargmodes => '{o,o,o,o,o,o,o,o,o,o,o}',
proargnames =>
'{slot_name,plugin,slot_type,datoid,temporary,active,active_pid,xmin,catalog_xmin,restart_lsn,confirmed_flush_lsn}',
prosrc => 'pg_get_replication_slots' },
+{ oid => '3425',
+ descr => 'syscache statistics',
+ proname => 'pg_get_syscache_stats', prorows => '100', proisstrict => 'f',
+ proretset => 't', provolatile => 'v', prorettype => 'record',
+ proargtypes => 'int4',
+ proallargtypes => '{int4,oid,oid,int8,int8,int8,int8,int8,_int4,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid,relid,indid,size,ntup,searches,hits,neg_hits,ageclass,last_update}',
+ prosrc => 'pgstat_get_syscache_stats' },
{ oid => '3786', descr => 'set up a logical replication slot',
proname => 'pg_create_logical_replication_slot', provolatile => 'v',
proparallel => 'u', prorettype => 'record', proargtypes => 'name name bool',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index c9e35003a5..69b9a976f0 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -82,6 +82,7 @@ extern PGDLLIMPORT volatile sig_atomic_t InterruptPending;
extern PGDLLIMPORT volatile sig_atomic_t QueryCancelPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
+extern PGDLLIMPORT volatile sig_atomic_t IdleSyscacheStatsUpdateTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ConfigReloadPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 313ca5f3c3..ee9968f81a 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -1134,6 +1134,7 @@ extern bool pgstat_track_activities;
extern bool pgstat_track_counts;
extern int pgstat_track_functions;
extern PGDLLIMPORT int pgstat_track_activity_query_size;
+extern int pgstat_track_syscache_usage_interval;
extern char *pgstat_stat_directory;
extern char *pgstat_stat_tmpname;
extern char *pgstat_stat_filename;
@@ -1218,7 +1219,8 @@ extern PgStat_BackendFunctionEntry *find_funcstat_entry(Oid func_id);
extern void pgstat_initstats(Relation rel);
extern char *pgstat_clip_activity(const char *raw_activity);
-
+extern void pgstat_get_syscachestat_filename(bool permanent,
+ bool tempname, int backendid, char *filename, int len);
/* ----------
* pgstat_report_wait_start() -
*
@@ -1353,5 +1355,6 @@ extern PgStat_StatFuncEntry *pgstat_fetch_stat_funcentry(Oid funcid);
extern int pgstat_fetch_stat_numbackends(void);
extern PgStat_ArchiverStats *pgstat_fetch_stat_archiver(void);
extern PgStat_GlobalStats *pgstat_fetch_global(void);
-
+extern long pgstat_write_syscache_stats(bool force);
+extern void pgstat_track_syscache_assign_hook(int newval, void *extra);
#endif /* PGSTAT_H */
diff --git a/src/include/utils/catcache.h b/src/include/utils/catcache.h
index 5d24809900..4d51975920 100644
--- a/src/include/utils/catcache.h
+++ b/src/include/utils/catcache.h
@@ -65,10 +65,8 @@ typedef struct catcache
int cc_tupsize; /* total amount of catcache tuples */
/*
- * Keep these at the end, so that compiling catcache.c with CATCACHE_STATS
- * doesn't break ABI for other modules
+ * Statistics entries
*/
-#ifdef CATCACHE_STATS
long cc_searches; /* total # searches against this cache */
long cc_hits; /* # of matches against existing entry */
long cc_neg_hits; /* # of matches against negative entry */
@@ -81,7 +79,6 @@ typedef struct catcache
long cc_invals; /* # of entries invalidated from cache */
long cc_lsearches; /* total # list-searches */
long cc_lhits; /* # of matches against existing lists */
-#endif
} CatCache;
@@ -254,4 +251,8 @@ extern void PrepareToInvalidateCacheTuple(Relation relation,
extern void PrintCatCacheLeakWarning(HeapTuple tuple);
extern void PrintCatCacheListLeakWarning(CatCList *list);
+/* defined in syscache.h */
+typedef struct syscachestats SysCacheStats;
+extern void CatCacheGetStats(CatCache *cache, SysCacheStats *syscachestats);
+
#endif /* CATCACHE_H */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 95ee48954e..71b399c902 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -112,6 +112,24 @@ enum SysCacheIdentifier
#define SysCacheSize (USERMAPPINGUSERSERVER + 1)
};
+#define SYSCACHE_STATS_NAGECLASSES 6
+/* Struct for catcache tracking information */
+typedef struct syscachestats
+{
+ Oid reloid; /* target relation */
+ Oid indoid; /* index */
+ size_t size; /* size of the catcache */
+ int ntuples; /* number of tuples resides in the catcache */
+ int nsearches; /* number of searches */
+ int nhits; /* number of cache hits */
+ int nneg_hits; /* number of negative cache hits */
+ /* age classes in seconds */
+ int ageclasses[SYSCACHE_STATS_NAGECLASSES];
+ /* number of tuples fall into the corresponding age class */
+ int nclass_entries[SYSCACHE_STATS_NAGECLASSES];
+} SysCacheStats;
+
+
extern void InitCatalogCache(void);
extern void InitCatalogCachePhase2(void);
@@ -164,6 +182,7 @@ extern void SysCacheInvalidate(int cacheId, uint32 hashValue);
extern bool RelationInvalidatesSnapshotsOnly(Oid relid);
extern bool RelationHasSysCache(Oid relid);
extern bool RelationSupportsSysCache(Oid relid);
+extern SysCacheStats *SysCacheGetStats(int cacheId);
/*
* The use of the macros below rather than direct calls to the corresponding
diff --git a/src/include/utils/timeout.h b/src/include/utils/timeout.h
index 9244a2a7b7..0ab441a364 100644
--- a/src/include/utils/timeout.h
+++ b/src/include/utils/timeout.h
@@ -31,6 +31,7 @@ typedef enum TimeoutId
STANDBY_TIMEOUT,
STANDBY_LOCK_TIMEOUT,
IDLE_IN_TRANSACTION_SESSION_TIMEOUT,
+ IDLE_CATCACHE_UPDATE_TIMEOUT,
/* First user-definable timeout reason */
USER_TIMEOUT,
/* Maximum number of timeout reasons */
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index e384cd2279..1991e75e97 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1919,6 +1919,28 @@ pg_stat_sys_tables| SELECT pg_stat_all_tables.relid,
pg_stat_all_tables.autoanalyze_count
FROM pg_stat_all_tables
WHERE ((pg_stat_all_tables.schemaname = ANY (ARRAY['pg_catalog'::name, 'information_schema'::name])) OR
(pg_stat_all_tables.schemaname~ '^pg_toast'::text));
+pg_stat_syscache| SELECT s.pid,
+ (s.relid)::regclass AS relname,
+ (s.indid)::regclass AS cache_name,
+ s.size,
+ s.ntup AS ntuples,
+ s.searches,
+ s.hits,
+ s.neg_hits,
+ s.ageclass,
+ s.last_update
+ FROM (pg_stat_activity a
+ JOIN LATERAL ( SELECT a.pid,
+ pg_get_syscache_stats.relid,
+ pg_get_syscache_stats.indid,
+ pg_get_syscache_stats.size,
+ pg_get_syscache_stats.ntup,
+ pg_get_syscache_stats.searches,
+ pg_get_syscache_stats.hits,
+ pg_get_syscache_stats.neg_hits,
+ pg_get_syscache_stats.ageclass,
+ pg_get_syscache_stats.last_update
+ FROM pg_get_syscache_stats(a.pid) pg_get_syscache_stats(relid, indid, size, ntup, searches, hits, neg_hits,
ageclass,last_update)) s ON ((a.pid = s.pid)));
pg_stat_user_functions| SELECT p.oid AS funcid,
n.nspname AS schemaname,
p.proname AS funcname,
@@ -2350,7 +2372,7 @@ pg_settings|pg_settings_n|CREATE RULE pg_settings_n AS
ON UPDATE TO pg_catalog.pg_settings DO INSTEAD NOTHING;
pg_settings|pg_settings_u|CREATE RULE pg_settings_u AS
ON UPDATE TO pg_catalog.pg_settings
- WHERE (new.name = old.name) DO SELECT set_config(old.name, new.setting, false) AS set_config;
+ WHERE (new.name = old.name) DO SELECT set_config(old.name, new.setting, false, false) AS set_config;
rtest_emp|rtest_emp_del|CREATE RULE rtest_emp_del AS
ON DELETE TO public.rtest_emp DO INSERT INTO rtest_emplog (ename, who, action, newsal, oldsal)
VALUES (old.ename, CURRENT_USER, 'fired'::bpchar, '$0.00'::money, old.salary);
--
2.16.3
From 4434b92429d9b60baed6f45bf8132a67225b0671 Mon Sep 17 00:00:00 2001
From: Kyotaro Horiguchi <horiguchi.kyotaro@lab.ntt.co.jp>
Date: Fri, 18 Jan 2019 17:16:12 +0900
Subject: [PATCH 3/3] Remote GUC setting feature and non-xact GUC config.
This adds two features at once. (will be split later).
One is non-transactional GUC setting feature. This allows setting GUC
variable set by the action GUC_ACTION_NONXACT(the name requires
condieration) survive beyond rollback. It is required by remote guc
setting to work sanely. Without the feature a remote-set value within
a trasction will disappear when involved in rollback. The only local
interface for the NONXACT action is set_config(name, value,
is_local=false, is_nonxact = true).
The second is remote guc setting feature. It uses ProcSignal to notify
the target server.
---
doc/src/sgml/config.sgml | 4 +
doc/src/sgml/func.sgml | 30 ++
src/backend/catalog/system_views.sql | 7 +-
src/backend/postmaster/pgstat.c | 3 +
src/backend/storage/ipc/ipci.c | 2 +
src/backend/storage/ipc/procsignal.c | 4 +
src/backend/tcop/postgres.c | 10 +
src/backend/utils/misc/README | 26 +-
src/backend/utils/misc/guc.c | 619 +++++++++++++++++++++++++++++++++--
src/include/catalog/pg_proc.dat | 10 +-
src/include/pgstat.h | 3 +-
src/include/storage/procsignal.h | 3 +
src/include/utils/guc.h | 13 +-
src/include/utils/guc_tables.h | 5 +-
src/test/regress/expected/guc.out | 223 +++++++++++++
src/test/regress/sql/guc.sql | 88 +++++
16 files changed, 1002 insertions(+), 48 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 6dd024340b..d024d9b069 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -281,6 +281,10 @@ UPDATE pg_settings SET setting = reset_val WHERE name = 'configuration_parameter
</listitem>
</itemizedlist>
+ <para>
+ Also values on other sessions can be set using the SQL
+ function <function>pg_set_backend_setting</function>.
+ </para>
</sect2>
<sect2>
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 4930ec17f6..aeb0c4483a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -18687,6 +18687,20 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>text</type></entry>
<entry>set parameter and return new value</entry>
</row>
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_set_backend_setting</primary>
+ </indexterm>
+ <literal><function>pg_set_backend_config(
+ <parameter>process_id</parameter>,
+ <parameter>setting_name</parameter>,
+ <parameter>new_value</parameter>)
+ </function></literal>
+ </entry>
+ <entry><type>bool</type></entry>
+ <entry>set parameter on another session</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -18741,6 +18755,22 @@ SELECT set_config('log_statement_stats', 'off', false);
------------
off
(1 row)
+</programlisting>
+ </para>
+
+ <para>
+ <function>pg_set_backend_config</function> sets the parameter
+ <parameter>setting_name</parameter> to
+ <parameter>new_value</parameter> on the other session with PID
+ <parameter>process_id</parameter>. The setting is always session-local and
+ returns true if succeeded. An example:
+<programlisting>
+SELECT pg_set_backend_config(2134, 'work_mem', '16MB');
+
+pg_set_backend_config
+------------
+ t
+(1 row)
</programlisting>
</para>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 30e2da935a..3d2e341c19 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -474,7 +474,7 @@ CREATE VIEW pg_settings AS
CREATE RULE pg_settings_u AS
ON UPDATE TO pg_settings
WHERE new.name = old.name DO
- SELECT set_config(old.name, new.setting, 'f');
+ SELECT set_config(old.name, new.setting, 'f', 'f');
CREATE RULE pg_settings_n AS
ON UPDATE TO pg_settings
@@ -1049,6 +1049,11 @@ CREATE OR REPLACE FUNCTION
RETURNS boolean STRICT VOLATILE LANGUAGE INTERNAL AS 'pg_promote'
PARALLEL SAFE;
+CREATE OR REPLACE FUNCTION set_config (
+ setting_name text, new_value text, is_local boolean, is_nonxact boolean DEFAULT false)
+ RETURNS text STRICT VOLATILE LANGUAGE internal AS 'set_config_by_name'
+ PARALLEL UNSAFE;
+
-- legacy definition for compatibility with 9.3
CREATE OR REPLACE FUNCTION
json_populate_record(base anyelement, from_json json, use_json_as_text boolean DEFAULT false)
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
index 2c0c6b343e..5d6c0edcd9 100644
--- a/src/backend/postmaster/pgstat.c
+++ b/src/backend/postmaster/pgstat.c
@@ -3707,6 +3707,9 @@ pgstat_get_wait_ipc(WaitEventIPC w)
case WAIT_EVENT_SYNC_REP:
event_name = "SyncRep";
break;
+ case WAIT_EVENT_REMOTE_GUC:
+ event_name = "RemoteGUC";
+ break;
/* no default case, so that compiler will warn */
}
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 2849e47d99..044107b354 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -148,6 +148,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, GucShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
@@ -267,6 +268,7 @@ CreateSharedMemoryAndSemaphores(bool makePrivate, int port)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ GucShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 7605b2c367..98c0f84378 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -27,6 +27,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "utils/guc.h"
/*
@@ -292,6 +293,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ if (CheckProcSignal(PROCSIG_REMOTE_GUC))
+ HandleRemoteGucSetInterrupt();
+
SetLatch(MyLatch);
latch_sigusr1_handler();
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index e7972e645f..3db2a7eacc 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3165,6 +3165,10 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ /* We don't want chage GUC variables while running a query */
+ if (RemoteGucChangePending && DoingCommandRead)
+ HandleGucRemoteChanges();
}
@@ -4201,6 +4205,12 @@ PostgresMain(int argc, char *argv[],
send_ready_for_query = false;
}
+ /*
+ * (2.5) Process some pending works.
+ */
+ if (RemoteGucChangePending)
+ HandleGucRemoteChanges();
+
/*
* (2) Allow asynchronous signals to be executed immediately if they
* come in while we are waiting for client input. (This must be
diff --git a/src/backend/utils/misc/README b/src/backend/utils/misc/README
index 6e294386f7..42ae6c1a8f 100644
--- a/src/backend/utils/misc/README
+++ b/src/backend/utils/misc/README
@@ -169,10 +169,14 @@ Entry to a function with a SET option:
Plain SET command:
If no stack entry of current level:
- Push new stack entry w/prior value and state SET
+ Push new stack entry w/prior value and state SET or
+ push new stack entry w/o value and state NONXACT.
else if stack entry's state is SAVE, SET, or LOCAL:
change stack state to SET, don't change saved value
(here we are forgetting effects of prior set action)
+ else if stack entry's state is NONXACT:
+ change stack state to NONXACT_SET, set the current value to
+ prior.
else (entry must have state SET+LOCAL):
discard its masked value, change state to SET
(here we are forgetting effects of prior SET and SET LOCAL)
@@ -185,13 +189,20 @@ SET LOCAL command:
else if stack entry's state is SAVE or LOCAL or SET+LOCAL:
no change to stack entry
(in SAVE case, SET LOCAL will be forgotten at func exit)
+ else if stack entry's state is NONXACT:
+ set current value to both prior and masked slots. set state
+ NONXACT+LOCAL.
else (entry must have state SET):
put current active into its masked slot, set state SET+LOCAL
Now set new value.
+Setting by NONXACT action (no command exists):
+ Always blow away existing stack then create a new NONXACT entry.
+
Transaction or subtransaction abort:
- Pop stack entries, restoring prior value, until top < subxact depth
+ Pop stack entries, restoring prior value unless the stack entry's
+ state is NONXACT, until top < subxact depth
Transaction or subtransaction commit (incl. successful function exit):
@@ -199,9 +210,9 @@ Transaction or subtransaction commit (incl. successful function exit):
if entry's state is SAVE:
pop, restoring prior value
- else if level is 1 and entry's state is SET+LOCAL:
+ else if level is 1 and entry's state is SET+LOCAL or NONXACT+LOCAL:
pop, restoring *masked* value
- else if level is 1 and entry's state is SET:
+ else if level is 1 and entry's state is SET or NONXACT+SET:
pop, discarding old value
else if level is 1 and entry's state is LOCAL:
pop, restoring prior value
@@ -210,9 +221,9 @@ Transaction or subtransaction commit (incl. successful function exit):
else
merge entries of level N-1 and N as specified below
-The merged entry will have level N-1 and prior = older prior, so easiest
-to keep older entry and free newer. There are 12 possibilities since
-we already handled level N state = SAVE:
+The merged entry will have level N-1 and prior = older prior, so
+easiest to keep older entry and free newer. Disregarding to NONXACT,
+here are 12 possibilities since we already handled level N state = SAVE:
N-1 N
@@ -232,6 +243,7 @@ SET+LOCAL SET discard top prior and second masked, state SET
SET+LOCAL LOCAL discard top prior, no change to stack entry
SET+LOCAL SET+LOCAL discard top prior, copy masked, state S+L
+(TODO: states involving NONXACT)
RESET is executed like a SET, but using the reset_val as the desired new
value. (We do not provide a RESET LOCAL command, but SET LOCAL TO DEFAULT
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index e8d7b6998a..5a4eaed622 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -217,6 +217,37 @@ static ConfigVariable *ProcessConfigFileInternal(GucContext context,
bool applySettings, int elevel);
+/* Enum and struct to command GUC setting to another backend */
+typedef enum
+{
+ REMGUC_VACANT,
+ REMGUC_REQUEST,
+ REMGUC_INPROCESS,
+ REMGUC_DONE,
+ REMGUC_CANCELING,
+ REMGUC_CANCELED,
+} remote_guc_status;
+
+#define GUC_REMOTE_MAX_VALUE_LEN 1024 /* an arbitrary value */
+#define GUC_REMOTE_CANCEL_TIMEOUT 5000 /* in milliseconds */
+
+typedef struct
+{
+ remote_guc_status state;
+ char name[NAMEDATALEN];
+ char value[GUC_REMOTE_MAX_VALUE_LEN];
+ int sourcepid;
+ int targetpid;
+ Oid userid;
+ bool success;
+ volatile Latch *sender_latch;
+ LWLock lock;
+} GucRemoteSetting;
+
+static GucRemoteSetting *remote_setting;
+
+volatile bool RemoteGucChangePending = false;
+
/*
* Options for enum values defined in this module.
*
@@ -3161,7 +3192,7 @@ static struct config_int ConfigureNamesInt[] =
},
&pgstat_track_syscache_usage_interval,
0, 0, INT_MAX / 2,
- NULL, NULL, NULL
+ NULL, &pgstat_track_syscache_assign_hook, NULL
},
{
@@ -4730,7 +4761,6 @@ discard_stack_value(struct config_generic *gconf, config_var_value *val)
set_extra_field(gconf, &(val->extra), NULL);
}
-
/*
* Fetch the sorted array pointer (exported for help_config.c's use ONLY)
*/
@@ -5522,6 +5552,22 @@ push_old_value(struct config_generic *gconf, GucAction action)
/* Do we already have a stack entry of the current nest level? */
stack = gconf->stack;
+
+ /* NONXACT action make existing stack useles */
+ if (action == GUC_ACTION_NONXACT)
+ {
+ while (stack)
+ {
+ GucStack *prev = stack->prev;
+
+ discard_stack_value(gconf, &stack->prior);
+ discard_stack_value(gconf, &stack->masked);
+ pfree(stack);
+ stack = prev;
+ }
+ stack = gconf->stack = NULL;
+ }
+
if (stack && stack->nest_level >= GUCNestLevel)
{
/* Yes, so adjust its state if necessary */
@@ -5529,28 +5575,63 @@ push_old_value(struct config_generic *gconf, GucAction action)
switch (action)
{
case GUC_ACTION_SET:
- /* SET overrides any prior action at same nest level */
- if (stack->state == GUC_SET_LOCAL)
+ if (stack->state == GUC_NONXACT)
{
- /* must discard old masked value */
- discard_stack_value(gconf, &stack->masked);
+ /* NONXACT rollbacks to the current value */
+ stack->scontext = gconf->scontext;
+ set_stack_value(gconf, &stack->prior);
+ stack->state = GUC_NONXACT_SET;
}
- stack->state = GUC_SET;
+ else
+ {
+ /* SET overrides other prior actions at same nest level */
+ if (stack->state == GUC_SET_LOCAL)
+ {
+ /* must discard old masked value */
+ discard_stack_value(gconf, &stack->masked);
+ }
+ stack->state = GUC_SET;
+ }
+
break;
+
case GUC_ACTION_LOCAL:
if (stack->state == GUC_SET)
{
- /* SET followed by SET LOCAL, remember SET's value */
+ /* SET followed by SET LOCAL, remember it's value */
stack->masked_scontext = gconf->scontext;
set_stack_value(gconf, &stack->masked);
stack->state = GUC_SET_LOCAL;
}
+ else if (stack->state == GUC_NONXACT)
+ {
+ /*
+ * NONXACT followed by SET LOCAL, both prior and masked
+ * are set to the current value
+ */
+ stack->scontext = gconf->scontext;
+ set_stack_value(gconf, &stack->prior);
+ stack->masked_scontext = stack->scontext;
+ stack->masked = stack->prior;
+ stack->state = GUC_NONXACT_LOCAL;
+ }
+ else if (stack->state == GUC_NONXACT_SET)
+ {
+ /* NONXACT_SET followed by SET LOCAL, set masked */
+ stack->masked_scontext = gconf->scontext;
+ set_stack_value(gconf, &stack->masked);
+ stack->state = GUC_NONXACT_LOCAL;
+ }
/* in all other cases, no change to stack entry */
break;
case GUC_ACTION_SAVE:
/* Could only have a prior SAVE of same variable */
Assert(stack->state == GUC_SAVE);
break;
+
+ case GUC_ACTION_NONXACT:
+ Assert(false);
+ break;
}
Assert(guc_dirty); /* must be set already */
return;
@@ -5566,6 +5647,7 @@ push_old_value(struct config_generic *gconf, GucAction action)
stack->prev = gconf->stack;
stack->nest_level = GUCNestLevel;
+
switch (action)
{
case GUC_ACTION_SET:
@@ -5577,10 +5659,15 @@ push_old_value(struct config_generic *gconf, GucAction action)
case GUC_ACTION_SAVE:
stack->state = GUC_SAVE;
break;
+ case GUC_ACTION_NONXACT:
+ stack->state = GUC_NONXACT;
+ break;
}
stack->source = gconf->source;
stack->scontext = gconf->scontext;
- set_stack_value(gconf, &stack->prior);
+
+ if (action != GUC_ACTION_NONXACT)
+ set_stack_value(gconf, &stack->prior);
gconf->stack = stack;
@@ -5675,22 +5762,31 @@ AtEOXact_GUC(bool isCommit, int nestLevel)
* stack entries to avoid leaking memory. If we do set one of
* those flags, unused fields will be cleaned up after restoring.
*/
- if (!isCommit) /* if abort, always restore prior value */
- restorePrior = true;
+ if (!isCommit)
+ {
+ /* GUC_NONXACT does't rollback */
+ if (stack->state != GUC_NONXACT)
+ restorePrior = true;
+ }
else if (stack->state == GUC_SAVE)
restorePrior = true;
else if (stack->nest_level == 1)
{
/* transaction commit */
- if (stack->state == GUC_SET_LOCAL)
+ if (stack->state == GUC_SET_LOCAL ||
+ stack->state == GUC_NONXACT_LOCAL)
restoreMasked = true;
- else if (stack->state == GUC_SET)
+ else if (stack->state == GUC_SET ||
+ stack->state == GUC_NONXACT_SET)
{
/* we keep the current active value */
discard_stack_value(gconf, &stack->prior);
}
- else /* must be GUC_LOCAL */
+ else if (stack->state != GUC_NONXACT)
+ {
+ /* must be GUC_LOCAL */
restorePrior = true;
+ }
}
else if (prev == NULL ||
prev->nest_level < stack->nest_level - 1)
@@ -5712,11 +5808,27 @@ AtEOXact_GUC(bool isCommit, int nestLevel)
break;
case GUC_SET:
- /* next level always becomes SET */
- discard_stack_value(gconf, &stack->prior);
- if (prev->state == GUC_SET_LOCAL)
+ if (prev->state == GUC_SET ||
+ prev->state == GUC_NONXACT_SET)
+ {
+ discard_stack_value(gconf, &stack->prior);
+ }
+ else if (prev->state == GUC_NONXACT)
+ {
+ prev->scontext = stack->scontext;
+ prev->prior = stack->prior;
+ prev->state = GUC_NONXACT_SET;
+ }
+ else if (prev->state == GUC_SET_LOCAL ||
+ prev->state == GUC_NONXACT_LOCAL)
+ {
+ discard_stack_value(gconf, &stack->prior);
discard_stack_value(gconf, &prev->masked);
- prev->state = GUC_SET;
+ if (prev->state == GUC_SET_LOCAL)
+ prev->state = GUC_SET;
+ else
+ prev->state = GUC_NONXACT_SET;
+ }
break;
case GUC_LOCAL:
@@ -5727,6 +5839,16 @@ AtEOXact_GUC(bool isCommit, int nestLevel)
prev->masked = stack->prior;
prev->state = GUC_SET_LOCAL;
}
+ else if (prev->state == GUC_NONXACT)
+ {
+ prev->prior = stack->masked;
+ prev->scontext = stack->masked_scontext;
+ prev->masked = stack->masked;
+ prev->masked_scontext = stack->masked_scontext;
+ discard_stack_value(gconf, &stack->prior);
+ discard_stack_value(gconf, &stack->masked);
+ prev->state = GUC_NONXACT_SET;
+ }
else
{
/* else just forget this stack level */
@@ -5735,15 +5857,32 @@ AtEOXact_GUC(bool isCommit, int nestLevel)
break;
case GUC_SET_LOCAL:
- /* prior state at this level no longer wanted */
- discard_stack_value(gconf, &stack->prior);
- /* copy down the masked state */
- prev->masked_scontext = stack->masked_scontext;
- if (prev->state == GUC_SET_LOCAL)
- discard_stack_value(gconf, &prev->masked);
- prev->masked = stack->masked;
- prev->state = GUC_SET_LOCAL;
+ if (prev->state == GUC_NONXACT)
+ {
+ prev->prior = stack->prior;
+ prev->masked = stack->prior;
+ discard_stack_value(gconf, &stack->prior);
+ discard_stack_value(gconf, &stack->masked);
+ prev->state = GUC_NONXACT_SET;
+ }
+ else if (prev->state != GUC_NONXACT_SET)
+ {
+ /* prior state at this level no longer wanted */
+ discard_stack_value(gconf, &stack->prior);
+ /* copy down the masked state */
+ prev->masked_scontext = stack->masked_scontext;
+ if (prev->state == GUC_SET_LOCAL)
+ discard_stack_value(gconf, &prev->masked);
+ prev->masked = stack->masked;
+ prev->state = GUC_SET_LOCAL;
+ }
break;
+ case GUC_NONXACT:
+ case GUC_NONXACT_SET:
+ case GUC_NONXACT_LOCAL:
+ Assert(false);
+ break;
+
}
}
@@ -8024,7 +8163,8 @@ set_config_by_name(PG_FUNCTION_ARGS)
char *name;
char *value;
char *new_value;
- bool is_local;
+ int set_action = GUC_ACTION_SET;
+
if (PG_ARGISNULL(0))
ereport(ERROR,
@@ -8044,18 +8184,27 @@ set_config_by_name(PG_FUNCTION_ARGS)
* Get the desired state of is_local. Default to false if provided value
* is NULL
*/
- if (PG_ARGISNULL(2))
- is_local = false;
- else
- is_local = PG_GETARG_BOOL(2);
+ if (!PG_ARGISNULL(2) && PG_GETARG_BOOL(2))
+ set_action = GUC_ACTION_LOCAL;
+
+ /*
+ * Get the desired state of is_nonxact. Default to false if provided value
+ * is NULL
+ */
+ if (!PG_ARGISNULL(3) && PG_GETARG_BOOL(3))
+ {
+ if (set_action == GUC_ACTION_LOCAL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("Only one of is_local and is_nonxact can be true")));
+ set_action = GUC_ACTION_NONXACT;
+ }
/* Note SET DEFAULT (argstring == NULL) is equivalent to RESET */
(void) set_config_option(name,
value,
(superuser() ? PGC_SUSET : PGC_USERSET),
- PGC_S_SESSION,
- is_local ? GUC_ACTION_LOCAL : GUC_ACTION_SET,
- true, 0, false);
+ PGC_S_SESSION, set_action, true, 0, false);
/* get the new current value */
new_value = GetConfigOptionByName(name, NULL, false);
@@ -8064,7 +8213,6 @@ set_config_by_name(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(cstring_to_text(new_value));
}
-
/*
* Common code for DefineCustomXXXVariable subroutines: allocate the
* new variable's config struct and fill in generic fields.
@@ -8263,6 +8411,13 @@ reapply_stacked_values(struct config_generic *variable,
WARNING, false);
break;
+ case GUC_NONXACT:
+ (void) set_config_option(name, curvalue,
+ curscontext, cursource,
+ GUC_ACTION_NONXACT, true,
+ WARNING, false);
+ break;
+
case GUC_LOCAL:
(void) set_config_option(name, curvalue,
curscontext, cursource,
@@ -8282,6 +8437,33 @@ reapply_stacked_values(struct config_generic *variable,
GUC_ACTION_LOCAL, true,
WARNING, false);
break;
+
+ case GUC_NONXACT_SET:
+ /* first, apply the masked value as SET */
+ (void) set_config_option(name, stack->masked.val.stringval,
+ stack->masked_scontext, PGC_S_SESSION,
+ GUC_ACTION_NONXACT, true,
+ WARNING, false);
+ /* then apply the current value as LOCAL */
+ (void) set_config_option(name, curvalue,
+ curscontext, cursource,
+ GUC_ACTION_SET, true,
+ WARNING, false);
+ break;
+
+ case GUC_NONXACT_LOCAL:
+ /* first, apply the masked value as SET */
+ (void) set_config_option(name, stack->masked.val.stringval,
+ stack->masked_scontext, PGC_S_SESSION,
+ GUC_ACTION_NONXACT, true,
+ WARNING, false);
+ /* then apply the current value as LOCAL */
+ (void) set_config_option(name, curvalue,
+ curscontext, cursource,
+ GUC_ACTION_LOCAL, true,
+ WARNING, false);
+ break;
+
}
/* If we successfully made a stack entry, adjust its nest level */
@@ -10260,6 +10442,373 @@ GUCArrayReset(ArrayType *array)
return newarray;
}
+Size
+GucShmemSize(void)
+{
+ Size size;
+
+ size = sizeof(GucRemoteSetting);
+
+ return size;
+}
+
+void
+GucShmemInit(void)
+{
+ Size size;
+ bool found;
+
+ size = sizeof(GucRemoteSetting);
+ remote_setting = (GucRemoteSetting *)
+ ShmemInitStruct("GUC remote setting", size, &found);
+
+ if (!found)
+ {
+ MemSet(remote_setting, 0, size);
+ LWLockInitialize(&remote_setting->lock, LWLockNewTrancheId());
+ }
+
+ LWLockRegisterTranche(remote_setting->lock.tranche, "guc_remote");
+}
+
+/*
+ * set_backend_config: SQL callable function to set GUC variable of remote
+ * session.
+ */
+Datum
+set_backend_config(PG_FUNCTION_ARGS)
+{
+ int pid = PG_GETARG_INT32(0);
+ char *name = text_to_cstring(PG_GETARG_TEXT_P(1));
+ char *value = text_to_cstring(PG_GETARG_TEXT_P(2));
+ TimestampTz cancel_start;
+ PgBackendStatus *beentry;
+ int beid;
+ int rc;
+
+ if (strlen(name) >= NAMEDATALEN)
+ ereport(ERROR,
+ (errcode(ERRCODE_NAME_TOO_LONG),
+ errmsg("name of GUC variable is too long")));
+ if (strlen(value) >= GUC_REMOTE_MAX_VALUE_LEN)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("value is too long"),
+ errdetail("Maximum acceptable length of value is %d",
+ GUC_REMOTE_MAX_VALUE_LEN - 1)));
+
+ /* find beentry for given pid */
+ beentry = NULL;
+ for (beid = 1;
+ (beentry = pgstat_fetch_stat_beentry(beid)) &&
+ beentry->st_procpid != pid ;
+ beid++);
+
+ /*
+ * This will be checked out by SendProcSignal but do here to emit
+ * appropriate message message.
+ */
+ if (!beentry)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("process PID %d not found", pid)));
+
+ /* allow only client backends */
+ if (beentry->st_backendType != B_BACKEND)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("not a client backend")));
+
+ /*
+ * Wait if someone is sending a request. We need to wait with timeout
+ * since the current user of the struct doesn't wake me up.
+ */
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+ while (remote_setting->state != REMGUC_VACANT)
+ {
+ LWLockRelease(&remote_setting->lock);
+ rc = WaitLatch(&MyProc->procLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
+ 200, PG_WAIT_ACTIVITY);
+
+ if (rc & WL_POSTMASTER_DEATH)
+ return (Datum) BoolGetDatum(false);
+
+ CHECK_FOR_INTERRUPTS();
+
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+ }
+
+ /* my turn, send a request */
+ Assert(remote_setting->state == REMGUC_VACANT);
+
+ remote_setting->state = REMGUC_REQUEST;
+ remote_setting->sourcepid = MyProcPid;
+ remote_setting->targetpid = pid;
+ remote_setting->userid = GetUserId();
+
+ strncpy(remote_setting->name, name, NAMEDATALEN);
+ remote_setting->name[NAMEDATALEN - 1] = 0;
+ strncpy(remote_setting->value, value, GUC_REMOTE_MAX_VALUE_LEN);
+ remote_setting->value[GUC_REMOTE_MAX_VALUE_LEN - 1] = 0;
+ remote_setting->sender_latch = MyLatch;
+
+ LWLockRelease(&remote_setting->lock);
+
+ if (SendProcSignal(pid, PROCSIG_REMOTE_GUC, InvalidBackendId) < 0)
+ {
+ remote_setting->state = REMGUC_VACANT;
+ ereport(ERROR,
+ (errmsg("could not signal backend with PID %d: %m", pid)));
+ }
+
+ /*
+ * This request is processed only while idle time of peer so it may take a
+ * long time before we get a response.
+ */
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+ while (remote_setting->state != REMGUC_DONE)
+ {
+ LWLockRelease(&remote_setting->lock);
+ rc = WaitLatch(&MyProc->procLatch,
+ WL_LATCH_SET | WL_POSTMASTER_DEATH,
+ -1, PG_WAIT_ACTIVITY);
+
+ /* don't care of the state in the case.. */
+ if (rc & WL_POSTMASTER_DEATH)
+ return (Datum) BoolGetDatum(false);
+
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+
+ /* get out if we got a query cancel request */
+ if (QueryCancelPending)
+ break;
+ }
+
+ /*
+ * Cancel the requset if possible. We cannot cancel the request in the
+ * case peer have processed it. We don't see QueryCancelPending but the
+ * request status so that the case is handled properly.
+ */
+ if (remote_setting->state == REMGUC_REQUEST)
+ {
+ Assert(QueryCancelPending);
+
+ remote_setting->state = REMGUC_CANCELING;
+ LWLockRelease(&remote_setting->lock);
+
+ if (SendProcSignal(pid,
+ PROCSIG_REMOTE_GUC, InvalidBackendId) < 0)
+ {
+ remote_setting->state = REMGUC_VACANT;
+ ereport(ERROR,
+ (errmsg("could not signal backend with PID %d: %m",
+ pid)));
+ }
+
+ /* Peer must respond shortly, don't sleep for a long time. */
+
+ cancel_start = GetCurrentTimestamp();
+
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+ while (remote_setting->state != REMGUC_CANCELED &&
+ !TimestampDifferenceExceeds(cancel_start, GetCurrentTimestamp(),
+ GUC_REMOTE_CANCEL_TIMEOUT))
+ {
+ LWLockRelease(&remote_setting->lock);
+ rc = WaitLatch(&MyProc->procLatch,
+ WL_LATCH_SET | WL_TIMEOUT | WL_POSTMASTER_DEATH,
+ GUC_REMOTE_CANCEL_TIMEOUT, PG_WAIT_ACTIVITY);
+
+ /* don't care of the state in the case.. */
+ if (rc & WL_POSTMASTER_DEATH)
+ return (Datum) BoolGetDatum(false);
+
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+ }
+
+ if (remote_setting->state != REMGUC_CANCELED)
+ {
+ remote_setting->state = REMGUC_VACANT;
+ ereport(ERROR, (errmsg("failed cancelling remote GUC request")));
+ }
+
+ remote_setting->state = REMGUC_VACANT;
+ LWLockRelease(&remote_setting->lock);
+
+ ereport(INFO,
+ (errmsg("remote GUC change request to PID %d is canceled",
+ pid)));
+
+ return (Datum) BoolGetDatum(false);
+ }
+
+ Assert (remote_setting->state == REMGUC_DONE);
+
+ /* ereport exits on query cancel, we need this before that */
+ remote_setting->state = REMGUC_VACANT;
+
+ if (QueryCancelPending)
+ ereport(INFO,
+ (errmsg("remote GUC change request to PID %d already completed",
+ pid)));
+
+ if (!remote_setting->success)
+ ereport(ERROR,
+ (errmsg("%s", remote_setting->value)));
+
+ LWLockRelease(&remote_setting->lock);
+
+ return (Datum) BoolGetDatum(true);
+}
+
+
+void
+HandleRemoteGucSetInterrupt(void)
+{
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+
+ /* check if any request is being sent to me */
+ if (remote_setting->targetpid == MyProcPid)
+ {
+ switch (remote_setting->state)
+ {
+ case REMGUC_REQUEST:
+ InterruptPending = true;
+ RemoteGucChangePending = true;
+ break;
+ case REMGUC_CANCELING:
+ InterruptPending = true;
+ RemoteGucChangePending = true;
+ remote_setting->state = REMGUC_CANCELED;
+ SetLatch(remote_setting->sender_latch);
+ break;
+ default:
+ break;
+ }
+ }
+ LWLockRelease(&remote_setting->lock);
+}
+
+void
+HandleGucRemoteChanges(void)
+{
+ MemoryContext currentcxt = CurrentMemoryContext;
+ bool canceling = false;
+ bool process_request = true;
+ int saveInterruptHoldoffCount = 0;
+ int saveQueryCancelHoldoffCount = 0;
+
+ RemoteGucChangePending = false;
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+
+ /* skip if this request is no longer for me */
+ if (remote_setting->targetpid != MyProcPid)
+ process_request = false;
+ else
+ {
+ switch (remote_setting->state)
+ {
+ case REMGUC_REQUEST:
+ remote_setting->state = REMGUC_INPROCESS;
+ break;
+ case REMGUC_CANCELING:
+ /*
+ * This request is already canceled but entered this function
+ * before receiving signal. Cancel the request here.
+ */
+ remote_setting->state = REMGUC_CANCELED;
+ remote_setting->success = false;
+ canceling = true;
+ break;
+ case REMGUC_VACANT:
+ case REMGUC_CANCELED:
+ case REMGUC_INPROCESS:
+ case REMGUC_DONE:
+ /* Just ignore the cases */
+ process_request = false;
+ break;
+ }
+ }
+
+ LWLockRelease(&remote_setting->lock);
+
+ if (!process_request)
+ return;
+
+ if (canceling)
+ {
+ SetLatch(remote_setting->sender_latch);
+ return;
+ }
+
+
+ /* Okay, actually modify variable */
+ remote_setting->success = true;
+
+ PG_TRY();
+ {
+ bool has_privilege;
+ bool is_superuser;
+ bool end_transaction = false;
+ /*
+ * XXXX: ERROR resets the following varialbes but we don't want that.
+ */
+ saveInterruptHoldoffCount = InterruptHoldoffCount;
+ saveQueryCancelHoldoffCount = QueryCancelHoldoffCount;
+
+ /* superuser_arg requires a transaction */
+ if (!IsTransactionState())
+ {
+ StartTransactionCommand();
+ end_transaction = true;
+ }
+ is_superuser = superuser_arg(remote_setting->userid);
+ has_privilege = is_superuser ||
+ has_privs_of_role(remote_setting->userid, GetUserId());
+
+ if (end_transaction)
+ CommitTransactionCommand();
+
+ if (!has_privilege)
+ elog(ERROR, "role %u is not allowed to set GUC variables on the session with PID %d",
+ remote_setting->userid, MyProcPid);
+
+ (void) set_config_option(remote_setting->name, remote_setting->value,
+ is_superuser ? PGC_SUSET : PGC_USERSET,
+ PGC_S_SESSION, GUC_ACTION_NONXACT,
+ true, ERROR, false);
+ }
+ PG_CATCH();
+ {
+ ErrorData *errdata;
+ MemoryContextSwitchTo(currentcxt);
+ errdata = CopyErrorData();
+ remote_setting->success = false;
+ strncpy(remote_setting->value, errdata->message,
+ GUC_REMOTE_MAX_VALUE_LEN);
+ remote_setting->value[GUC_REMOTE_MAX_VALUE_LEN - 1] = 0;
+ FlushErrorState();
+
+ /* restore the saved value */
+ InterruptHoldoffCount = saveInterruptHoldoffCount ;
+ QueryCancelHoldoffCount = saveQueryCancelHoldoffCount;
+
+ }
+ PG_END_TRY();
+
+ ereport(LOG,
+ (errmsg("GUC variable \"%s\" is changed to \"%s\" by request from another backend with PID %d",
+ remote_setting->name, remote_setting->value,
+ remote_setting->sourcepid)));
+
+ LWLockAcquire(&remote_setting->lock, LW_EXCLUSIVE);
+ remote_setting->state = REMGUC_DONE;
+ LWLockRelease(&remote_setting->lock);
+
+ SetLatch(remote_setting->sender_latch);
+}
+
/*
* Validate a proposed option setting for GUCArrayAdd/Delete/Reset.
*
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 11fc1f3075..54d0c3917e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -5700,8 +5700,8 @@
proargtypes => 'text bool', prosrc => 'show_config_by_name_missing_ok' },
{ oid => '2078', descr => 'SET X as a function',
proname => 'set_config', proisstrict => 'f', provolatile => 'v',
- proparallel => 'u', prorettype => 'text', proargtypes => 'text text bool',
- prosrc => 'set_config_by_name' },
+ proparallel => 'u', prorettype => 'text',
+ proargtypes => 'text text bool bool', prosrc => 'set_config_by_name' },
{ oid => '2084', descr => 'SHOW ALL as a function',
proname => 'pg_show_all_settings', prorows => '1000', proretset => 't',
provolatile => 's', prorettype => 'record', proargtypes => '',
@@ -9678,6 +9678,12 @@
proargmodes => '{i,o,o,o,o,o,o,o,o,o}',
proargnames => '{pid,relid,indid,size,ntup,searches,hits,neg_hits,ageclass,last_update}',
prosrc => 'pgstat_get_syscache_stats' },
+{ oid => '3424',
+ descr => 'set config of another backend',
+ proname => 'pg_set_backend_config', proisstrict => 'f',
+ proretset => 'f', provolatile => 'v', proparallel => 'u',
+ prorettype => 'bool', proargtypes => 'int4 text text',
+ prosrc => 'set_backend_config' },
{ oid => '3786', descr => 'set up a logical replication slot',
proname => 'pg_create_logical_replication_slot', provolatile => 'v',
proparallel => 'u', prorettype => 'record', proargtypes => 'name name bool',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index ee9968f81a..70b926a8d1 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -833,7 +833,8 @@ typedef enum
WAIT_EVENT_REPLICATION_ORIGIN_DROP,
WAIT_EVENT_REPLICATION_SLOT_DROP,
WAIT_EVENT_SAFE_SNAPSHOT,
- WAIT_EVENT_SYNC_REP
+ WAIT_EVENT_SYNC_REP,
+ WAIT_EVENT_REMOTE_GUC
} WaitEventIPC;
/* ----------
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 9f2f965d5c..040877f5eb 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -42,6 +42,9 @@ typedef enum
PROCSIG_RECOVERY_CONFLICT_BUFFERPIN,
PROCSIG_RECOVERY_CONFLICT_STARTUP_DEADLOCK,
+ /* Remote GUC setting */
+ PROCSIG_REMOTE_GUC,
+
NUM_PROCSIGNALS /* Must be last! */
} ProcSignalReason;
diff --git a/src/include/utils/guc.h b/src/include/utils/guc.h
index c07e7b945e..1e12773906 100644
--- a/src/include/utils/guc.h
+++ b/src/include/utils/guc.h
@@ -193,7 +193,8 @@ typedef enum
/* Types of set_config_option actions */
GUC_ACTION_SET, /* regular SET command */
GUC_ACTION_LOCAL, /* SET LOCAL command */
- GUC_ACTION_SAVE /* function SET option, or temp assignment */
+ GUC_ACTION_SAVE, /* function SET option, or temp assignment */
+ GUC_ACTION_NONXACT /* transactional setting */
} GucAction;
#define GUC_QUALIFIER_SEPARATOR '.'
@@ -269,6 +270,8 @@ extern int tcp_keepalives_idle;
extern int tcp_keepalives_interval;
extern int tcp_keepalives_count;
+extern volatile bool RemoteGucChangePending;
+
#ifdef TRACE_SORT
extern bool trace_sort;
#endif
@@ -276,6 +279,11 @@ extern bool trace_sort;
/*
* Functions exported by guc.c
*/
+extern Size GucShmemSize(void);
+extern void GucShmemInit(void);
+extern Datum set_backend_setting(PG_FUNCTION_ARGS);
+extern void HandleRemoteGucSetInterrupt(void);
+extern void HandleGucRemoteChanges(void);
extern void SetConfigOption(const char *name, const char *value,
GucContext context, GucSource source);
@@ -395,6 +403,9 @@ extern Size EstimateGUCStateSpace(void);
extern void SerializeGUCState(Size maxsize, char *start_address);
extern void RestoreGUCState(void *gucstate);
+/* Remote GUC setting */
+extern void HandleGucRemoteChanges(void);
+
/* Support for messages reported from GUC check hooks */
extern PGDLLIMPORT char *GUC_check_errmsg_string;
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
index a0970b2e1c..c00520e90c 100644
--- a/src/include/utils/guc_tables.h
+++ b/src/include/utils/guc_tables.h
@@ -115,7 +115,10 @@ typedef enum
GUC_SAVE, /* entry caused by function SET option */
GUC_SET, /* entry caused by plain SET command */
GUC_LOCAL, /* entry caused by SET LOCAL command */
- GUC_SET_LOCAL /* entry caused by SET then SET LOCAL */
+ GUC_NONXACT, /* entry caused by non-transactional ops */
+ GUC_SET_LOCAL, /* entry caused by SET then SET LOCAL */
+ GUC_NONXACT_SET, /* entry caused by NONXACT then SET */
+ GUC_NONXACT_LOCAL /* entry caused by NONXACT then (SET)LOCAL */
} GucStackState;
typedef struct guc_stack
diff --git a/src/test/regress/expected/guc.out b/src/test/regress/expected/guc.out
index b0d7351145..2d19697a8c 100644
--- a/src/test/regress/expected/guc.out
+++ b/src/test/regress/expected/guc.out
@@ -476,6 +476,229 @@ SELECT '2006-08-13 12:34:56'::timestamptz;
2006-08-13 12:34:56-07
(1 row)
+-- NONXACT followed by SET, SET LOCAL through COMMIT
+BEGIN;
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+ set_config
+------------
+ 128kB
+(1 row)
+
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SHOW work_mem; -- must see 512kB
+ work_mem
+----------
+ 512kB
+(1 row)
+
+COMMIT;
+SHOW work_mem; -- must see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+-- NONXACT followed by SET, SET LOCAL through ROLLBACK
+BEGIN;
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+ set_config
+------------
+ 128kB
+(1 row)
+
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SHOW work_mem; -- must see 512kB
+ work_mem
+----------
+ 512kB
+(1 row)
+
+ROLLBACK;
+SHOW work_mem; -- must see 128kB
+ work_mem
+----------
+ 128kB
+(1 row)
+
+-- SET, SET LOCAL followed by NONXACT through COMMIT
+BEGIN;
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+ set_config
+------------
+ 128kB
+(1 row)
+
+SHOW work_mem; -- must see 128kB
+ work_mem
+----------
+ 128kB
+(1 row)
+
+COMMIT;
+SHOW work_mem; -- must see 128kB
+ work_mem
+----------
+ 128kB
+(1 row)
+
+-- SET, SET LOCAL followed by NONXACT through ROLLBACK
+BEGIN;
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+ set_config
+------------
+ 128kB
+(1 row)
+
+SHOW work_mem; -- must see 128kB
+ work_mem
+----------
+ 128kB
+(1 row)
+
+ROLLBACK;
+SHOW work_mem; -- must see 128kB
+ work_mem
+----------
+ 128kB
+(1 row)
+
+-- NONXACT and SAVEPOINT
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+ set_config
+------------
+ 256kB
+(1 row)
+
+SHOW work_mem;
+ work_mem
+----------
+ 256kB
+(1 row)
+
+SET LOCAL work_mem TO '384kB';
+RELEASE SAVEPOINT a;
+SHOW work_mem; -- will see 384kB
+ work_mem
+----------
+ 384kB
+(1 row)
+
+COMMIT;
+SHOW work_mem; -- will see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+--
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+ set_config
+------------
+ 256kB
+(1 row)
+
+SHOW work_mem;
+ work_mem
+----------
+ 256kB
+(1 row)
+
+SET LOCAL work_mem TO '384kB';
+ROLLBACK TO SAVEPOINT a;
+SHOW work_mem; -- will see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+ROLLBACK;
+SHOW work_mem; -- will see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+--
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SET LOCAL work_mem TO '384kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+ set_config
+------------
+ 256kB
+(1 row)
+
+SHOW work_mem;
+ work_mem
+----------
+ 256kB
+(1 row)
+
+SET LOCAL work_mem TO '384kB';
+RELEASE SAVEPOINT a;
+SHOW work_mem; -- will see 384kB
+ work_mem
+----------
+ 384kB
+(1 row)
+
+ROLLBACK;
+SHOW work_mem; -- will see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+--
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SET LOCAL work_mem TO '384kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+ set_config
+------------
+ 256kB
+(1 row)
+
+SHOW work_mem;
+ work_mem
+----------
+ 256kB
+(1 row)
+
+SET LOCAL work_mem TO '384kB';
+ROLLBACK TO SAVEPOINT a;
+SHOW work_mem; -- will see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+COMMIT;
+SHOW work_mem; -- will see 256kB
+ work_mem
+----------
+ 256kB
+(1 row)
+
+SET work_mem TO DEFAULT;
--
-- Test RESET. We use datestyle because the reset value is forced by
-- pg_regress, so it doesn't depend on the installation's configuration.
diff --git a/src/test/regress/sql/guc.sql b/src/test/regress/sql/guc.sql
index 3b854ac496..bbb91aaa98 100644
--- a/src/test/regress/sql/guc.sql
+++ b/src/test/regress/sql/guc.sql
@@ -133,6 +133,94 @@ SHOW vacuum_cost_delay;
SHOW datestyle;
SELECT '2006-08-13 12:34:56'::timestamptz;
+-- NONXACT followed by SET, SET LOCAL through COMMIT
+BEGIN;
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SHOW work_mem; -- must see 512kB
+COMMIT;
+SHOW work_mem; -- must see 256kB
+
+-- NONXACT followed by SET, SET LOCAL through ROLLBACK
+BEGIN;
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SHOW work_mem; -- must see 512kB
+ROLLBACK;
+SHOW work_mem; -- must see 128kB
+
+-- SET, SET LOCAL followed by NONXACT through COMMIT
+BEGIN;
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+SHOW work_mem; -- must see 128kB
+COMMIT;
+SHOW work_mem; -- must see 128kB
+
+-- SET, SET LOCAL followed by NONXACT through ROLLBACK
+BEGIN;
+SET work_mem to '256kB';
+SET LOCAL work_mem to '512kB';
+SELECT set_config('work_mem', '128kB', false, true); -- NONXACT
+SHOW work_mem; -- must see 128kB
+ROLLBACK;
+SHOW work_mem; -- must see 128kB
+
+-- NONXACT and SAVEPOINT
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+SHOW work_mem;
+SET LOCAL work_mem TO '384kB';
+RELEASE SAVEPOINT a;
+SHOW work_mem; -- will see 384kB
+COMMIT;
+SHOW work_mem; -- will see 256kB
+--
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+SHOW work_mem;
+SET LOCAL work_mem TO '384kB';
+ROLLBACK TO SAVEPOINT a;
+SHOW work_mem; -- will see 256kB
+ROLLBACK;
+SHOW work_mem; -- will see 256kB
+--
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SET LOCAL work_mem TO '384kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+SHOW work_mem;
+SET LOCAL work_mem TO '384kB';
+RELEASE SAVEPOINT a;
+SHOW work_mem; -- will see 384kB
+ROLLBACK;
+SHOW work_mem; -- will see 256kB
+--
+SET work_mem TO '64kB';
+BEGIN;
+SET work_mem TO '128kB';
+SET LOCAL work_mem TO '384kB';
+SAVEPOINT a;
+SELECT set_config('work_mem', '256kB', false, true); -- NONXACT
+SHOW work_mem;
+SET LOCAL work_mem TO '384kB';
+ROLLBACK TO SAVEPOINT a;
+SHOW work_mem; -- will see 256kB
+COMMIT;
+SHOW work_mem; -- will see 256kB
+
+SET work_mem TO DEFAULT;
--
-- Test RESET. We use datestyle because the reset value is forced by
-- pg_regress, so it doesn't depend on the installation's configuration.
--
2.16.3
pgsql-hackers by date: