Delay Memoize hashtable build until executor run - Mailing list pgsql-hackers

From David Rowley
Subject Delay Memoize hashtable build until executor run
Date
Msg-id CAApHDvoJktJ5XL=Kjh2a2TFr64R-7eQZV-+jcJrUwoES2GLiWg@mail.gmail.com
Whole thread Raw
Responses Re: Delay Memoize hashtable build until executor run
List pgsql-hackers
Currently, nodeMemoize.c builds the hashtable for the cache during
executor startup.  This is not what is done in hash joins. I think we
should make the two behave the same way.

Per [1] and the corresponding discussion leading to that, making a
possibly large allocation at executor startup can lead to excessively
long EXPLAIN (not EXPLAIN ANALYZE) times.  This can confuse users as
we don't mention in EXPLAIN where the time is being spent.

Although there's not yet any conclusion that Memoize is to blame,
there's another report from someone confused about where this time is
being spent in [2].

Working on the Memoize code, I originally created the hash table
during executor startup to save on having to check we have a table
each time the node is executed.  However, the branch for this should
be quite predictable and I doubt it'll add any overhead that we would
notice.

The patch to do this is attached.

David

[1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=1e731ed12aa
[2] https://postgr.es/m/61e642df-5f48-4e4e-b4c3-58936f90ddaa@thefreecat.org

Attachment

pgsql-hackers by date:

Previous
From: Yugo NAGATA
Date:
Subject: Re: Small fix on COPY ON_ERROR document
Next
From: "David G. Johnston"
Date:
Subject: Re: Small fix on COPY ON_ERROR document