While working on eliminating potential deadlock risks of WAIT command, an agent noticed a similar risk for creating logical replication slots on hot standby. Here's how it goes:
A statement such as:
SELECT * FROM pg_create_logical_replication_slot('s', 'pgoutput');
retains its executor-registered statement snapshot while DecodingContextFindStartpoint() searches for a consistent decoding point. Its WAL reader, read_local_xlog_page_guts(), limits reads to GetXLogReplayRecPtr().
There seems to be a problematic ordering: 1) The slot-creation statement holds a snapshot and starts waiting for an additional WAL record to be replayed. 2) The operation in primary generates a cleanup record conflicting with that snapshot, for example through UPDATE followed by VACUUM while hot_standby_feedback is off
3) A running-transactions record needed to complete slot initialization, which follows the cleanup record
4) Startup process waits for the slot creator's snapshot before replaying cleanup, while the creator waits for the replay to advance
This seems serious if max_standby_streaming_delay = -1, since nothing left to break it automatically. I haven't found a report in the mailing lists since the debut of this feature. The reason for that could be an encounterance of it requires a fair coincidence of several factors and the symptom of being affected is manifested an interval of delay, which could be masked by the creation of the slot with finite delay setting
I don't see a clear solution to this potential issue, because the interface is a function, which means that the held snapshots cannot be popped cleanly since they belong to the surrounding executor.
Slot synchronization also has a replay-wait path that may participate in these cycles. I haven't looked into it yet.
