Re: speed up a logical replica setup - Mailing list pgsql-hackers

From Euler Taveira
Subject Re: speed up a logical replica setup
Date
Msg-id 8d8d9960-8130-4216-b0e4-adf36b8b1c43@app.fastmail.com
Whole thread Raw
In response to RE: speed up a logical replica setup  ("Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>)
Responses RE: speed up a logical replica setup
List pgsql-hackers
On Thu, Feb 1, 2024, at 9:47 AM, Hayato Kuroda (Fujitsu) wrote:
I made fix patches to solve reported issues by Fabrízio.

* v13-0001: Same as v11-0001 made by Euler.
* v13-0002: Fixes ERRORs while dropping replication slots [1].
           If you want to see codes which we get agreement, please apply until 0002.
=== experimental patches ===
* v13-0003: Avoids to use replication connections. The issue [2] was solved on my env.
* v13-0004: Removes -P option and use primary_conninfo instead.
* v13-0005: Refactors data structures

Thanks for rebasing the proposed patches. I'm attaching a new patch.

As I said in the previous email [1] I fixed some issues from your previous review:

* use pg_fatal() if possible. There are some cases that it couldn't replace
  pg_log_error() + exit(1) because it requires a hint.
* pg_resetwal output. Send standard output to /dev/null to avoid extra message.
* check privileges. Make sure the current role can execute CREATE SUBSCRIPTION
  and pg_replication_origin_advance().
* log directory. Refactor code that setup the log file used as server log.
* run with restricted token (Windows).
* v13-0002. Merged.
* v13-0003. I applied a modified version. I returned only the required
  information for each query.
* group command-line options into a new struct CreateSubscriberOptions. The
  exception is the dry_run option.
* rename functions that obtain system identifier.

WIP

I'm still working on the data structures to group options. I don't like the way
it was grouped in v13-0005. There is too many levels to reach database name.
The setup_subscriber() function requires the 3 data structures.

The documentation update is almost there. I will include the modifications in
the next patch.

Regarding v13-0004, it seems a good UI that's why I wrote a comment about it.
However, it comes with a restriction that requires a similar HBA rule for both
regular and replication connections. Is it an acceptable restriction? We might
paint ourselves into the corner. A reasonable proposal is not to remove this
option. Instead, it should be optional. If it is not provided, primary_conninfo
is used.



--
Euler Taveira

Attachment

pgsql-hackers by date:

Previous
From: Tom Lane
Date:
Subject: Re: POC: GROUP BY optimization
Next
From: James Coleman
Date:
Subject: Re: set_cheapest without checking pathlist