[WIP] pg_restore -j with IAM authentication fails on reconnect (expiring credentials) - Mailing list pgsql-hackers

From Fabrizio Castelli
Subject [WIP] pg_restore -j with IAM authentication fails on reconnect (expiring credentials)
Date
Msg-id CAEEjhOLiD=WwLko78MZr+N3cLGSR2_kxe0bEf0HNdMrd=R4B4A@mail.gmail.com
Whole thread
List pgsql-hackers
Hello hackers,

I'd like design feedback on a problem I'm hitting with parallel restores and short-lived credentials.

Context: Cloud IAM authentication (AWS RDS/Aurora PostgreSQL) uses tokens that are valid for a fixed window — 15 minutes — supplied through the password channel.

Problem: A parallel pg_restore -j N that runs longer than the token lifetime fails on the leader's postfork reconnect with FATAL: ... authentication failed.
The jobs uses to reconnect and the reconnect reuses the password cached at first connect (AH->savedPassword), and an explicit password makes libpq skip PGPASSFILE/PGPASSWORD resolution — so a freshly refreshed token is ignored and the expired one is replayed.
This is correct for static passwords but incompatible with expiring credentials (like IAM token)

Possible solutions:
  1. An opt-in option to re-resolve the password on reconnect: pass no explicit password and let libpq resolve per-connection, with the user keeping a current token in the passfile/env. Default would preserve current behavior (e.g. --no-reuse-password / --reconnect-password=reuse|resolve).
  2. Alternatively, avoid the leader reconnect so no re-auth is needed — though this may affect resource usage and cuts against the current phased design, so I see it as the weaker option.

Could this direction be agreed ?
At the moment this issue is blocking the possibility to work with IAM and having multiple jobs for restoring a db (at the moment this means to make a choice between security and performance)

Thanks a lot in advance,
Fabrizio

pgsql-hackers by date:

Previous
From: baotiao@gmail.com
Date:
Subject: Partition-aware simplification of constant IN lists after partition pruning
Next
From: Shubhra Jain
Date:
Subject: Re: Prevent premature startup of pg_upgrade targets