Thread: Amazon RDS auth tokens in .pgpass

Nicholas Chammas <nicholas.chammas@gmail.com> writes:
> I am trying to connect to a Postgres database on Amazon RDS using IAM
> authentication. This works by having IAM generate an authentication token
> that you use as the password for the database user you're connecting as.
> You can read more about this here:
> https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.AWSCLI.PostgreSQL.html
> Here's an example of what one of these auth tokens looks like (with some of
> the sections redacted/altered from a real token):

> ```
>
some-host.us-east-1.rds.amazonaws.com:5432/?DBUser=someuser&Action=connect&X-Amz-SignedHeaders=host&X-Amz-Security-Token=FwoGZXIvYXdzEHcaDD1hC2q3GGNNaftOvCLRAetaDArXOt6kpr1Ac83hzwtPxEojvZzARJN%2Ftys%2BkLnxsP6FmHmIMmMERWeGBiJmNcUyXWYY%2BSU9oduSSeAv%2BCpYy028Cep%2Bpyl1Km3B5axPAA2q0L4NWa41LQOayWF8F7%2FlB540%2B0aSkZ%2BIucM%2BLZXTcQl3Q0nJiIgu65lhuME4q3Mvst1ZEaZWfUegCWGaX0npajiKbNU2Ut3FH%2F6046RxlNwO4jg4vteTcPk%2BfgMjuAkf5gFR9EYMTziRx1dJGJn8VGFLTugGVITQI%2FEC1iuqKNb%2Fs%2FoFMisfzgF1nd7kxcZYYAmQtugBnDuJuIboYwYAXI2qn7HDbhgQ5v%2FwPWfZieWO&X-Amz-Credential=ABCDEFGHIJKLMNOPQRST%2F20200831%2Fus-east-1%2Frds-db%2Faws4_request&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Expires=900&X-Amz-Date=20200831T163708Z&X-Amz-Signature=fb41c59dcaf4e013eda6885a4d456549366d2612ecf06307d2443b6a44461ab7
> ```

Ugh :-(

> It's only the combination of a) .pgpass and b) RDS auth token that doesn't
> work. This leads me to believe something is going wrong on a code path
> specific to .pgpass, or that there are some undocumented restrictions on
> what can go in .pgpass.

Looking at the source code, there's a hard-wired restriction that lines of
.pgpass can't be more than 320 characters long (well, NAMEDATALEN*5, but
very few builds don't have NAMEDATALEN=64).  I see that somebody very
recently added code to make libpq print a warning for overlength lines,
but I wonder why they didn't just, um, remove the restriction.  We had
not previously heard of a use-case for passwords with hundreds of
characters in them, but I guess we need to cope.

If you're in a position to rebuild libpq, could you check that changing
LINELEN in fe-connect.c to something large enough (like 1K) fixes your
problem?  While that's clearly one issue, it'd be good to verify that
there's not another one lurking behind it.

            regards, tom lane

Stephen Frost <sfrost@snowman.net> writes:
> My memory might be faulty, but I've got some specific recollection of
> people complaining about this before and the community response being
> "who would ever need such very long things?!"

Yeah, so I guessed wrong ;-).  Never too late to absorb new facts though.

That thread does point out that passwordFromFile() is far from the
only place that assumes passwords aren't going to be longer than
what would be sane to enter manually.  I wonder whether we need to
worry about the other bottlenecks.

            regards, tom lane

On 8/31/20 1:21 PM, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
>> My memory might be faulty, but I've got some specific recollection of
>> people complaining about this before and the community response being
>> "who would ever need such very long things?!"
> Yeah, so I guessed wrong ;-).  Never too late to absorb new facts though.
>
> That thread does point out that passwordFromFile() is far from the
> only place that assumes passwords aren't going to be longer than
> what would be sane to enter manually.  I wonder whether we need to
> worry about the other bottlenecks.

Would it be better to add a .pgtoken file which gets examined (and bypasses 
those other length restrictions) if the user isn't in .pgpass?  Or is that a 
feature which would have to wait for v14?

-- 
Angular momentum makes the world go 'round.

Stephen Frost <sfrost@snowman.net> writes:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> That thread does point out that passwordFromFile() is far from the
>> only place that assumes passwords aren't going to be longer than
>> what would be sane to enter manually.  I wonder whether we need to
>> worry about the other bottlenecks.

> Well, as I said in that thread two years ago, seems like we should make
> it work everywhere and be consistent between frontend and backend
> regarding what's supported.  Perhaps even clearly document what the
> limit is too...

In the case of passwordFromFile(), the line doesn't only contain a
password.  There's also a hostname that has no a-priori upper length,
and some other fields too; not to mention that if we have a convention
for comments then it's unfriendly to have an a-priori upper length for
comment lines.  So I'm thinking that the correct thing to do in
passwordFromFile() is use an expansible buffer and fail only on OOM.
There remains, though, the question of whether any of the other limits
are problematic.

            regards, tom lane

Nicholas Chammas <nicholas.chammas@gmail.com> writes:
> On Mon, Aug 31, 2020 at 2:04 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Looking at the source code, there's a hard-wired restriction that lines of
>> .pgpass can't be more than 320 characters long (well, NAMEDATALEN*5, but
>> very few builds don't have NAMEDATALEN=64).  I see that somebody very
>> recently added code to make libpq print a warning for overlength lines,
>> but I wonder why they didn't just, um, remove the restriction.  We had
>> not previously heard of a use-case for passwords with hundreds of
>> characters in them, but I guess we need to cope.

> Just FYI, the auth tokens generated by Amazon RDS appear to be 796 bytes
> long.

Thanks.  I've pushed a fix to remove libpq's undocumented restriction
on the length of a .pgpass line.  It will be in November's releases.

            regards, tom lane