Thread: BUG #18856: Include directives for postgresql.conf, pg_hba.conf, pg_ident.conf behave inconsistently.

The following bug has been logged on the website:

Bug reference:      18856
Logged by:          Robert Pufky
Email address:      rpufky@gmail.com
PostgreSQL version: 17.4
Operating system:   Debian Bookworm
Description:

This is a nuanced error; please read the entire report. Only changed lines
are shown.

Effectively, the include_dir, include_if_exists, and include directives
behave differently for each of the files, even though there is no
documentation stating any deviation in behavior. My working assumption is
that the behavior in postgresql.conf for those directives carries over to
pg_hba.conf and pg_ident.conf; which it does not.

Complete setup and testing cases below validating what I am seeing:

System Configuration:
* debian bookworm, latest patches, all repositories; 2025-03-18
* Postgres 17.4, from postgres repos - https://apt.postgres.org/;
2025-03-18
* Default package installation (apt update && apt dist-upgrade && apt
install postgres)

/tmp/pg.conf:
max_connections=1000

/tmp/pg_ident.conf
local_all postgres trust
/tmp/pg_hba.conf
local all all trust

Test: postgresql.conf
* Quoting sources work correctly. This is expected based on documentation.

.../17/main/postgresql.conf:
include_dir = 'conf.d'
include_if_exists = '/tmp/pg.conf'

-> postgresql service starts. No errors. Expected.
---
.../17/main/postgresql.conf:
include_dir 'conf.d'
include_if_exists '/tmp/pg.conf'

-> postgresql service starts. No errors. Expected.

Test: postgresql.conf unquoted
* Not quoting sources work correctly. This is expected based on
documentation.

.../17/main/postgresql.conf:
include_dir = conf.d
include_if_exists = /tmp/pg.conf

-> postgresql service fails to start. Expected.
2025-03-18 21:32:09.317 GMT [8970] LOG:  syntax error in file
"/etc/postgresql/17/main/postgresql.conf" line 834, near token "conf.d"
2025-03-18 21:32:09.317 GMT [8970] LOG:  syntax error in file
"/etc/postgresql/17/main/postgresql.conf" line 836, near token "/"
2025-03-18 21:32:09.317 GMT [8970] FATAL:  configuration file
"/etc/postgresql/17/main/postgresql.conf" contains errors
---
.../17/main/postgresql.conf:
include_dir conf.d
include_if_exists /tmp/pg.conf

-> postgresql service fails to start. Expected.
2025-03-18 21:32:09.317 GMT [8970] LOG:  syntax error in file
"/etc/postgresql/17/main/postgresql.conf" line 834, near token "conf.d"
2025-03-18 21:32:09.317 GMT [8970] LOG:  syntax error in file
"/etc/postgresql/17/main/postgresql.conf" line 836, near token "/"
2025-03-18 21:32:09.317 GMT [8970] FATAL:  configuration file
"/etc/postgresql/17/main/postgresql.conf" contains errors


Test: pg_ident.conf
* Expectation: Quoting sources as before works as-in postgresql.conf;
per pg_ident.conf documentation and include documentation

.../17/main/pg_hba.conf
local all all peer map=local_all

.../17/main/pg_ident.conf
include_dir = 'conf.d'
include_if_exists = '/tmp/pg_ident.conf'

-> postgresql service starts. No errors. Expected.
---
.../17/main/pg_hba.conf
local all all peer map=local_all

.../17/main/pg_ident.conf
include_dir = conf.d
include_if_exists = /tmp/pg_ident.conf

-> postgresql service starts. No errors. DEVIATION from postgresql.conf
behavior.
---
.../17/main/pg_hba.conf
local all all peer map=local_all

.../17/main/pg_ident.conf
include_dir conf.d
include_if_exists tmp/pg_ident.conf

-> postgresql service starts. No errors. DEVIATION from postgresql.conf
behavior.


Test: pg_hba.conf
* Expectation: Quoting sources as before works as-in postgresql.conf;
per pg_hba.conf documentation and include documentation

.../17/main/pg_hba.conf
include_dir = 'conf.d'
include_if_exists = '/tmp/pg_hba.conf'

-> postgresql service fails to start. DEVIATION from postgresql.conf
behavior.
2025-03-18 22:04:47.437 UTC [9110] LOG:  invalid connection type
"include_dir"
2025-03-18 22:04:47.437 UTC [9110] CONTEXT:  line 134 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:04:47.437 UTC [9110] LOG:  invalid connection type
"include_if_exists"
2025-03-18 22:04:47.437 UTC [9110] CONTEXT:  line 135 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:04:47.437 UTC [9110] FATAL:  could not load
/etc/postgresql/17/main/pg_hba.conf
* include_if_exists explicitly supported in documentation.
---
.../17/main/pg_hba.conf
local all all peer map=local_all

.../17/main/pg_ident.conf
include_dir = conf.d
include_if_exists = /tmp/pg_hba.conf

-> postgresql service fails to start. DEVIATION from postgresql.conf
behavior.
2025-03-18 22:01:04.933 UTC [9109] LOG:  invalid connection type
"include_dir"
2025-03-18 22:01:04.933 UTC [9109] CONTEXT:  line 134 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:01:04.933 UTC [9109] LOG:  invalid connection type
"include_if_exists"
2025-03-18 22:01:04.933 UTC [9109] CONTEXT:  line 135 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:01:04.933 UTC [9109] FATAL:  could not load
/etc/postgresql/17/main/pg_hba.conf
* include_dir, include_if_exists explicitly supported in documentation.
---
.../17/main/pg_hba.conf
local all all peer map=local_all

.../17/main/pg_ident.conf
include_dir 'conf.d'
include_if_exists '/tmp/pg_hba.conf'

-> postgresql service fails to start. DEVIATION from postgresql.conf
behavior.
2025-03-18 22:05:26.253 UTC [9111] LOG:  could not open configuration
directory "/etc/postgresql/17/main/'conf.d'": No such file or directory
2025-03-18 22:05:26.253 UTC [9111] CONTEXT:  line 134 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:05:26.253 UTC [9111] LOG:  could not open file
"/etc/postgresql/17/main/'/tmp/pg_hba.conf'": No such file or directory
2025-03-18 22:05:26.253 UTC [9111] CONTEXT:  line 135 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:05:26.253 UTC [9111] LOG:  skipping missing authentication
file "/etc/postgresql/17/main/'/tmp/pg_hba.conf'"
2025-03-18 22:05:26.253 UTC [9111] CONTEXT:  line 135 of configuration file
"/etc/postgresql/17/main/pg_hba.conf"
2025-03-18 22:05:26.253 UTC [9111] FATAL:  could not load
/etc/postgresql/17/main/pg_hba.conf
* Looks like this now directly appends any quoted option directly to the
data_directory path, including quotes, no path resolution.
---
.../17/main/pg_hba.conf
local all all peer map=local_all

.../17/main/pg_ident.conf
include_dir conf.d
include_if_exists /tmp/pg_hba.conf

-> postgresql service starts. DEVIATION from postgresql.conf behavior.
select * from pg_hba_file_rules where file_name='/tmp/hba.conf';
 rule_number |    file_name     | line_number | type  | database | user_name
| address | netmask | auth_method | options | error
-------------+------------------+-------------+-------+----------+-----------+---------+---------+-------
------+---------+-------
           8 | /tmp/pg_hba.conf |           1 | local | {all}    | {all}    
|         |         | trust
      |         |
(1 row)
* Looks like this loaded everything correctly?

I believe this is an actual bug, or documentation needs to be explicitly
updated to reflect behavior changes for include directives.

include documentation source:
https://www.postgresql.org/docs/17/config-setting.html#CONFIG-INCLUDES
pg_ident.conf documentation source:
https://www.postgresql.org/docs/17/auth-username-maps.html
pg_hba.conf documentation source:
https://www.postgresql.org/docs/current/auth-pg-hba-conf.html

Thanks!

On Tue, Mar 18, 2025 at 03:38:06PM -0700, Robert Pufky wrote:
> For the 'Test: pg_hba.conf' tests, I included the wrong testing notes, they
> should be:
>
> Test 2:
> .../17/main/pg_hba.conf
> include_dir = conf.d
> include_if_exists = /tmp/pg_hba.conf
>
> Test 3:
> .../17/main/pg_hba.conf
> include_dir 'conf.d'
> include_if_exists '/tmp/pg_hba.conf'
>
> Test 4:
> .../17/main/pg_hba.conf
> include_dir conf.d
> include_if_exists /tmp/pg_hba.conf

The logic used for configuration files regarding GUCs (aka
postgresql.conf) and HBA/ident files are bound to different rules when
it comes to quotes.  For the HBA/ident files, the logic is around
next_token() in hba.c.  For GUC files, the logic is much smarter, see
guc-file.l.  The handling of the quotes and separators is different
based on these rules, so, while we have a consistent set of grammar
keywords to use for both, you may see differences like the ones you
are reporting here.

Your confusion here is behind the use of the "=" parameter with one of
the include directives, making the HBA/ident code parsing think that
this is a connection option, but well, it's not.   The documentation
does not mention that this combination is OK:
https://www.postgresql.org/docs/devel/auth-pg-hba-conf.html
https://www.postgresql.org/docs/devel/auth-username-maps.html

Perhaps you're right and we should expand this error message to
provide more details, or make the token parsing slightly smarter in
this case by feeding optional "=" separators.  However, I am not
convinced that this is worth the effort just on the ground to make
HBA/ident parsing closer to the GUC parsing, because they are entirely
different facilities, and the docs clearly state that '=' is not part
of the supported grammar for the include keywords in the HBA and ident
parts.  Hence, the HBA parsing generating this "invalid connection
type" error is not completely wrong, because it's telling what it sees
based on the tokens parsed.
--
Michael