Re: Proposal: pg_confcheck - syntactic & semantic validation of postgresql configuration files - Mailing list pgsql-hackers

On 10/14/2015 01:30 PM, Andres Freund wrote:
> On 2015-10-14 01:54:46 +0300, Amir Rohan wrote:
>> Andres, please see upthread for quite a bit on what it doesn't do, and
>> why having it in the server is both an advantages and a shortcoming.
> 
> As far as I have skimmed the thread it's only talking about shortcoming
> in case it requires a running server. Which -C doesn't.
> 

That's helpful, no one has suggested -C yet. This was new to me.

Here's what the usage says:

Usage: postgres [OPTION]...

Options: -C NAME            print value of run-time parameter, then exit

```

P1. The fact that -C will warn about bad values when you query for an
unrelated name (as it requires you to do) is cunningly and successfully
elided. I expected to have to check every one individually. How about a
no-argument version of "-C"?

P2. Next, that it will print the value of a "run-time" parameter,
without an actual process is another nice surprise, which I wouldn't
have guessed.

The error messages are all one could wish for:

LOG:  invalid value for parameter "wal_level": "archive2"
HINT:  Available values: minimal, archive, hot_standby, logical.
LOG:  parameter "fsync" requires a Boolean value

*and* it prints everything it finds wrong, not stopping at the first
one. very nice.

it does fail the "dependent options" test:
$ postgres -C "archive_mode"
on
$ postgres -C wal_level
minimal

no errors, great, let's try it:
$ pg_ctl restart

FATAL:  WAL archival cannot be enabled when wal_level is "minimal"

> I don't think there's any fundamental difference between some external
> binary parsing & validating the config file and the postgres binary
> doing it. 

-C does a much better job then pg_file_settings for this task, I agree.
Still, it doesn't answer the already mentioned points of:
1) You need a server (binary, not a running server) to check a config
(reasonable), with a version matching what you'll run on (again, reasonable)
2) It doesn't catch more complicated problems like dependent options.
3) It doesn't catch bad ideas, only errors.
4) You can't easily extend the checks performed, without forking
postgres or going through the (lengthy, rigorous) cf process.
5) Because it checks syntax only, you don't get the benefits of having
an official place for the community to easily contribute rules that
warn you against config pitfalls, so that all users benefits.
See my OP for real-world examples and links about this problem.

> There *is* the question of the utility being able to to
> process options from multiple major releases, but I don't think that's a
> particularly worthwhile goal here.
> 

For one thing, I think it's been made clear that either few people know
about -C or few use it as part of their usual workflow. That in itself
is an issue. Any bloggers reading this?

Next, you may not agree semantic checks/advice is useful. I only
agree it's easy to dismiss until it saves your (i.e. a user's) ass
that first time.

I would *highly* recommend the talk mentioned in the OP:

"...Lag - What's Wrong with My Slave?"  (Samantha Billington, 2015)
https://youtu.be/If22EwtCFKc

Not just for the concrete examples (*you* know those by heart), but to
really hear the frustration in a real production user's voice when they
deploy pg in production, hit major operational difficulties, spend lots
of time and effort trying to root-cause and finally realize they simply
needed to "turn on FOO". Most of these problem can be caught very easily
with a conf linter.

Amir