Thread: What is the motivation of include directive and configuration files outside PGDATA

What is the motivation of include directive and configuration files outside PGDATA

From

"Takayuki Tsunakawa"

Date:

17 January 2007, 01:26:44

Hello,

Let me ask about the background of configuration files.  I couldn't
find the relevant information in the 8.2 documentation.  I'm sorry to
cause you trouble.

In section "17.1. Setting Parameters", include directive is described.
Why was this directive prepared?  What usage is assumed?  Is it for
GUI tools, or for placing custom parameters in other files?

In section "17.2. File Locations", the following parameters are
described:

data_directory (string)
config_file (string)
hba_file (string)
ident_file (string)
external_pid_file (string)

What demand is assumed for placing configuration files outside the
data directory?  Is it for placing configuration files in /etc to
conform to the FHS (Filesystem Hierarchy Standard, as documented
below) to enable the backup of all configuration files on the system?

http://www.pathname.com/fhs/

Re: What is the motivation of include directive and

From

Bruce Momjian

Date:

17 January 2007, 02:47:48

Takayuki Tsunakawa wrote:
> Hello,
> 
> Let me ask about the background of configuration files.  I couldn't
> find the relevant information in the 8.2 documentation.  I'm sorry to
> cause you trouble.
> 
> In section "17.1. Setting Parameters", include directive is described.
> Why was this directive prepared?  What usage is assumed?  Is it for
> GUI tools, or for placing custom parameters in other files?
> 
> In section "17.2. File Locations", the following parameters are
> described:
> 
> data_directory (string)
> config_file (string)
> hba_file (string)
> ident_file (string)
> external_pid_file (string)
> 
> What demand is assumed for placing configuration files outside the
> data directory?  Is it for placing configuration files in /etc to
> conform to the FHS (Filesystem Hierarchy Standard, as documented
> below) to enable the backup of all configuration files on the system?

It was designed for people who have multiple postmasters, but and want
to centralize some of that configuation.

--  Bruce Momjian   bruce@momjian.us EnterpriseDB    http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +

Re: What is the motivation of include directive and

From

"Takayuki Tsunakawa"

Date:

17 January 2007, 03:40:40

From: "Bruce Momjian" <bruce@momjian.us>
> Takayuki Tsunakawa wrote:
>> In section "17.1. Setting Parameters", include directive is
described.
>> Why was this directive prepared?  What usage is assumed?  Is it for
>> GUI tools, or for placing custom parameters in other files?
>>
>> In section "17.2. File Locations", the following parameters are
>> described:
>> What demand is assumed for placing configuration files outside the
>> data directory?  Is it for placing configuration files in /etc to
>> conform to the FHS (Filesystem Hierarchy Standard, as documented
>> below) to enable the backup of all configuration files on the
system?
>
> It was designed for people who have multiple postmasters, but and
want
> to centralize some of that configuation.

Thank you.  But I'm still a bit unclear.
Do you mean those functionalities were designed for, e.g.,
organizations that provide hosting service -- runs multiple database
instances on a single large machine?  The image is:

/DataDirForCustomer1/postgresql.conf
[content]
include '/etc/postgresql_common.conf'
hba_file='/etc/pg_hba.conf'
...settings specific to customer 1

/DataDirForCustomer2/postgresql.conf
[content]
include '/etc/postgresql_common.conf'
hba_file='/etc/pg_hba.conf'
...settings specific to customer 2

/etc/postgresql_common.conf
[content]
...settings common among all customers

Was this mechanism invented in response to the demand from users, or
invented from the idea of PostgreSQL developers?

Still, I don't understand well why config files need to be placed
outside the data directory, except for daring conform to FHS.

Re: What is the motivation of include directive and

From

Tom Lane

Date:

17 January 2007, 03:50:53

"Takayuki Tsunakawa" <tsunakawa.takay@jp.fujitsu.com> writes:
> Still, I don't understand well why config files need to be placed
> outside the data directory, except for daring conform to FHS.

The killer argument for it is that most of what is in $PGDATA should be
excluded from your normal filesystem backup method, because you need to
be using some database-aware mechanism for backing up the database.  But
the config files are perfectly suited for standard filesystem backup,
and indeed will *not* be covered by, say, pg_dumpall.  So putting them
somewhere else helps in creating a coherent backup strategy.
        regards, tom lane

Re: What is the motivation of include directive and

From

"Takayuki Tsunakawa"

Date:

17 January 2007, 04:18:38

From: "Tom Lane" <tgl@sss.pgh.pa.us>
> "Takayuki Tsunakawa" <tsunakawa.takay@jp.fujitsu.com> writes:
>> Still, I don't understand well why config files need to be placed
>> outside the data directory, except for daring conform to FHS.
>
> The killer argument for it is that most of what is in $PGDATA should
be
> excluded from your normal filesystem backup method, because you need
to
> be using some database-aware mechanism for backing up the database.
But
> the config files are perfectly suited for standard filesystem
backup,
> and indeed will *not* be covered by, say, pg_dumpall.  So putting
them
> somewhere else helps in creating a coherent backup strategy.

Thank you, I've understood the reason for placement.  As I supposed,
it is provided mainly for placing config files in /etc to allow
frequent backup of the configuration as a whole system, because most
config files of UNIX are stored in /etc (recommended so in FHS) and
/etc is small.

Re: What is the motivation of include directive and

From

Andrew Dunstan

Date:

17 January 2007, 13:15:30

Tom Lane wrote:
> "Takayuki Tsunakawa" <tsunakawa.takay@jp.fujitsu.com> writes:
>   
>> Still, I don't understand well why config files need to be placed
>> outside the data directory, except for daring conform to FHS.
>>     
>
> The killer argument for it is that most of what is in $PGDATA should be
> excluded from your normal filesystem backup method, because you need to
> be using some database-aware mechanism for backing up the database.  But
> the config files are perfectly suited for standard filesystem backup,
> and indeed will *not* be covered by, say, pg_dumpall.  So putting them
> somewhere else helps in creating a coherent backup strategy.
>
>             

My backup setup excludes subdirectories of the data dir, rather than the 
data dir itself.

Meeting FHS requirements is no bad thing, though. And the ability to 
include a common configuration set in multiple instances is surely 
useful to a number of people. After all, you aren't forced to use these 
facilities - I typically don't.

cheers

andrew

Re: What is the motivation of include directive and

From

"Takayuki Tsunakawa"

Date:

18 January 2007, 03:41:30

From: "Andrew Dunstan" <andrew@dunslane.net>
> Meeting FHS requirements is no bad thing, though. And the ability to
> include a common configuration set in multiple instances is surely
> useful to a number of people. After all, you aren't forced to use
these
> facilities - I typically don't.

Thank you, Andrew-san.
What I want to know is a more concreet thing.

How useful are those facilities to what kind of users in what cases?
Is there a reason why users in the real world positively use those
facilities?

If I want to develop an add-on tool that manipulates PostgreSQL
settings even when PostgreSQL is stopped, should that tool support the
environments where include directive is used and config files are
placed outside the data directory?  If the need for include directive
and flexible config file placement is very low or vague, I thought I
could say "this tool does not support those environments".

Are any facilities (functions) provided in PostgreSQL that external
tool developers can use to read/write config files?  Suppose a user
created a database instance manually, with the config files as
follows:

[$PGDATA/postgresql.conf]
...
max_connections=10
include '/some_dir/some.conf'
include '/other_dir/other.conf'

[/some_dir/some.conf]
...
max_connections=50

[/other_dir/other.conf]
...
max_connections=100


If an external tool wants to determine the current value of
max_connections or modify the setting, what should the tool do?
Should the tool parse and write out the files by itself?

Re: What is the motivation of include directive and

From

Andrew Dunstan

Date:

18 January 2007, 13:37:34

Takayuki Tsunakawa wrote:
> From: "Andrew Dunstan" <andrew@dunslane.net>
>   
>> Meeting FHS requirements is no bad thing, though. And the ability to
>> include a common configuration set in multiple instances is surely
>> useful to a number of people. After all, you aren't forced to use
>>     
> these
>   
>> facilities - I typically don't.
>>     
>
> Thank you, Andrew-san.
> What I want to know is a more concreet thing.
>
> How useful are those facilities to what kind of users in what cases?
> Is there a reason why users in the real world positively use those
> facilities?
>
>   

If you want to find out about usage patterns this is probably not the 
best place to survey users - the hackers are not very representative of 
users in general. I have a sneaking suspicion that include directives 
are not used a whole lot, but alternative file locations are much more 
widely used.

My off-the-cuff advice about your tool would be to process any include 
directives when reading the config file, but initially just write out 
new settings to the top level file (at the end, so they don't get 
overridden by the includes).

cheers

andrew