Thread: Postgresql.conf cleanup

Postgresql.conf cleanup

From
Josh Berkus
Date:
All,

I'm working on cleaning up postgresql.conf and pg_settings for the 
release.  Attached is a sample WIP.  It's not in patch form because I'm 
not done yet; I've just been editing postgresql.conf and need to fix the 
docs and pg_settings to match.

Issues encountered and changes made:

PostgreSQL.conf
----------------

suggestions: added section with the 7 most important obvious settings at 
the top and suggestions on how to calculate them.  If people like this, 
I'll add it to the Tutorial in the docs as well.

seq_scan_cost: this is independant of all of the other _costs.  I can't 
think of any way in which that doesn't make the whole set of costs 
unmanageable.  For example, if you want to change seq_scan_cost in order 
to make query cost more-or-less match up with ms execution time, you 
have to modify all 6 settings.   If we do implement per-tablespace 
costs, then we'll need per-tablespace random_page_cost as well.  Or am I 
missing something?

(change requires restart): this phrase appears over 20 times in the 
notes.  This is enough times to be really repetitive and take up a lot 
of scrolling space, while not actually covering all startup-time 
parameters.  We should either (a) remove all such notes and rely on 
docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.  Votes?

Vacuum: all vacuum & autovacuum parameters put under their own section.

Client Cost Defaults: this section became a "catch-all" for all userset 
parameters which people weren't sure what to do with.  I've divided it 
into logical subsections, and moved some parameters to other sections 
where they logically belong (for example, explain_pretty_print belongs 
in Query Tuning).

pg_settings issues
--------------------

transaction_isolation and transaction_read_only appear more than once in 
the pg_settings pseudo_table.   The setting column is supposed to be unique.


Given the amount of cleanup/improvement which I'm seeing as necessary 
for the GUCs, I'm wondering if I put this off too long for 8.3.

--Josh





Re: Postgresql.conf cleanup

From
Josh Berkus
Date:
This time *with* the attachment.


> All,
>
> I'm working on cleaning up postgresql.conf and pg_settings for the
> release.  Attached is a sample WIP.  It's not in patch form because I'm
> not done yet; I've just been editing postgresql.conf and need to fix the
> docs and pg_settings to match.
>
> Issues encountered and changes made:
>
> PostgreSQL.conf
> ----------------
>
> suggestions: added section with the 7 most important obvious settings at
> the top and suggestions on how to calculate them.  If people like this,
> I'll add it to the Tutorial in the docs as well.
>
> seq_scan_cost: this is independant of all of the other _costs.  I can't
> think of any way in which that doesn't make the whole set of costs
> unmanageable.  For example, if you want to change seq_scan_cost in order
> to make query cost more-or-less match up with ms execution time, you
> have to modify all 6 settings.   If we do implement per-tablespace
> costs, then we'll need per-tablespace random_page_cost as well.  Or am I
> missing something?
>
> (change requires restart): this phrase appears over 20 times in the
> notes.  This is enough times to be really repetitive and take up a lot
> of scrolling space, while not actually covering all startup-time
> parameters.  We should either (a) remove all such notes and rely on
> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them.
>  Votes?
>
> Vacuum: all vacuum & autovacuum parameters put under their own section.
>
> Client Cost Defaults: this section became a "catch-all" for all userset
> parameters which people weren't sure what to do with.  I've divided it
> into logical subsections, and moved some parameters to other sections
> where they logically belong (for example, explain_pretty_print belongs
> in Query Tuning).
>
> pg_settings issues
> --------------------
>
> transaction_isolation and transaction_read_only appear more than once in
> the pg_settings pseudo_table.   The setting column is supposed to be
> unique.
>
>
> Given the amount of cleanup/improvement which I'm seeing as necessary
> for the GUCs, I'm wondering if I put this off too long for 8.3.
>
> --Josh
>
>
>
>

# -----------------------------
# PostgreSQL configuration file
# -----------------------------
#
# This file consists of lines of the form:
#
#   name = value
#
# (The '=' is optional.)  White space may be used.  Comments are introduced
# with '#' anywhere on a line.  The complete list of option names and
# allowed values can be found in the PostgreSQL documentation.  The
# commented-out settings shown in this file represent the default values.
#
# Please note that re-commenting a setting is NOT sufficient to revert it
# to the default value, unless you restart the server.
#
# Any option can also be given as a command line switch to the server,
# e.g., 'postgres -c log_connections=on'.  Some options can be changed at
# run-time with the 'SET' SQL command.
#
# This file is read on server startup and when the server receives a
# SIGHUP.  If you edit the file on a running system, you have to SIGHUP the
# server for the changes to take effect, or use "pg_ctl reload". Some
# settings, which are marked below, require a server shutdown and restart
# to take effect.
#
# Memory units:  kB = kilobytes MB = megabytes GB = gigabytes
# Time units:    ms = milliseconds s = seconds min = minutes h = hours d = days

#---------------------------------------------------------------------------
# IMPORTANT PERFORMANCE VALUES TO SET
#---------------------------------------------------------------------------

# What follows are some rough recommendations of values which most users
# should set immediately after installation for good performance.  The
# calculations below are "rules of thumb" and are not intended to replace
# knowledgeable tuning. Please see the full documentation and the
# pgsql-performance mailing list for more information and suggestions.
#
# Note that RAM below refers to RAM which is available to PostgreSQL,
# so on shared servers the RAM in the calculation should be reduced from
# total system RAM accordingly.  Settings are listed in the order they
# appear below.
#
# max_connections = no. of concurrent sessions you need to support
# shared_buffers = RAM * 0.2 (this may require system configuration)
# work_mem = ( RAM * 0.5 ) / max_connections, or less
# maintenance_work_mem = RAM/8, up to 256MB
# wal_buffers = 1MB
# max_fsm_pages = expected database size * 0.1
# checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)
# effective_cache_size = RAM * 0.7


#---------------------------------------------------------------------------
# FILE LOCATIONS
#---------------------------------------------------------------------------

# The default values of these variables are driven from the -D command line
# switch or PGDATA environment variable, represented here as ConfigDir.

#data_directory = 'ConfigDir'        # use data in another directory
                    # (change requires restart)
#hba_file = 'ConfigDir/pg_hba.conf'    # host-based authentication file
                    # (change requires restart)
#ident_file = 'ConfigDir/pg_ident.conf'    # ident configuration file
                    # (change requires restart)

# If external_pid_file is not explicitly set, no extra PID file is written.
#external_pid_file = '(none)'        # write an extra PID file
                    # (change requires restart)


#---------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#---------------------------------------------------------------------------

# - Connection Settings -

#listen_addresses = 'localhost'        # what IP address(es) to listen on;
                    # comma-separated list of addresses;
                    # defaults to 'localhost', '*' = all
                    # (change requires restart)
#port = 5432                # (change requires restart)
#max_connections = 100            # (change requires restart)
# Note: increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction).  You
# might also need to raise shared_buffers to support more connections.
#superuser_reserved_connections = 3    # (change requires restart)
#unix_socket_directory = ''        # (change requires restart)
#unix_socket_group = ''            # (change requires restart)
#unix_socket_permissions = 0777        # octal
                    # (change requires restart)
#bonjour_name = ''            # defaults to the computer name
                    # (change requires restart)

# - Security & Authentication -

#authentication_timeout = 1min        # 1s-600s
#ssl = off                # (change requires restart)
#ssl_ciphers = 'ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH'    # Allowed SSL ciphers
                    # (change requires restart)
#password_encryption = on
#db_user_namespace = off

# - Kerberos -
#krb_server_keyfile = ''        # (change requires restart)
#krb_srvname = 'postgres'        # (change requires restart)
#krb_server_hostname = ''        # empty string matches any keytab entry
                    # (change requires restart)
#krb_caseins_users = off        # (change requires restart)

# - TCP Keepalives -
# see 'man 7 tcp' for details

#tcp_keepalives_idle = 0        # TCP_KEEPIDLE, in seconds;
                    # 0 selects the system default
#tcp_keepalives_interval = 0        # TCP_KEEPINTVL, in seconds;
                    # 0 selects the system default
#tcp_keepalives_count = 0        # TCP_KEEPCNT;
                    # 0 selects the system default


#---------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)
#---------------------------------------------------------------------------

# - Memory -

#shared_buffers = 32MB            # min 128kB or max_connections*16kB
                    # (change requires restart)
#temp_buffers = 8MB            # min 800kB
#max_prepared_transactions = 5        # can be 0 or more
                    # (change requires restart)
# Note: increasing max_prepared_transactions costs ~600 bytes of shared memory
# per transaction slot, plus lock space (see max_locks_per_transaction).
#work_mem = 1MB                # min 64kB
#maintenance_work_mem = 16MB        # min 1MB
#max_stack_depth = 2MB            # min 100kB

# - Free Space Map -

#max_fsm_pages = 204800            # min max_fsm_relations*16, 6 bytes each
                    # (change requires restart)
#max_fsm_relations = 1000        # min 100, ~70 bytes each
                    # (change requires restart)

# - Kernel Resource Usage -

#max_files_per_process = 1000        # min 25
                    # (change requires restart)
#shared_preload_libraries = ''        # (change requires restart)

# - Background writer -

#bgwriter_delay = 200ms            # 10-10000ms between rounds
#bgwriter_lru_percent = 1.0        # 0-100% of LRU buffers scanned/round
#bgwriter_lru_maxpages = 5        # 0-1000 buffers max written/round
#bgwriter_all_percent = 0.333        # 0-100% of all buffers scanned/round
#bgwriter_all_maxpages = 5        # 0-1000 buffers max written/round


#---------------------------------------------------------------------------
# WRITE AHEAD LOG
#---------------------------------------------------------------------------

# - Settings -

#fsync = on                # turns forced synchronization on or off
#wal_sync_method = fsync        # the default is the first option
                    # supported by the operating system:
                    #   open_datasync
                    #   fdatasync
                    #   fsync
                    #   fsync_writethrough
                    #   open_sync
#full_page_writes = on            # recover from partial page writes
#wal_buffers = 64kB            # min 32kB
                    # (change requires restart)
#commit_delay = 0            # range 0-100000, in microseconds
#commit_siblings = 5            # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3        # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min        # range 30s-1h
#checkpoint_warning = 30s        # 0 is off

# - Archiving -

#archive_command = ''        # command to use to archive a logfile segment
#archive_timeout = 0        # force a logfile segment switch after this
                # many seconds; 0 is off

#---------------------------------------------------------------------------
# VACUUM AND AUTOVACUUM
#---------------------------------------------------------------------------

# - General -

#vacuum_freeze_min_age = 100000000

# - Cost-Based Vacuum Delay -

#vacuum_cost_delay = 0            # 0-1000 milliseconds
#vacuum_cost_page_hit = 1        # 0-10000 credits
#vacuum_cost_page_miss = 10        # 0-10000 credits
#vacuum_cost_page_dirty = 20        # 0-10000 credits
#vacuum_cost_limit = 200        # 0-10000 credits

# - Autovacuum -

#autovacuum = on            # enable autovacuum subprocess?
                    # 'on' requires stats_start_collector
                    # and stats_row_level to also be on
#autovacuum_max_workers = 3        # max # of autovacuum subprocesses
#autovacuum_naptime = 1min        # time between autovacuum runs
#log_autovacuum = -1            # -1 is disabled, 0 logs all actions
                    # and their durations, > 0 logs only
                    # actions running at least N msec.
#autovacuum_vacuum_threshold = 500    # min # of tuple updates before
                    # vacuum
#autovacuum_analyze_threshold = 250    # min # of tuple updates before
                    # analyze
#autovacuum_vacuum_scale_factor = 0.2    # fraction of rel size before
                    # vacuum
#autovacuum_analyze_scale_factor = 0.1    # fraction of rel size before
                    # analyze
#autovacuum_freeze_max_age = 200000000    # maximum XID age before forced vacuum
                    # (change requires restart)
#autovacuum_vacuum_cost_delay = -1    # default vacuum cost delay for
                    # autovacuum, -1 means use
                    # vacuum_cost_delay
#autovacuum_vacuum_cost_limit = -1    # default vacuum cost limit for
                    # autovacuum, -1 means use
                    # vacuum_cost_limit

#---------------------------------------------------------------------------
# QUERY TUNING
#---------------------------------------------------------------------------

# - Planner Method Configuration -

#enable_bitmapscan = on
#enable_hashagg = on
#enable_hashjoin = on
#enable_indexscan = on
#enable_mergejoin = on
#enable_nestloop = on
#enable_seqscan = on
#enable_sort = on
#enable_tidscan = on

# - Planner Cost Constants -

#seq_page_cost = 1.0            # measured on an arbitrary scale
#random_page_cost = 4.0            # same scale as above
#cpu_tuple_cost = 0.01            # same scale as above
#cpu_index_tuple_cost = 0.005        # same scale as above
#cpu_operator_cost = 0.0025        # same scale as above
#effective_cache_size = 128MB

# - Genetic Query Optimizer -

#geqo = on
#geqo_threshold = 12
#geqo_effort = 5            # range 1-10
#geqo_pool_size = 0            # selects default based on effort
#geqo_generations = 0            # selects default based on effort
#geqo_selection_bias = 2.0        # range 1.5-2.0

# - Other Planner Options -

#default_statistics_target = 10        # range 1-1000
#constraint_exclusion = off
#from_collapse_limit = 8
#join_collapse_limit = 8        # 1 disables collapsing of explicit
                    # JOINs
#explain_pretty_print = on


#---------------------------------------------------------------------------
# ERROR REPORTING AND LOGGING
#---------------------------------------------------------------------------

# - Where to Log -

#log_destination = 'stderr'        # Valid values are combinations of
                    # stderr, syslog and eventlog,
                    # depending on platform.

# This is used when logging to stderr:
#redirect_stderr = off            # Enable capturing of stderr into log
                    # files
                    # (change requires restart)

# These are only used if redirect_stderr is on:
#log_directory = 'pg_log'        # Directory where log files are written
                    # Can be absolute or relative to PGDATA
#log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log' # Log file name pattern.
                    # Can include strftime() escapes
#log_truncate_on_rotation = off     # Truncate rather than appending to files
                    # of the same name.
#log_rotation_age = 1d            # Automatic rotation of logfiles will
                    # happen after that time.  0 to
                    # disable.
#log_rotation_size = 10MB        # Automatic rotation of logfiles will
                    # happen after that much log
                    # output.  0 to disable.

# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'


# - When to Log -

#client_min_messages = notice        # Values in order of decreasing detail:
                    #   debug5, debug4, debug3, debug2,
                    #   debug1, log, notice, warning, error
#log_min_messages = notice        # Values in order of decreasing detail:
                    #   debug5, debug4, debug3, debug2,
                    #   debug1, info, notice, warning,
                    #   error, log, fatal, panic
#log_error_verbosity = default        # terse, default, or verbose messages
#log_min_error_statement = error    # Values in order of decreasing detail:
                    #   debug5, debug4, debug3, debug2,
                    #   debug1, info, notice, warning,
                    #   error, log, fatal, panic (off)
#log_min_duration_statement = -1    # -1 is disabled, 0 logs all statements
                    # and their durations, > 0 logs only
                    # statements running at least N msec.
#silent_mode = off            # DO NOT USE without syslog or
                    # redirect_stderr

# - What to Log -

#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = off
#log_connections = off
#log_disconnections = off
#log_duration = off
#log_line_prefix = ''            # Special values:
                    #   %u = user name
                    #   %d = database name
                    #   %r = remote host and port
                    #   %h = remote host
                    #   %p = PID
                    #   %t = timestamp (no milliseconds)
                    #   %m = timestamp with milliseconds
                    #   %i = command tag
                    #   %c = session id
                    #   %l = session line number
                    #   %s = session start timestamp
                    #   %x = transaction id
                    #   %q = stop here in non-session
                    #        processes
                    #   %% = '%'
                    # e.g. '<%u%%%d> '
#log_statement = 'none'            # none, ddl, mod, all
#log_hostname = off
#log_lock_waits = off            # Log lock waits longer than deadlock_timeout
#log_temp_files = -1            # Log temporary files equal or larger
                    # than the specified number of kilobytes.
                    # -1 disables;  0 logs all temp files

#---------------------------------------------------------------------------
# RUNTIME STATISTICS
#---------------------------------------------------------------------------

# - Query/Index Statistics Collector -

#stats_command_string = on
#update_process_title = on

#stats_start_collector = on        # needed for block or row stats
                    # (change requires restart)
#stats_block_level = off
#stats_row_level = on
#stats_reset_on_server_start = off    # (change requires restart)


# - Statistics Monitoring -

#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off

#---------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
#---------------------------------------------------------------------------

# - Statement Behavior -

#search_path = '"$user",public'        # schema names
#default_tablespace = ''        # a tablespace name, '' uses
                    # the default
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
#statement_timeout = 0            # 0 is disabled

# - Locale and Formatting -

#datestyle = 'iso, mdy'
#timezone = unknown            # actually, defaults to TZ
                    # environment setting
#timezone_abbreviations = 'Default'     # select the set of available timezone
                    # abbreviations. Currently, there are
                    #   Default
                    #   Australia
                    #   India
                    # However you can also create your own
                    # file in share/timezonesets/.
#extra_float_digits = 0            # min -15, max 2
#client_encoding = sql_ascii        # actually, defaults to database
                    # encoding

# These settings are initialized by initdb -- they might be changed
#lc_messages = 'C'            # locale for system error message
                    # strings
#lc_monetary = 'C'            # locale for monetary formatting
#lc_numeric = 'C'            # locale for number formatting
#lc_time = 'C'                # locale for time formatting

# - Libraries -
#dynamic_library_path = '$libdir'
#local_preload_libraries = ''

# - XML -
#xmlbinary = 'base64'
#xmloption = 'content'

# - Other Defaults -

#check_function_bodies = on
#session_replication_role = "origin"

#---------------------------------------------------------------------------
# LOCK MANAGEMENT
#---------------------------------------------------------------------------

#deadlock_timeout = 1s
#max_locks_per_transaction = 64        # min 10
                    # (change requires restart)
# Note: each lock table slot uses ~270 bytes of shared memory, and there are
# max_locks_per_transaction * (max_connections + max_prepared_transactions)
# lock table slots.


#---------------------------------------------------------------------------
# VERSION/PLATFORM COMPATIBILITY
#---------------------------------------------------------------------------

# - Previous Postgres Versions -

#add_missing_from = off
#array_nulls = on
#backslash_quote = safe_encoding    # on, off, or safe_encoding
#default_with_oids = off
#escape_string_warning = on
#sql_inheritance = on
#regex_flavor = advanced        # advanced, extended, or basic
#standard_conforming_strings = off

# - Other Platforms & Clients -

#transform_null_equals = off


#---------------------------------------------------------------------------
# CUSTOMIZED OPTIONS
#---------------------------------------------------------------------------

#custom_variable_classes = ''        # list of custom variable class names

Re: Postgresql.conf cleanup

From
Jim Nasby
Date:
On Jul 2, 2007, at 6:03 AM, Josh Berkus wrote:
> (change requires restart): this phrase appears over 20 times in the  
> notes.  This is enough times to be really repetitive and take up a  
> lot of scrolling space, while not actually covering all startup- 
> time parameters.  We should either (a) remove all such notes and  
> rely on docs, or (b) make an annotation symbol (e.g. *R) and mark  
> 100% of them.  Votes?

Probably the #1 question I'm asked is "does this mean I need to  
restart?". +1 for marking everything.
--
Jim Nasby                                            jim@nasby.net
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)




Re: Postgresql.conf cleanup

From
Tom Lane
Date:
Josh Berkus <josh@agliodbs.com> writes:
>> I'm working on cleaning up postgresql.conf and pg_settings for the 
>> ...

>> seq_scan_cost: this is independant of all of the other _costs.

So?  All the other costs are independent of it, too.  I don't understand
what problem you have with it.

>> (change requires restart): this phrase appears over 20 times in the 
>> notes.  This is enough times to be really repetitive and take up a lot 
>> of scrolling space, while not actually covering all startup-time 
>> parameters.  We should either (a) remove all such notes and rely on 
>> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. 

That was put in deliberately not long ago, so I doubt (a) will pass.
(b) seems fine to me.

>> transaction_isolation and transaction_read_only appear more than once in 
>> the pg_settings pseudo_table.

Not for me.

> # work_mem = ( RAM * 0.5 ) / max_connections, or less

That seems guaranteed to drive people into swap hell, unless they
execute only trivial queries.

> # wal_buffers = 1MB

Is there really evidence in favor of such a high setting for this,
either?  (I expect the walwriter in the async-commit patch will change
the landscape here, btw.)

> # max_fsm_pages = expected database size * 0.1

This might be too small.

> # checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)

This seems definitely too small --- for write-intensive databases I like
to set it to 30 or so, which should eat about a GB if I did the
arithmetic right.

> #explain_pretty_print = on

Putting this under "planner options" is wrong and illogical.

The file seems to be missing the effects of some recently committed
patches, eg, bgwriter_all_percent shouldn't be there anymore.
        regards, tom lane


Re: Postgresql.conf cleanup

From
"Joshua D. Drake"
Date:
Tom Lane wrote:

>>> (change requires restart): this phrase appears over 20 times in the 
>>> notes.  This is enough times to be really repetitive and take up a lot 
>>> of scrolling space, while not actually covering all startup-time 
>>> parameters.  We should either (a) remove all such notes and rely on 
>>> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. 
> 
> That was put in deliberately not long ago, so I doubt (a) will pass.
> (b) seems fine to me.

+1 on (b), -1 on (a)


>> # work_mem = ( RAM * 0.5 ) / max_connections, or less
> 
> That seems guaranteed to drive people into swap hell, unless they
> execute only trivial queries.

Maybe he meant .05, which would be semi-reasonable?


>> # checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)
> 
> This seems definitely too small --- for write-intensive databases I like
> to set it to 30 or so, which should eat about a GB if I did the
> arithmetic right.

Hmpf, I set it to 30 just to get it out of the way. I would agree that 
8-16 is too small.

Sincerely,

Joshua D. Drake


-- 
      === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive  PostgreSQL solutions since 1997             http://www.commandprompt.com/

Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



Re: Postgresql.conf cleanup

From
Greg Smith
Date:
On Mon, 2 Jul 2007, Tom Lane wrote:

>> # wal_buffers = 1MB
> Is there really evidence in favor of such a high setting for this,
> either?

I noticed consistant improvements in throughput on pgbench results with 
lots of clients going from the default to 256KB, flatlining above that; it 
seemed sufficiently large for any system I've used.  I've taken to using 
1MB anyway nowadays because others suggested that number, and it seemed to 
be well beyond the useful range and thus never likely to throttle 
anything.  Is there any downside to it being larger than necessary beyond 
what seems like a trivial amount of additional RAM?

>> # checkpoint_segments = 8 to 16 if you have the disk space (0.3 to 0.6 GB)
> This seems definitely too small --- for write-intensive databases I like
> to set it to 30 or so, which should eat about a GB if I did the
> arithmetic right.

You did--I approximate larger values in my head by saying 1GB at 30 
segments and scaling up from there.  But don't forget this is impacted by 
the LDC change, with the segments expected to be active now

(2 + checkpoint_completion_target) * checkpoint_segments + 1

so with a default install setting the segments to 30 will creep that up to 
closer to a 1.2GB footprint.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD


Re: Postgresql.conf cleanup

From
Tom Lane
Date:
Greg Smith <gsmith@gregsmith.com> writes:
> On Mon, 2 Jul 2007, Tom Lane wrote:
>>> # wal_buffers = 1MB
>> Is there really evidence in favor of such a high setting for this,
>> either?

> I noticed consistant improvements in throughput on pgbench results with 
> lots of clients going from the default to 256KB, flatlining above that; it 
> seemed sufficiently large for any system I've used.  I've taken to using 
> 1MB anyway nowadays because others suggested that number, and it seemed to 
> be well beyond the useful range and thus never likely to throttle 
> anything.  Is there any downside to it being larger than necessary beyond 
> what seems like a trivial amount of additional RAM?

There might be some value in keeping wal_buffers small enough to fit in
L2 cache; not sure.

But pgbench is not really the poster child for large wal_buffers,
because it consists exclusively of short transactions.  The gain from
enlarging wal_buffers stops the moment it passes your largest
time-between-commits, since a commit has to flush out whatever's in
there.

There's probably not much point in arguing this now, though; once the
async commit patch is in there we will have to re-measure all the
behavior and develop new recommendations (and, quite possibly, a new
default value).  The existence of the walwriter will reduce the useful
size of wal_buffers, but the existence of async commit might increase it.
        regards, tom lane


Re: Postgresql.conf cleanup

From
Peter Eisentraut
Date:
Am Montag, 2. Juli 2007 13:03 schrieb Josh Berkus:
> (change requires restart): this phrase appears over 20 times in the
> notes.  This is enough times to be really repetitive and take up a lot
> of scrolling space, while not actually covering all startup-time
> parameters.

Which ones are missing?

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/


Re: Postgresql.conf cleanup

From
Bruce Momjian
Date:
Josh, is any of this happening for 8.3?

---------------------------------------------------------------------------

Josh Berkus wrote:
> All,
> 
> I'm working on cleaning up postgresql.conf and pg_settings for the 
> release.  Attached is a sample WIP.  It's not in patch form because I'm 
> not done yet; I've just been editing postgresql.conf and need to fix the 
> docs and pg_settings to match.
> 
> Issues encountered and changes made:
> 
> PostgreSQL.conf
> ----------------
> 
> suggestions: added section with the 7 most important obvious settings at 
> the top and suggestions on how to calculate them.  If people like this, 
> I'll add it to the Tutorial in the docs as well.
> 
> seq_scan_cost: this is independant of all of the other _costs.  I can't 
> think of any way in which that doesn't make the whole set of costs 
> unmanageable.  For example, if you want to change seq_scan_cost in order 
> to make query cost more-or-less match up with ms execution time, you 
> have to modify all 6 settings.   If we do implement per-tablespace 
> costs, then we'll need per-tablespace random_page_cost as well.  Or am I 
> missing something?
> 
> (change requires restart): this phrase appears over 20 times in the 
> notes.  This is enough times to be really repetitive and take up a lot 
> of scrolling space, while not actually covering all startup-time 
> parameters.  We should either (a) remove all such notes and rely on 
> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. 
>   Votes?
> 
> Vacuum: all vacuum & autovacuum parameters put under their own section.
> 
> Client Cost Defaults: this section became a "catch-all" for all userset 
> parameters which people weren't sure what to do with.  I've divided it 
> into logical subsections, and moved some parameters to other sections 
> where they logically belong (for example, explain_pretty_print belongs 
> in Query Tuning).
> 
> pg_settings issues
> --------------------
> 
> transaction_isolation and transaction_read_only appear more than once in 
> the pg_settings pseudo_table.   The setting column is supposed to be unique.
> 
> 
> Given the amount of cleanup/improvement which I'm seeing as necessary 
> for the GUCs, I'm wondering if I put this off too long for 8.3.
> 
> --Josh
> 
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +


Re: Postgresql.conf cleanup

From
"Joshua D. Drake"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian wrote:
> Josh, is any of this happening for 8.3?

I would be willing to take a stab at this and submit by Monday.

Joshua D. Drake


> 
> ---------------------------------------------------------------------------
> 
> Josh Berkus wrote:
>> All,
>>
>> I'm working on cleaning up postgresql.conf and pg_settings for the 
>> release.  Attached is a sample WIP.  It's not in patch form because I'm 
>> not done yet; I've just been editing postgresql.conf and need to fix the 
>> docs and pg_settings to match.
>>
>> Issues encountered and changes made:
>>
>> PostgreSQL.conf
>> ----------------
>>
>> suggestions: added section with the 7 most important obvious settings at 
>> the top and suggestions on how to calculate them.  If people like this, 
>> I'll add it to the Tutorial in the docs as well.
>>
>> seq_scan_cost: this is independant of all of the other _costs.  I can't 
>> think of any way in which that doesn't make the whole set of costs 
>> unmanageable.  For example, if you want to change seq_scan_cost in order 
>> to make query cost more-or-less match up with ms execution time, you 
>> have to modify all 6 settings.   If we do implement per-tablespace 
>> costs, then we'll need per-tablespace random_page_cost as well.  Or am I 
>> missing something?
>>
>> (change requires restart): this phrase appears over 20 times in the 
>> notes.  This is enough times to be really repetitive and take up a lot 
>> of scrolling space, while not actually covering all startup-time 
>> parameters.  We should either (a) remove all such notes and rely on 
>> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. New tasks
>>   Votes?
>>
>> Vacuum: all vacuum & autovacuum parameters put under their own section.
>>
>> Client Cost Defaults: this section became a "catch-all" for all userset 
>> parameters which people weren't sure what to do with.  I've divided it 
>> into logical subsections, and moved some parameters to other sections 
>> where they logically belong (for example, explain_pretty_print belongs 
>> in Query Tuning).
>>
>> pg_settings issues
>> --------------------
>>
>> transaction_isolation and transaction_read_only appear more than once in 
>> the pg_settings pseudo_table.   The setting column is supposed to be unique.
>>
>>
>> Given the amount of cleanup/improvement which I'm seeing as necessary 
>> for the GUCs, I'm wondering if I put this off too long for 8.3.
>>
>> --Josh
>>
>>
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 5: don't forget to increase your free space map settings
> 


- --
     === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/        UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG6WfUATb/zqfZUUQRAqK5AJ46wEGl3MklaS1Y/cdyOKtUAf15WQCdFc3y
8lmhvlh/NiLwOExeGlDH75k=
=14WD
-----END PGP SIGNATURE-----


Re: Postgresql.conf cleanup

From
"Joshua D. Drake"
Date:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian wrote:
> Josh, is any of this happening for 8.3?

Should I run with this or let it lay?

> 
> ---------------------------------------------------------------------------
> 
> Josh Berkus wrote:
>> All,
>>
>> I'm working on cleaning up postgresql.conf and pg_settings for the 
>> release.  Attached is a sample WIP.  It's not in patch form because I'm 
>> not done yet; I've just been editing postgresql.conf and need to fix the 
>> docs and pg_settings to match.
>>
>> Issues encountered and changes made:
>>
>> PostgreSQL.conf
>> ----------------
>>
>> suggestions: added section with the 7 most important obvious settings at 
>> the top and suggestions on how to calculate them.  If people like this, 
>> I'll add it to the Tutorial in the docs as well.
>>
>> seq_scan_cost: this is independant of all of the other _costs.  I can't 
>> think of any way in which that doesn't make the whole set of costs 
>> unmanageable.  For example, if you want to change seq_scan_cost in order 
>> to make query cost more-or-less match up with ms execution time, you 
>> have to modify all 6 settings.   If we do implement per-tablespace 
>> costs, then we'll need per-tablespace random_page_cost as well.  Or am I 
>> missing something?
>>
>> (change requires restart): this phrase appears over 20 times in the 
>> notes.  This is enough times to be really repetitive and take up a lot 
>> of scrolling space, while not actually covering all startup-time 
>> parameters.  We should either (a) remove all such notes and rely on 
>> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. 
>>   Votes?
>>
>> Vacuum: all vacuum & autovacuum parameters put under their own section.
>>
>> Client Cost Defaults: this section became a "catch-all" for all userset 
>> parameters which people weren't sure what to do with.  I've divided it 
>> into logical subsections, and moved some parameters to other sections 
>> where they logically belong (for example, explain_pretty_print belongs 
>> in Query Tuning).
>>
>> pg_settings issues
>> --------------------
>>
>> transaction_isolation and transaction_read_only appear more than once in 
>> the pg_settings pseudo_table.   The setting column is supposed to be unique.
>>
>>
>> Given the amount of cleanup/improvement which I'm seeing as necessary 
>> for the GUCs, I'm wondering if I put this off too long for 8.3.
>>
>> --Josh
>>
>>
>>
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 5: don't forget to increase your free space map settings
> 


- --
     === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/        UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG6XLDATb/zqfZUUQRAt4eAJ93xvOvRRIWnqOgZzj1LmnZF1TvGwCfbMd9
Sm/parspTeRDOqZ7KQ3mHXM=
=Uv7U
-----END PGP SIGNATURE-----


Re: Postgresql.conf cleanup

From
Josh Berkus
Date:
Bruce,

> Josh, is any of this happening for 8.3?

Hmmm, just the format cleanup.  I haven't heard any objections, but I haven't 
heard any comments on the underlying broken functionality either (like 
seq_scan_cost), which are beyond me to fix.

Patch next week.

-- 
Josh Berkus
PostgreSQL @ Sun
San Francisco


Re: Postgresql.conf cleanup

From
Bruce Momjian
Date:
This has been saved for the 8.4 release:
http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Josh Berkus wrote:
> All,
> 
> I'm working on cleaning up postgresql.conf and pg_settings for the 
> release.  Attached is a sample WIP.  It's not in patch form because I'm 
> not done yet; I've just been editing postgresql.conf and need to fix the 
> docs and pg_settings to match.
> 
> Issues encountered and changes made:
> 
> PostgreSQL.conf
> ----------------
> 
> suggestions: added section with the 7 most important obvious settings at 
> the top and suggestions on how to calculate them.  If people like this, 
> I'll add it to the Tutorial in the docs as well.
> 
> seq_scan_cost: this is independant of all of the other _costs.  I can't 
> think of any way in which that doesn't make the whole set of costs 
> unmanageable.  For example, if you want to change seq_scan_cost in order 
> to make query cost more-or-less match up with ms execution time, you 
> have to modify all 6 settings.   If we do implement per-tablespace 
> costs, then we'll need per-tablespace random_page_cost as well.  Or am I 
> missing something?
> 
> (change requires restart): this phrase appears over 20 times in the 
> notes.  This is enough times to be really repetitive and take up a lot 
> of scrolling space, while not actually covering all startup-time 
> parameters.  We should either (a) remove all such notes and rely on 
> docs, or (b) make an annotation symbol (e.g. *R) and mark 100% of them. 
>   Votes?
> 
> Vacuum: all vacuum & autovacuum parameters put under their own section.
> 
> Client Cost Defaults: this section became a "catch-all" for all userset 
> parameters which people weren't sure what to do with.  I've divided it 
> into logical subsections, and moved some parameters to other sections 
> where they logically belong (for example, explain_pretty_print belongs 
> in Query Tuning).
> 
> pg_settings issues
> --------------------
> 
> transaction_isolation and transaction_read_only appear more than once in 
> the pg_settings pseudo_table.   The setting column is supposed to be unique.
> 
> 
> Given the amount of cleanup/improvement which I'm seeing as necessary 
> for the GUCs, I'm wondering if I put this off too long for 8.3.
> 
> --Josh
> 
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings

--  Bruce Momjian  <bruce@momjian.us>          http://momjian.us EnterpriseDB
http://www.enterprisedb.com
 + If your life is a hard drive, Christ can be your backup. +