Thread: TODO questions
While marking up TODO for potential new-hacker items I've run across some items that probably need more explanation: o Allow commenting of variables in postgresql.conf to restore them to defaults This doesn't work already? * Allow triggers to be disabled [trigger] Isn't this going to be in 8.1? * SQL*Net listener that makes PostgreSQL appear as an Oracle database to clients Any reason not to do this for other databases; notably MySQL? o Do VACUUM FULL if table is nearly empty? Since that results in an exclusive table lock (which is vastly different from regular vacuum), wouldn't it be better to just throw a warning? * Reduce WAL traffic so only modified values are written rather than entire rows? Shouldn't this be marked as depending on eliminate need to write full pages? (At least ISTM it won't do any good as long as we're writing full pages) -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
Jim C. Nasby wrote: > While marking up TODO for potential new-hacker items I've run across > some items that probably need more explanation: > > o Allow commenting of variables in postgresql.conf to restore them > to defaults > > This doesn't work already? The idea here is the when you comment something out, it should restore its default. Right now it keeps the previously uncommented out value, which confuses people. New text is: o Allow commenting of variables in postgresql.conf to restore them to defaults Currently, if a variable is commented out, it keeps the previous uncommented value until a server restarted. > * Allow triggers to be disabled [trigger] > > Isn't this going to be in 8.1? Yes, just marked it as done because it went into CVS 12 hours ago. > * SQL*Net listener that makes PostgreSQL appear as an Oracle database > to clients > > Any reason not to do this for other databases; notably MySQL? I suppose, but no one has asked for it, while they have for Oracle. > > o Do VACUUM FULL if table is nearly empty? > > Since that results in an exclusive table lock (which is vastly different > from regular vacuum), wouldn't it be better to just throw a warning? Good point. I guess that's why the question mark was there. New text: o Suggest a VACUUM FULL if table is nearly empty > * Reduce WAL traffic so only modified values are written rather than > entire rows? > > Shouldn't this be marked as depending on eliminate need to write full > pages? (At least ISTM it won't do any good as long as we're writing full > pages) Not really --- the per-row writes and the full page writes are two different operations for two different purposes. The first is for WAL crash recovery, the second is to prevent partial page writes. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > The idea here is the when you comment something out, it should restore > its default. Right now it keeps the previously uncommented out value, > which confuses people. I think it will continue to confuse people. I will bring up the idea again here of simply uncommenting *all* settings, which would make things very clear and also make our conf file work like most others. - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200508241037 https://www.biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkMMhngACgkQvJuQZxSWSsh6HQCgmltyTuSzRwUyP+y0IRZbWIpR eN0AoKGF5uFhtD3cM74I7mTfHawoCDLo =gBav -----END PGP SIGNATURE-----
Bruce Momjian <pgman@candle.pha.pa.us> writes: > Jim C. Nasby wrote: >> o Allow commenting of variables in postgresql.conf to restore them >> to defaults >> >> This doesn't work already? > The idea here is the when you comment something out, it should restore > its default. Right now it keeps the previously uncommented out value, > which confuses people. But the contrary position is that a comment is a comment, not something that should act to change the state of the postmaster. It's probably worth pointing out that there's not consensus about all of the TODO items, particularly not the ones Bruce has marked with question marks. regards, tom lane
>>The idea here is the when you comment something out, it should restore >>its default. Right now it keeps the previously uncommented out value, >>which confuses people. >> >> > >But the contrary position is that a comment is a comment, not something >that should act to change the state of the postmaster. > > Not in any conf I have ever seen. If I comment out a parameter I expect that the parameter will either be disabled or set to the default depending on the parameter. #fsync = false Should mean: fsync = false -- fsync = true Should mean: fsync = true -- #fsync = true: Should mean: fsync = false If false is the default value. -- Sincerely, Joshua D. Drake >It's probably worth pointing out that there's not consensus about all >of the TODO items, particularly not the ones Bruce has marked with >question marks. > > regards, tom lane > >---------------------------(end of broadcast)--------------------------- >TIP 3: Have you checked our extensive FAQ? > > http://www.postgresql.org/docs/faq > >
Tom Lane wrote: >But the contrary position is that a comment is a comment, not something >that should act to change the state of the postmaster. > > > > I think that's a mis-statement of the issue, as I understand it, which seems to me to be this: Should the absence of an explicit setting in the config file result in keeping the previous setting of the item or in resetting it to the default value? Of course, the question is made more complex by the fact that you can set things by means other than the config file, and having your settings revert to some default might indeed cause some surprises. Notwithstanding that, I'm fairly firmly in the revert to default camp - I think on balance it conforms to the principle of least surprise. cheers andrew
Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > >>Jim C. Nasby wrote: >> >>>o Allow commenting of variables in postgresql.conf to restore them >>>to defaults >>> >>>This doesn't work already? > > >>The idea here is the when you comment something out, it should restore >>its default. Right now it keeps the previously uncommented out value, >>which confuses people. > > > But the contrary position is that a comment is a comment, not something > that should act to change the state of the postmaster. > I understand what you mean, but to be fair, it's not the comment that is changing the behavior of the postmaster. Rather, it's the absence of a setting in the configuration file that would suggest using the default. It seems to me like a question of whether the configuration file should be treated as: a) the set of options for the postmaster b) a set of changes to the options for the postmaster The current behavior is (b), but many think (a) is more intuitive in the context of a configuration file. Perhaps (b) would be more intuitive if it were in the context of (for example) an admin tool that took options on standard input. Regards,Jeff Davis
Tom Lane <tgl@sss.pgh.pa.us> writes: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > > The idea here is the when you comment something out, it should restore > > its default. Right now it keeps the previously uncommented out value, > > which confuses people. > > But the contrary position is that a comment is a comment, not something > that should act to change the state of the postmaster. As someone else said I think there's a fundamental difference here on what "reloading" means. In every other system I've seen, when you "reload" a config file the system goes through the exact same process (semantically at least) that it does when starting up. Ie, it start with a fresh slate of defaults and loads the config file which sets parameters and overrides those defaults. The Postgres way is that it remembers the old values and loads the config files on top of that. You end up with a situation equivalent to having the new config file tacked onto the old one. The problem with the Postgres semantics is that you end up with a system in a state that isn't represented in either the new config file or the old one. This means if you restart Postgres will come up in a different state from what was running. The conventional semantics give the sysadmin a nice guarantee that he or she can reload the config file and if it works then he can be confident that the server is using the same configuration that it will be using after a restart. I fear a lot of sysadmins after being bitten by this confusion once will decide that it's unsafe to simply reload config files in Postgres and it's necessary to stop and start the server to be sure the new config file is correct and won't cause problems after a subsequent restart. -- greg
OK, TODO updated: o %Allow commenting of variables in postgresql.conf to restore them to defaults Currently, if a variable is commented out, it keeps the previous uncommented value until a server restarted. Logically, a reload should set the same values as a server restart. --------------------------------------------------------------------------- Jim C. Nasby wrote: > On Wed, Aug 24, 2005 at 03:11:31PM -0400, Greg Stark wrote: > > In every other system I've seen, when you "reload" a config file the system > > goes through the exact same process (semantically at least) that it does when > > starting up. Ie, it start with a fresh slate of defaults and loads the config > > file which sets parameters and overrides those defaults. > > I think this is a good summary of the desired behavior, and that it > should be added to the TODO description. > -- > Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com > Pervasive Software http://pervasive.com 512-569-9461 > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
On Wed, Aug 24, 2005 at 03:11:31PM -0400, Greg Stark wrote: > In every other system I've seen, when you "reload" a config file the system > goes through the exact same process (semantically at least) that it does when > starting up. Ie, it start with a fresh slate of defaults and loads the config > file which sets parameters and overrides those defaults. I think this is a good summary of the desired behavior, and that it should be added to the TODO description. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
Robert Treat wrote: > I've always been of the impression that this idea just wont work. For > example, if I set shared_buffers to some number, start my database, then > comment the line out and "reload" my conf file, it just isnt going to reset > to the default. (Or at least to make it do so requires a *lot* more work than > just modifing the guc / conf file mechanisms). True, but you can't modify shared_buffers anyway unless you restart, so people should realize that. --------------------------------------------------------------------------- > IMHO the suggestion to just uncomment out all of the variables to begin with > would cut down on confusion a great deal and actually be easy to implement, > and as such is probably the way to go for now. > > Robert Treat > > On Wednesday 24 August 2005 15:43, Bruce Momjian wrote: > > OK, TODO updated: > > > > o %Allow commenting of variables in postgresql.conf to restore them > > to defaults > > > > Currently, if a variable is commented out, it keeps the > > previous uncommented value until a server restarted. > > Logically, a reload should set the same values as a > > server restart. > > > > > > --------------------------------------------------------------------------- > > > > Jim C. Nasby wrote: > > > On Wed, Aug 24, 2005 at 03:11:31PM -0400, Greg Stark wrote: > > > > In every other system I've seen, when you "reload" a config file the > > > > system goes through the exact same process (semantically at least) that > > > > it does when starting up. Ie, it start with a fresh slate of defaults > > > > and loads the config file which sets parameters and overrides those > > > > defaults. > > > > > > I think this is a good summary of the desired behavior, and that it > > > should be added to the TODO description. > > > -- > > > Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com > > > Pervasive Software http://pervasive.com 512-569-9461 > > -- > Robert Treat > Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
I've always been of the impression that this idea just wont work. For example, if I set shared_buffers to some number, start my database, then comment the line out and "reload" my conf file, it just isnt going to reset to the default. (Or at least to make it do so requires a *lot* more work than just modifing the guc / conf file mechanisms). IMHO the suggestion to just uncomment out all of the variables to begin with would cut down on confusion a great deal and actually be easy to implement, and as such is probably the way to go for now. Robert Treat On Wednesday 24 August 2005 15:43, Bruce Momjian wrote: > OK, TODO updated: > > o %Allow commenting of variables in postgresql.conf to restore them > to defaults > > Currently, if a variable is commented out, it keeps the > previous uncommented value until a server restarted. > Logically, a reload should set the same values as a > server restart. > > > --------------------------------------------------------------------------- > > Jim C. Nasby wrote: > > On Wed, Aug 24, 2005 at 03:11:31PM -0400, Greg Stark wrote: > > > In every other system I've seen, when you "reload" a config file the > > > system goes through the exact same process (semantically at least) that > > > it does when starting up. Ie, it start with a fresh slate of defaults > > > and loads the config file which sets parameters and overrides those > > > defaults. > > > > I think this is a good summary of the desired behavior, and that it > > should be added to the TODO description. > > -- > > Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com > > Pervasive Software http://pervasive.com 512-569-9461 -- Robert Treat Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL
OK, additional sentence removed. --------------------------------------------------------------------------- Tom Lane wrote: > Bruce Momjian <pgman@candle.pha.pa.us> writes: > > OK, TODO updated: > > o %Allow commenting of variables in postgresql.conf to restore them > > to defaults > > > Currently, if a variable is commented out, it keeps the > > previous uncommented value until a server restarted. > > Logically, a reload should set the same values as a > > server restart. > > That all sounds nice, but unless you intend to fix all the constraints > that force some values to be set-only-at-postmaster-start, it's never > going to be possible to promise that a reload has the same effect as > restarting the server. We could do this for values that are not > PGC_POSTMASTER, but the argument given above for doing it is bogus. > > regards, tom lane > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073
Bruce Momjian <pgman@candle.pha.pa.us> writes: > OK, TODO updated: > o %Allow commenting of variables in postgresql.conf to restore them > to defaults > Currently, if a variable is commented out, it keeps the > previous uncommented value until a server restarted. > Logically, a reload should set the same values as a > server restart. That all sounds nice, but unless you intend to fix all the constraints that force some values to be set-only-at-postmaster-start, it's never going to be possible to promise that a reload has the same effect as restarting the server. We could do this for values that are not PGC_POSTMASTER, but the argument given above for doing it is bogus. regards, tom lane
> -----Original Message----- > From: Bruce Momjian [mailto:pgman@candle.pha.pa.us] > Sent: Wednesday, August 24, 2005 3:40 PM > To: Robert Treat > Cc: pgsql-hackers@postgresql.org; Jim Nasby; Greg Stark; Tom Lane > Subject: Re: [HACKERS] TODO questions > > > Robert Treat wrote: > > I've always been of the impression that this idea just wont > work. For > > example, if I set shared_buffers to some number, start my > database, then > > comment the line out and "reload" my conf file, it just > isnt going to reset > > to the default. (Or at least to make it do so requires a > *lot* more work than > > just modifing the guc / conf file mechanisms). > > True, but you can't modify shared_buffers anyway unless you > restart, so > people should realize that. Which is why I think it would be best to get as close as we can to the configuration that you'd get during a restart andwarn about everything else. So, for example, if you're runing with a non-default shared_buffers and you comment shared_buffersout and do a reload, you should get a warning that shared_buffers has changed from xxx -> yyy but that changewon't take effect until the server is restarted. That way there's absolutely no confusion. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
Tom Lane <tgl@sss.pgh.pa.us> writes: > That all sounds nice, but unless you intend to fix all the constraints > that force some values to be set-only-at-postmaster-start, it's never > going to be possible to promise that a reload has the same effect as > restarting the server. We could do this for values that are not > PGC_POSTMASTER, but the argument given above for doing it is bogus. Well that's true, that's a limitation of Postgres's reloading config files. Certainly I think it should be a goal to avoid any such guc variables where it's worth the effort. That doesn't mean you have to make the "problem" worse and go in a completely different direction. I would say it would be reasonable to print a warning if loading the new config file results in a different value for any guc variable that can't be changed. If that's too awkward then at least it would be nice to put a warning line in the initial default config file to mark any such guc variables. -- greg
Joshua D. Drake wrote: >>> ...when you comment something out, it should restore >> ...the contrary position is that a comment is a comment... > ...If I comment out a parameter I expect... The most unambiguous behavior would be to not have commented out values in the config file at all. If someone wants to change a value and keep a reminder of what the old value was, he's welcome to comment out the original line himself. (assuming that was the thinking behind commenting them out in the first place)
On Wed, Aug 24, 2005 at 10:03:28PM -0700, Ron Mayer wrote: > Joshua D. Drake wrote: > >>>...when you comment something out, it should restore > >>...the contrary position is that a comment is a comment... > >...If I comment out a parameter I expect... > > > The most unambiguous behavior would be to not have > commented out values in the config file at all. > > If someone wants to change a value and keep a reminder of > what the old value was, he's welcome to comment out the > original line himself. (assuming that was the thinking > behind commenting them out in the first place) That only makes sense if you also remove the concept of default values; something I don't think we want to do. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
Ron Mayer <rm_pg@cheapcomplexdevices.com> writes: > The most unambiguous behavior would be to not have > commented out values in the config file at all. Yeah, Robert Treat suggested that upthread, and I think it's been pushed by others too. The only argument I can see against it is that it'll take longer for the system to process such a file; but do we really care about a few more microseconds to respond to SIGHUP? This does not address the problem that changing PGC_POSTMASTER values in the file won't do anything without a postmaster restart. Greg Stark's suggestion of marking each PGC_POSTMASTER variable with a warning comment in postgresql.conf seems reasonable to me, though. So, the low-tech solution to these gripes seems to be:* uncomment all the entries in postgresql.conf* add comments to flagthe values that can't be changed by SIGHUP Can we agree on taking these measures? regards, tom lane
"Jim C. Nasby" <jnasby@pervasive.com> writes: > On Wed, Aug 24, 2005 at 10:03:28PM -0700, Ron Mayer wrote: >> The most unambiguous behavior would be to not have >> commented out values in the config file at all. > That only makes sense if you also remove the concept of default values; > something I don't think we want to do. Well, the hardwired default values are really only there to ensure sanity if the config file fails to provide values. We already have to make sure that the hardwired defaults match what it says in postgresql.conf.sample, if only for documentation reasons. So I'm not seeing a strong argument here. regards, tom lane
On Thu, 25 Aug 2005, Tom Lane wrote: > Ron Mayer <rm_pg@cheapcomplexdevices.com> writes: >> The most unambiguous behavior would be to not have >> commented out values in the config file at all. > > Yeah, Robert Treat suggested that upthread, and I think it's been pushed > by others too. > > The only argument I can see against it is that it'll take longer for the > system to process such a file; but do we really care about a few more > microseconds to respond to SIGHUP? This has probably been mentioned already, but it makes it much harder to see which values have been altered from their default values. At the very least, the default values should be in the comments together with the min and max values. Also, suppose that we change the default value of parameter in a new release. That change wouldn't be picked up by upgraders that keep using their old config file. I don't know how big an issue that is in practice. We could have a special value DEFAULT that would mean that the variable should be set to it's default. The config file would look like this: shared_buffers = DEFAULT # default 1000 work_mem = DEFAULT # default 1024 ... That would avoid the surprise that commenting a line and reloading doesn't actually do anything, while also making it obvious which values have been modified. > This does not address the problem that changing PGC_POSTMASTER values in > the file won't do anything without a postmaster restart. Greg Stark's > suggestion of marking each PGC_POSTMASTER variable with a warning > comment in postgresql.conf seems reasonable to me, though. If we keep the current behaviour, could we also issue a runtime warning if a PGC_POSTMASTER variable was changed? Also, could we issue a warning if a variable is not at it's default value, but is missing from the new config file? - Heikki
Tom Lane wrote: > So, the low-tech solution to these gripes seems to be: > * uncomment all the entries in postgresql.conf > * add comments to flag the values that can't be changed by SIGHUP > > Can we agree on taking these measures? Doesn't this still mean that a SIGHUP may give you a different configuration to a full restart, even if you don't touch any of the values that can't be changed via SIGHUP? Perhaps we could stick with the current commented-out-defaults scheme if SIGHUP reset all settings to their compiled-in defaults before rereading the config file, i.e. the same logic as normal startup would do (except for those cases where we can't change a setting at runtime). -O
On Thu, Aug 25, 2005 at 12:09:21 +0300, Heikki Linnakangas <hlinnaka@iki.fi> wrote: > > This has probably been mentioned already, but it makes it much harder to > see which values have been altered from their default values. At the very > least, the default values should be in the comments together with the min > and max values. I think it can be expected that people who care will comment out the original line and add a new one for the changed setting. Or if worse comes to worse they can get a fresh copy of postgresql.conf and run diff.
* Andrew Dunstan (andrew@dunslane.net) wrote: > Tom Lane wrote: > >But the contrary position is that a comment is a comment, not something > >that should act to change the state of the postmaster. > > I think that's a mis-statement of the issue, as I understand it, which > seems to me to be this: Should the absence of an explicit setting in the > config file result in keeping the previous setting of the item or in > resetting it to the default value? Of course, the question is made more > complex by the fact that you can set things by means other than the > config file, and having your settings revert to some default might > indeed cause some surprises. Notwithstanding that, I'm fairly firmly in > the revert to default camp - I think on balance it conforms to the > principle of least surprise. I agree w/ Andrew on this, revert-to-default *is* what's expected by most users/admins/etc, at least that I know. If we want something to be a differential against the running config then let's pull out whatever can be changed during runtime and write a .sql script that can be called from the backend (set via the .conf) upon start and SIGHUP. I think there'd be a great deal more understanding of how things work if we then tell people that if they want to change things, change the .sql and then run it, or run those commands themselves by hand; while the .conf contains only those things that require a postmaster full restart to be changed. Just my 2c. Thanks, Stephen
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Not in any conf I have ever seen. If I comment out a parameter > I expect that the parameter will either be disabled or set to the > default depending on the parameter. Or throw an error. I'm not sure what the big deal is with defaults - everyone should be tuning their conf files intially anyway, the defaults are just there to get you started. If you need a history, put postgresql.conf into your local revisioning system, starting with the supplied one (the "default") before you make any changes. If everything is uncommented, it all becomes clear exactly what will happen on reload. - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200508250958 https://www.biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAkMNzrsACgkQvJuQZxSWSsgbPgCgqTEOjdxhJ1TCA/Fnl9UGFo/C 6lwAoOc5HGvR1kciqOSbZMaghUakmCpy =+3h5 -----END PGP SIGNATURE-----
On Thu, Aug 25, 2005 at 02:09:10AM -0400, Tom Lane wrote: > "Jim C. Nasby" <jnasby@pervasive.com> writes: > > On Wed, Aug 24, 2005 at 10:03:28PM -0700, Ron Mayer wrote: > >> The most unambiguous behavior would be to not have > >> commented out values in the config file at all. > > > That only makes sense if you also remove the concept of default values; > > something I don't think we want to do. > > Well, the hardwired default values are really only there to ensure > sanity if the config file fails to provide values. We already have to > make sure that the hardwired defaults match what it says in > postgresql.conf.sample, if only for documentation reasons. So I'm > not seeing a strong argument here. So then what happens if someone accidentally deletes something from the config file? Or in-advertently comments it out? IMHO, the best way to 'fix' this in the short term is to put a big warning/notice in the config file letting people know what happens when a setting is commented out/missing. In the long term we should go through the same process for reading config settings on a reload as we do on startup (with the obvious exception of startup-only parameters). -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com 512-569-9461
Tom Lane wrote: > Ron Mayer <rm_pg@cheapcomplexdevices.com> writes: > > The most unambiguous behavior would be to not have > > commented out values in the config file at all. > > Yeah, Robert Treat suggested that upthread, and I think it's been pushed > by others too. > > The only argument I can see against it is that it'll take longer for the > system to process such a file; but do we really care about a few more > microseconds to respond to SIGHUP? > > This does not address the problem that changing PGC_POSTMASTER values in > the file won't do anything without a postmaster restart. Greg Stark's > suggestion of marking each PGC_POSTMASTER variable with a warning > comment in postgresql.conf seems reasonable to me, though. We should issue a warning in the server logs if someone tries to modify a postmastger-only value on reload. Added to TODO: o Issue a warning if a change-on-restart-only postgresql.conf value is modified and the server config filesare reloaded o Mark change-on-restart-only values in postgresql.conf -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001+ If your life is a hard drive, | 13 Roberts Road + Christ can be your backup. | Newtown Square, Pennsylvania19073