Thread: Glitch in handling of postmaster -o options
I have just noticed a flaw in the handling of "-o backend-options"
postmaster parameters. To wit: although these options will be passed
to all backends launched by the postmaster, they aren't passed to
checkpoint, xlog startup, and xlog shutdown subprocesses (everything
that goes through BootstrapMain). Since BootstrapMain doesn't
recognize the same set of options that PostgresMain does, this is
a necessary restriction. Unfortunately it means that checkpoint etc.
don't necessarily run with the same options as normal backends.
The particular case that I ran into is that I've been in the habit
of running test postmasters with "-o -F" to suppress fsync. Kernel
tracing showed that checkpoint processes were issuing fsyncs anyway,
and I eventually realized why: they're not seeing the command line
option.
While that's not a fatal problem, I could imagine *much* more serious
misbehavior from inconsistent settings of some GUC parameters. Since
backends believe that these parameters have PGC_POSTMASTER priority,
they'll accept changes that they probably oughtn't. For example,postmaster -o --shared_buffers=N
will cause things to blow up very nicely indeed: backends will have
a value of NBuffers that doesn't agree with what the postmaster has.
I wonder whether we should retire -o. Or change it so that the
postmaster parses the given options for itself (consequently adjusting
its copies of GUC variables) instead of passing them on to backends
for parsing at backend start time.
regards, tom lane
Tom Lane writes: > The particular case that I ran into is that I've been in the habit > of running test postmasters with "-o -F" to suppress fsync. Kernel > tracing showed that checkpoint processes were issuing fsyncs anyway, > and I eventually realized why: they're not seeing the command line > option. postmaster -F should be used in this particular case. I'm not sure about a general solution. I don't even know what the set of parameters for checkpoint processes is or why they could usefully differ from the postmaster's global settings. > While that's not a fatal problem, I could imagine *much* more serious > misbehavior from inconsistent settings of some GUC parameters. Since > backends believe that these parameters have PGC_POSTMASTER priority, > they'll accept changes that they probably oughtn't. For example, > postmaster -o --shared_buffers=N > will cause things to blow up very nicely indeed: backends will have > a value of NBuffers that doesn't agree with what the postmaster has. This is a bug. PG 7.1 wouldn't let this thing go through but with all the changes made for the RESET ALL functionality (I think) this has snuck in. My best guess is that this change was made to allow using SetConfigOption() in PostgresMain() with options that are really postmaster-global and are passed down to the backends. But AFAICS there aren't any such options anymore. > I wonder whether we should retire -o. The only two postgres options which I would consider user-space are -F and -S. The former also exists as a postmaster option, but that is newer and people are used to the older form. The second one doesn't for obvious reasons. With the config file this isn't so much of an issue any longer, but people do use it. (It's always been a goal of mine to merge the options that any of the backend processes accept. The -S option really is the killer for that.) -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Tom Lane wrote: > <snip> > > I wonder whether we should retire -o. Or change it so that the > postmaster parses the given options for itself (consequently adjusting > its copies of GUC variables) instead of passing them on to backends > for parsing at backend start time. Retiring -o would seem like a good idea. Just about every person I bump into that's new to PostgreSQL doesn't get -o right for some time. It's simple in concept, but different from how every other package works, so it confuses newcomers who don't know the difference between the different parts of PostgreSQL. It would be good if we could just having options that replace each -o option (i.e. -F instead of -o '-F', -x -y instead of -o '-x -y') so it's similar to how other programs command line arguments work. Regards and best wishes, Justin Clift > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
Justin Clift <justin@postgresql.org> writes:
> Retiring -o would seem like a good idea.
That was what I was thinking too. I can think of ways to reimplement
-o options so that they work safely ... but is it worth the trouble?
AFAICS, -o options confuse both people and machines, and have no
redeeming value beyond supporting old startup scripts. Which could
easily be updated.
Some judicious feature removal may be the best path here.
regards, tom lane
On Sun, Sep 30, 2001 at 12:54:25AM +0200, Peter Eisentraut wrote: > Tom Lane writes: > > While that's not a fatal problem, I could imagine *much* more serious > > misbehavior from inconsistent settings of some GUC parameters. Since > > backends believe that these parameters have PGC_POSTMASTER priority, > > they'll accept changes that they probably oughtn't. For example, > > postmaster -o --shared_buffers=N > > will cause things to blow up very nicely indeed: backends will have > > a value of NBuffers that doesn't agree with what the postmaster has. > > This is a bug. PG 7.1 wouldn't let this thing go through but with all the > changes made for the RESET ALL functionality (I think) this has snuck in. > > My best guess is that this change was made to allow using > SetConfigOption() in PostgresMain() with options that are really > postmaster-global and are passed down to the backends. But AFAICS there > aren't any such options anymore. > > > I wonder whether we should retire -o. How about putting -o stuff after -p? That way only postmaster code can set PGC_POSTMASTER options for a backend, no way for user to mess up. ATM this would break -o -F tho'. -- marko
Marko Kreen <marko@l-t.ee> writes:
>> I wonder whether we should retire -o.
> How about putting -o stuff after -p? That way only postmaster
> code can set PGC_POSTMASTER options for a backend, no way for
> user to mess up. ATM this would break -o -F tho'.
Indeed. If we're going to force people to change their scripts anyway,
IMHO we might as well go all the way...
regards, tom lane
> Marko Kreen <marko@l-t.ee> writes: > >> I wonder whether we should retire -o. > > > How about putting -o stuff after -p? That way only postmaster > > code can set PGC_POSTMASTER options for a backend, no way for > > user to mess up. ATM this would break -o -F tho'. Not sure what you are suggesting here. Should we keep -o but say all options after -o are passed to postgres backends: postmaster -a -b -c -o -f -g -h In this case, -abc goes to postmaster and -fgh goes to postgres. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
On Sun, Sep 30, 2001 at 02:13:34PM -0400, Bruce Momjian wrote: > > Marko Kreen <marko@l-t.ee> writes: > > >> I wonder whether we should retire -o. > > > > > How about putting -o stuff after -p? That way only postmaster > > > code can set PGC_POSTMASTER options for a backend, no way for > > > user to mess up. ATM this would break -o -F tho'. > > Not sure what you are suggesting here. Should we keep -o but say all > options after -o are passed to postgres backends: I am suggesting this. Like previosly discussed, postmaster -F should be used instead of postmaster '-o -F'. Other options with PGC_BACKEND, like -S keep on working. -- marko Index: src/backend/postmaster/postmaster.c =================================================================== RCS file: /opt/cvs/pgsql/pgsql/src/backend/postmaster/postmaster.c,v retrieving revision 1.243 diff -u -c -r1.243 postmaster.c *** src/backend/postmaster/postmaster.c 21 Sep 2001 20:31:48 -0000 1.243 --- src/backend/postmaster/postmaster.c 30 Sep 2001 15:35:44 -0000 *************** *** 2035,2048 **** av[ac++] = debugbuf; } - /* - * Pass any backend switches specified with -o in the postmaster's own - * command line. We assume these are secure. (It's OK to mangle - * ExtraOptions since we are now in the child process; this won't - * change the postmaster's copy.) - */ - split_opts(av, &ac, ExtraOptions); - /* Tell the backend what protocol the frontend is using. */ sprintf(protobuf, "-v%u", port->proto); av[ac++]= protobuf; --- 2035,2040 ---- *************** *** 2055,2060 **** --- 2047,2062 ---- StrNCpy(dbbuf, port->database, ARGV_SIZE); av[ac++] = dbbuf; + + /* + * Pass any backend switches specified with -o in the postmaster's own + * command line. (It's OK to mangle ExtraOptions since we are now in the + * child process; this won't change the postmaster's copy.) + * + * We dont assume anymore they are secure, now only PGC_BACKEND + * options can be specified that way. + */ + split_opts(av, &ac, ExtraOptions); /* * Pass the (insecure) option switches from the connection request.
Marko Kreen <marko@l-t.ee> writes:
> I am suggesting this.
> [ code snipped ]
Okay, that would mean that "-o '-S nnn'" still works, but "-o -F"
doesn't.
But ... the thing is, there is no reason for -o to exist anymore other
than backwards compatibility with existing startup scripts. -o doesn't
do anything you can't do more cleanly and sanely with GUC options
(--sort_mem, etc). So, I don't really see much value in keeping it
if you're going to break one of the more common usages --- which I'm
sure -o -F is.
Since the problem I identified is not likely to bite very many people,
my vote is not to try to apply a code solution now. I think we should
leave the code alone, and instead document in 7.2 that -o is deprecated
(and explain what to do instead), with the intention of removing it in
7.3. Giving people a release cycle's worth of notice seems sufficient.
Possibly we could also take this opportunity to deprecate -S and the
other options that are standing in the way of unified command line
options for postmasters and backends.
regards, tom lane
Hi all, There seem to be a few namespace conflicts for the options of postgres and postmaster. The one's I could identify from the man pages are : -i -N -o -p -S -s If we are going to deprecate -o, then we'll need to make sure we also introduce replacement names where these conflicts are. This way, in the future -o can be treated like a 'no-option' and everything would work. If we notify of the impending deprecation now, to actually occur in 7.3, would we be best intoducing alternative option names somewhere in the 7.2 beta cycle so people writing scripts for 7.2 can use the new names and know their scripts will work into the future? ??? Regards and best wishes, Justin Clift Tom Lane wrote: > > Marko Kreen <marko@l-t.ee> writes: > > I am suggesting this. > > [ code snipped ] > > Okay, that would mean that "-o '-S nnn'" still works, but "-o -F" > doesn't. > > But ... the thing is, there is no reason for -o to exist anymore other > than backwards compatibility with existing startup scripts. -o doesn't > do anything you can't do more cleanly and sanely with GUC options > (--sort_mem, etc). So, I don't really see much value in keeping it > if you're going to break one of the more common usages --- which I'm > sure -o -F is. > > Since the problem I identified is not likely to bite very many people, > my vote is not to try to apply a code solution now. I think we should > leave the code alone, and instead document in 7.2 that -o is deprecated > (and explain what to do instead), with the intention of removing it in > 7.3. Giving people a release cycle's worth of notice seems sufficient. > > Possibly we could also take this opportunity to deprecate -S and the > other options that are standing in the way of unified command line > options for postmasters and backends. > > regards, tom lane > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Would someone give me a status on this?
I don't think we need any code changes. If we decide to deprecate -o
(or anything else), it's just a documentation change. So we can argue
about it during beta ...
>> If we notify of the impending deprecation now, to actually occur in 7.3,
>> would we be best intoducing alternative option names somewhere in the
>> 7.2 beta cycle so people writing scripts for 7.2 can use the new names
>> and know their scripts will work into the future?
The alternative option names already exist, in the form of GUC
variables. For example, "--sort-mem=NNN" could replace -S NNN.
regards, tom lane
> Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Would someone give me a status on this? > > I don't think we need any code changes. If we decide to deprecate -o > (or anything else), it's just a documentation change. So we can argue > about it during beta ... > > >> If we notify of the impending deprecation now, to actually occur in 7.3, > >> would we be best intoducing alternative option names somewhere in the > >> 7.2 beta cycle so people writing scripts for 7.2 can use the new names > >> and know their scripts will work into the future? > > The alternative option names already exist, in the form of GUC > variables. For example, "--sort-mem=NNN" could replace -S NNN. I don't think we can remove -o behavior during beta because it will affect people using -S in startup scripts. I just wanted to know if I should record this on the TODO list. Added to TODO: * Remove behavior of postmaster -o after making postmaster/postgres flags unique -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Would someone give me a status on this? --------------------------------------------------------------------------- > Hi all, > > There seem to be a few namespace conflicts for the options of postgres > and postmaster. The one's I could identify from the man pages are : > > -i -N -o -p -S -s > > If we are going to deprecate -o, then we'll need to make sure we also > introduce replacement names where these conflicts are. This way, in the > future -o can be treated like a 'no-option' and everything would work. > > If we notify of the impending deprecation now, to actually occur in 7.3, > would we be best intoducing alternative option names somewhere in the > 7.2 beta cycle so people writing scripts for 7.2 can use the new names > and know their scripts will work into the future? > > ??? > > Regards and best wishes, > > Justin Clift > > > Tom Lane wrote: > > > > Marko Kreen <marko@l-t.ee> writes: > > > I am suggesting this. > > > [ code snipped ] > > > > Okay, that would mean that "-o '-S nnn'" still works, but "-o -F" > > doesn't. > > > > But ... the thing is, there is no reason for -o to exist anymore other > > than backwards compatibility with existing startup scripts. -o doesn't > > do anything you can't do more cleanly and sanely with GUC options > > (--sort_mem, etc). So, I don't really see much value in keeping it > > if you're going to break one of the more common usages --- which I'm > > sure -o -F is. > > > > Since the problem I identified is not likely to bite very many people, > > my vote is not to try to apply a code solution now. I think we should > > leave the code alone, and instead document in 7.2 that -o is deprecated > > (and explain what to do instead), with the intention of removing it in > > 7.3. Giving people a release cycle's worth of notice seems sufficient. > > > > Possibly we could also take this opportunity to deprecate -S and the > > other options that are standing in the way of unified command line > > options for postmasters and backends. > > > > regards, tom lane > > > > ---------------------------(end of broadcast)--------------------------- > > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > > -- > "My grandfather once told me that there are two kinds of people: those > who work and those who take the credit. He told me to try to be in the > first group; there was less competition there." > - Indira Gandhi > > ---------------------------(end of broadcast)--------------------------- > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org > -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Bruce Momjian <pgman@candle.pha.pa.us> writes: > I don't think we can remove -o behavior during beta because it will > affect people using -S in startup scripts. That was *not* the proposal under discussion. The proposal was to warn people in the 7.2 documentation that we plan to remove -o in 7.3. AFAICS there is no backwards-compatible way to clean up these switches, and so the best bet is to make an incompatible change --- after suitable warning. regards, tom lane
> Bruce Momjian <pgman@candle.pha.pa.us> writes: > > I don't think we can remove -o behavior during beta because it will > > affect people using -S in startup scripts. > > That was *not* the proposal under discussion. The proposal was to > warn people in the 7.2 documentation that we plan to remove -o in 7.3. > > AFAICS there is no backwards-compatible way to clean up these switches, > and so the best bet is to make an incompatible change --- after suitable > warning. OK, it is on the TODO list so we can do it when we want to. So you are considering only a documentation warning? I was thinking we should have single-letter replacements installed and print a warning to the user that the -o '-a -b -c' thing will go away in 7.3. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
> Bruce Momjian <pgman@candle.pha.pa.us> writes: > > Would someone give me a status on this? > > I don't think we need any code changes. If we decide to deprecate -o > (or anything else), it's just a documentation change. So we can argue > about it during beta ... > > >> If we notify of the impending deprecation now, to actually occur in 7.3, > >> would we be best intoducing alternative option names somewhere in the > >> 7.2 beta cycle so people writing scripts for 7.2 can use the new names > >> and know their scripts will work into the future? > > The alternative option names already exist, in the form of GUC > variables. For example, "--sort-mem=NNN" could replace -S NNN. OK, the long options already exist and people can use those in 7.2 without the -o, right? Do you have to have long option support in your OS to use them? Do we want to have options that _don't_ have single-letter versions? Certainly we can't have single-letter versions of all the GUC options but do we remove ones that were already there? I guess we could. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026