Thread: Glitch in handling of postmaster -o options

Glitch in handling of postmaster -o options

From
Tom Lane
Date:
I have just noticed a flaw in the handling of "-o backend-options"
postmaster parameters.  To wit: although these options will be passed
to all backends launched by the postmaster, they aren't passed to
checkpoint, xlog startup, and xlog shutdown subprocesses (everything
that goes through BootstrapMain).  Since BootstrapMain doesn't
recognize the same set of options that PostgresMain does, this is
a necessary restriction.  Unfortunately it means that checkpoint etc.
don't necessarily run with the same options as normal backends.

The particular case that I ran into is that I've been in the habit
of running test postmasters with "-o -F" to suppress fsync.  Kernel
tracing showed that checkpoint processes were issuing fsyncs anyway,
and I eventually realized why: they're not seeing the command line
option.

While that's not a fatal problem, I could imagine *much* more serious
misbehavior from inconsistent settings of some GUC parameters.  Since
backends believe that these parameters have PGC_POSTMASTER priority,
they'll accept changes that they probably oughtn't.  For example,postmaster -o --shared_buffers=N
will cause things to blow up very nicely indeed: backends will have
a value of NBuffers that doesn't agree with what the postmaster has.

I wonder whether we should retire -o.  Or change it so that the
postmaster parses the given options for itself (consequently adjusting
its copies of GUC variables) instead of passing them on to backends
for parsing at backend start time.
        regards, tom lane


Re: Glitch in handling of postmaster -o options

From
Peter Eisentraut
Date:
Tom Lane writes:

> The particular case that I ran into is that I've been in the habit
> of running test postmasters with "-o -F" to suppress fsync.  Kernel
> tracing showed that checkpoint processes were issuing fsyncs anyway,
> and I eventually realized why: they're not seeing the command line
> option.

postmaster -F should be used in this particular case.  I'm not sure about
a general solution.  I don't even know what the set of parameters for
checkpoint processes is or why they could usefully differ from the
postmaster's global settings.

> While that's not a fatal problem, I could imagine *much* more serious
> misbehavior from inconsistent settings of some GUC parameters.  Since
> backends believe that these parameters have PGC_POSTMASTER priority,
> they'll accept changes that they probably oughtn't.  For example,
>     postmaster -o --shared_buffers=N
> will cause things to blow up very nicely indeed: backends will have
> a value of NBuffers that doesn't agree with what the postmaster has.

This is a bug.  PG 7.1 wouldn't let this thing go through but with all the
changes made for the RESET ALL functionality (I think) this has snuck in.

My best guess is that this change was made to allow using
SetConfigOption() in PostgresMain() with options that are really
postmaster-global and are passed down to the backends.  But AFAICS there
aren't any such options anymore.

> I wonder whether we should retire -o.

The only two postgres options which I would consider user-space are -F and
-S.  The former also exists as a postmaster option, but that is newer and
people are used to the older form.  The second one doesn't for obvious
reasons.  With the config file this isn't so much of an issue any longer,
but people do use it.

(It's always been a goal of mine to merge the options that any of the
backend processes accept.  The -S option really is the killer for that.)

-- 
Peter Eisentraut   peter_e@gmx.net   http://funkturm.homeip.net/~peter



Re: Glitch in handling of postmaster -o options

From
Justin Clift
Date:
Tom Lane wrote:
> 
<snip>
> 
> I wonder whether we should retire -o.  Or change it so that the
> postmaster parses the given options for itself (consequently adjusting
> its copies of GUC variables) instead of passing them on to backends
> for parsing at backend start time.

Retiring -o would seem like a good idea.  Just about every person I bump
into that's new to PostgreSQL doesn't get -o right for some time.  It's
simple in concept, but different from how every other package works, so
it confuses newcomers who don't know the difference between the
different parts of PostgreSQL.

It would be good if we could just having options that replace each -o
option (i.e. -F instead of -o '-F', -x -y instead of -o '-x -y') so it's
similar to how other programs command line arguments work.

Regards and best wishes,

Justin Clift

> 
>                         regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly

-- 
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."    - Indira Gandhi


Re: Glitch in handling of postmaster -o options

From
Tom Lane
Date:
Justin Clift <justin@postgresql.org> writes:
> Retiring -o would seem like a good idea.

That was what I was thinking too.  I can think of ways to reimplement
-o options so that they work safely ... but is it worth the trouble?
AFAICS, -o options confuse both people and machines, and have no
redeeming value beyond supporting old startup scripts.  Which could
easily be updated.

Some judicious feature removal may be the best path here.
        regards, tom lane


Re: Glitch in handling of postmaster -o options

From
Marko Kreen
Date:
On Sun, Sep 30, 2001 at 12:54:25AM +0200, Peter Eisentraut wrote:
> Tom Lane writes:
> > While that's not a fatal problem, I could imagine *much* more serious
> > misbehavior from inconsistent settings of some GUC parameters.  Since
> > backends believe that these parameters have PGC_POSTMASTER priority,
> > they'll accept changes that they probably oughtn't.  For example,
> >     postmaster -o --shared_buffers=N
> > will cause things to blow up very nicely indeed: backends will have
> > a value of NBuffers that doesn't agree with what the postmaster has.
> 
> This is a bug.  PG 7.1 wouldn't let this thing go through but with all the
> changes made for the RESET ALL functionality (I think) this has snuck in.
> 
> My best guess is that this change was made to allow using
> SetConfigOption() in PostgresMain() with options that are really
> postmaster-global and are passed down to the backends.  But AFAICS there
> aren't any such options anymore.
> 
> > I wonder whether we should retire -o.

How about putting -o stuff after -p?  That way only postmaster
code can set PGC_POSTMASTER options for a backend, no way for
user to mess up.  ATM this would break -o -F tho'.

-- 
marko



Re: Glitch in handling of postmaster -o options

From
Tom Lane
Date:
Marko Kreen <marko@l-t.ee> writes:
>> I wonder whether we should retire -o.

> How about putting -o stuff after -p?  That way only postmaster
> code can set PGC_POSTMASTER options for a backend, no way for
> user to mess up.  ATM this would break -o -F tho'.

Indeed.  If we're going to force people to change their scripts anyway,
IMHO we might as well go all the way...
        regards, tom lane


Re: Glitch in handling of postmaster -o options

From
Bruce Momjian
Date:
> Marko Kreen <marko@l-t.ee> writes:
> >> I wonder whether we should retire -o.
> 
> > How about putting -o stuff after -p?  That way only postmaster
> > code can set PGC_POSTMASTER options for a backend, no way for
> > user to mess up.  ATM this would break -o -F tho'.

Not sure what you are suggesting here.  Should we keep -o but say all
options after -o are passed to postgres backends:
postmaster -a -b -c -o -f -g -h

In this case, -abc goes to postmaster and -fgh goes to postgres.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Glitch in handling of postmaster -o options

From
Marko Kreen
Date:
On Sun, Sep 30, 2001 at 02:13:34PM -0400, Bruce Momjian wrote:
> > Marko Kreen <marko@l-t.ee> writes:
> > >> I wonder whether we should retire -o.
> > 
> > > How about putting -o stuff after -p?  That way only postmaster
> > > code can set PGC_POSTMASTER options for a backend, no way for
> > > user to mess up.  ATM this would break -o -F tho'.
> 
> Not sure what you are suggesting here.  Should we keep -o but say all
> options after -o are passed to postgres backends:

I am suggesting this.

Like previosly discussed, postmaster -F should be used instead
of postmaster '-o -F'.  Other options with PGC_BACKEND, like -S
keep on working.

-- 
marko


Index: src/backend/postmaster/postmaster.c
===================================================================
RCS file: /opt/cvs/pgsql/pgsql/src/backend/postmaster/postmaster.c,v
retrieving revision 1.243
diff -u -c -r1.243 postmaster.c
*** src/backend/postmaster/postmaster.c    21 Sep 2001 20:31:48 -0000    1.243
--- src/backend/postmaster/postmaster.c    30 Sep 2001 15:35:44 -0000
***************
*** 2035,2048 ****         av[ac++] = debugbuf;     } 
-     /*
-      * Pass any backend switches specified with -o in the postmaster's own
-      * command line.  We assume these are secure. (It's OK to mangle
-      * ExtraOptions since we are now in the child process; this won't
-      * change the postmaster's copy.)
-      */
-     split_opts(av, &ac, ExtraOptions);
-      /* Tell the backend what protocol the frontend is using. */     sprintf(protobuf, "-v%u", port->proto);
av[ac++]= protobuf;
 
--- 2035,2040 ----
***************
*** 2055,2060 ****
--- 2047,2062 ----      StrNCpy(dbbuf, port->database, ARGV_SIZE);     av[ac++] = dbbuf;
+ 
+     /*
+      * Pass any backend switches specified with -o in the postmaster's own
+      * command line.  (It's OK to mangle ExtraOptions since we are now in the
+      * child process; this won't change the postmaster's copy.)
+      *
+      * We dont assume anymore they are secure, now only PGC_BACKEND
+      * options can be specified that way.
+      */
+     split_opts(av, &ac, ExtraOptions);      /*      * Pass the (insecure) option switches from the connection
request.


Re: Glitch in handling of postmaster -o options

From
Tom Lane
Date:
Marko Kreen <marko@l-t.ee> writes:
> I am suggesting this.
> [ code snipped ]

Okay, that would mean that "-o '-S nnn'" still works, but "-o -F"
doesn't.

But ... the thing is, there is no reason for -o to exist anymore other
than backwards compatibility with existing startup scripts.  -o doesn't
do anything you can't do more cleanly and sanely with GUC options
(--sort_mem, etc).  So, I don't really see much value in keeping it
if you're going to break one of the more common usages --- which I'm
sure -o -F is.

Since the problem I identified is not likely to bite very many people,
my vote is not to try to apply a code solution now.  I think we should
leave the code alone, and instead document in 7.2 that -o is deprecated
(and explain what to do instead), with the intention of removing it in
7.3.  Giving people a release cycle's worth of notice seems sufficient.

Possibly we could also take this opportunity to deprecate -S and the
other options that are standing in the way of unified command line
options for postmasters and backends.
        regards, tom lane


Re: Glitch in handling of postmaster -o options

From
Justin Clift
Date:
Hi all,

There seem to be a few namespace conflicts for the options of postgres
and postmaster.  The one's I could identify from the man pages are :

-i -N -o -p -S -s

If we are going to deprecate -o, then we'll need to make sure we also
introduce replacement names where these conflicts are.  This way, in the
future -o can be treated like a 'no-option' and everything would work.

If we notify of the impending deprecation now, to actually occur in 7.3,
would we be best intoducing alternative option names somewhere in the
7.2 beta cycle so people writing scripts for 7.2 can use the new names
and know their scripts will work into the future?

???

Regards and best wishes,

Justin Clift


Tom Lane wrote:
> 
> Marko Kreen <marko@l-t.ee> writes:
> > I am suggesting this.
> > [ code snipped ]
> 
> Okay, that would mean that "-o '-S nnn'" still works, but "-o -F"
> doesn't.
> 
> But ... the thing is, there is no reason for -o to exist anymore other
> than backwards compatibility with existing startup scripts.  -o doesn't
> do anything you can't do more cleanly and sanely with GUC options
> (--sort_mem, etc).  So, I don't really see much value in keeping it
> if you're going to break one of the more common usages --- which I'm
> sure -o -F is.
> 
> Since the problem I identified is not likely to bite very many people,
> my vote is not to try to apply a code solution now.  I think we should
> leave the code alone, and instead document in 7.2 that -o is deprecated
> (and explain what to do instead), with the intention of removing it in
> 7.3.  Giving people a release cycle's worth of notice seems sufficient.
> 
> Possibly we could also take this opportunity to deprecate -S and the
> other options that are standing in the way of unified command line
> options for postmasters and backends.
> 
>                         regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

-- 
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."    - Indira Gandhi


Re: Glitch in handling of postmaster -o options

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Would someone give me a status on this?

I don't think we need any code changes.  If we decide to deprecate -o
(or anything else), it's just a documentation change.  So we can argue
about it during beta ...

>> If we notify of the impending deprecation now, to actually occur in 7.3,
>> would we be best intoducing alternative option names somewhere in the
>> 7.2 beta cycle so people writing scripts for 7.2 can use the new names
>> and know their scripts will work into the future?

The alternative option names already exist, in the form of GUC
variables.  For example, "--sort-mem=NNN" could replace -S NNN.
        regards, tom lane


Re: Glitch in handling of postmaster -o options

From
Bruce Momjian
Date:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Would someone give me a status on this?
> 
> I don't think we need any code changes.  If we decide to deprecate -o
> (or anything else), it's just a documentation change.  So we can argue
> about it during beta ...
> 
> >> If we notify of the impending deprecation now, to actually occur in 7.3,
> >> would we be best intoducing alternative option names somewhere in the
> >> 7.2 beta cycle so people writing scripts for 7.2 can use the new names
> >> and know their scripts will work into the future?
> 
> The alternative option names already exist, in the form of GUC
> variables.  For example, "--sort-mem=NNN" could replace -S NNN.

I don't think we can remove -o behavior during beta because it will
affect people using -S in startup scripts.  I just wanted to know if I
should record this on the TODO list.  Added to TODO:
* Remove behavior of postmaster -o after making  postmaster/postgres flags unique

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Glitch in handling of postmaster -o options

From
Bruce Momjian
Date:
Would someone give me a status on this?

---------------------------------------------------------------------------

> Hi all,
> 
> There seem to be a few namespace conflicts for the options of postgres
> and postmaster.  The one's I could identify from the man pages are :
> 
> -i -N -o -p -S -s
> 
> If we are going to deprecate -o, then we'll need to make sure we also
> introduce replacement names where these conflicts are.  This way, in the
> future -o can be treated like a 'no-option' and everything would work.
> 
> If we notify of the impending deprecation now, to actually occur in 7.3,
> would we be best intoducing alternative option names somewhere in the
> 7.2 beta cycle so people writing scripts for 7.2 can use the new names
> and know their scripts will work into the future?
> 
> ???
> 
> Regards and best wishes,
> 
> Justin Clift
> 
> 
> Tom Lane wrote:
> > 
> > Marko Kreen <marko@l-t.ee> writes:
> > > I am suggesting this.
> > > [ code snipped ]
> > 
> > Okay, that would mean that "-o '-S nnn'" still works, but "-o -F"
> > doesn't.
> > 
> > But ... the thing is, there is no reason for -o to exist anymore other
> > than backwards compatibility with existing startup scripts.  -o doesn't
> > do anything you can't do more cleanly and sanely with GUC options
> > (--sort_mem, etc).  So, I don't really see much value in keeping it
> > if you're going to break one of the more common usages --- which I'm
> > sure -o -F is.
> > 
> > Since the problem I identified is not likely to bite very many people,
> > my vote is not to try to apply a code solution now.  I think we should
> > leave the code alone, and instead document in 7.2 that -o is deprecated
> > (and explain what to do instead), with the intention of removing it in
> > 7.3.  Giving people a release cycle's worth of notice seems sufficient.
> > 
> > Possibly we could also take this opportunity to deprecate -S and the
> > other options that are standing in the way of unified command line
> > options for postmasters and backends.
> > 
> >                         regards, tom lane
> > 
> > ---------------------------(end of broadcast)---------------------------
> > TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
> 
> -- 
> "My grandfather once told me that there are two kinds of people: those
> who work and those who take the credit. He told me to try to be in the
> first group; there was less competition there."
>      - Indira Gandhi
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
> 

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Glitch in handling of postmaster -o options

From
Tom Lane
Date:
Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I don't think we can remove -o behavior during beta because it will
> affect people using -S in startup scripts.

That was *not* the proposal under discussion.  The proposal was to
warn people in the 7.2 documentation that we plan to remove -o in 7.3.

AFAICS there is no backwards-compatible way to clean up these switches,
and so the best bet is to make an incompatible change --- after suitable
warning.        regards, tom lane


Re: Glitch in handling of postmaster -o options

From
Bruce Momjian
Date:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I don't think we can remove -o behavior during beta because it will
> > affect people using -S in startup scripts.
> 
> That was *not* the proposal under discussion.  The proposal was to
> warn people in the 7.2 documentation that we plan to remove -o in 7.3.
> 
> AFAICS there is no backwards-compatible way to clean up these switches,
> and so the best bet is to make an incompatible change --- after suitable
> warning.

OK, it is on the TODO list so we can do it when we want to.  So you are
considering only a documentation warning?  I was thinking we should have
single-letter replacements installed and print a warning to the user
that the -o '-a -b -c' thing will go away in 7.3.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: Glitch in handling of postmaster -o options

From
Bruce Momjian
Date:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > Would someone give me a status on this?
> 
> I don't think we need any code changes.  If we decide to deprecate -o
> (or anything else), it's just a documentation change.  So we can argue
> about it during beta ...
> 
> >> If we notify of the impending deprecation now, to actually occur in 7.3,
> >> would we be best intoducing alternative option names somewhere in the
> >> 7.2 beta cycle so people writing scripts for 7.2 can use the new names
> >> and know their scripts will work into the future?
> 
> The alternative option names already exist, in the form of GUC
> variables.  For example, "--sort-mem=NNN" could replace -S NNN.

OK, the long options already exist and people can use those in 7.2
without the -o, right?

Do you have to have long option support in your OS to use them?  Do we
want to have options that _don't_ have single-letter versions? 
Certainly we can't have single-letter versions of all the GUC options
but do we remove ones that were already there?  I guess we could.

--  Bruce Momjian                        |  http://candle.pha.pa.us pgman@candle.pha.pa.us               |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026