Thread: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

[HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Magnus Hagander
Date:
Magnus Hagander <magnus@hagander.net> writes:
>> 2017-03-18 14:00 GMT+01:00 Peter Eisentraut <peter.eisentraut@2ndquadrant.com>:
>>> I just noticed that createlang and droplang have been listed as
>>> deprecated since PG 9.1.
>>> Do we dare remove them?

> (I'd extend it to all the non-prefixed pg binaries, but let's open that can
> of worms right now, one thing at a time)

> To my mind, these two and createuser/dropuser are the only really serious
> namespacing problems among our standard binaries.  The ones with names
> ending in "db" don't seem likely to cause huge confusion.  I suppose that
> if we were naming it today, "psql" wouldn't get that name; but the chances
> of renaming that one are certainly zero, namespace conflict or no.

> But createuser/dropuser are a real problem, because they certainly could
> be mistaken for system-level utilities.

Yeah, I've seen people use those and get quite confused. Luckily they don't tend to *break* things, but it's definitely namespace pollution.

If we look through the binaries we have now that are not prefixed with pg_:

clusterdb, vacuumdb, reindexdb - while not clear that they are about *postgres*, it's pretty clear they're about a database. I wouldn't name them without pg_ today, but I doubt it's worth chaning. They also add actual value (being able to process multiple things)

createdb, dropdb - also not clear they're about postgres, more likely to be used by mistake but not that bad. That said, do they add any *value* beyond what you can do with psql -c "CREATE DATABASE"? I don't really see one, so I'd suggest dropping these too.

createuser, dropuser - definitely pollutes the namespace, people do sometimes try them for the wrong thing. Unlike the db ones they do add value though -- I don't think we have a psql way of in a single command doing what --pwprompt on createuser does, do we? But given that we are in the process of breaking a lot of other scripts for 10, perhaps we should rename it to pg_createuser?

--

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Tom Lane
Date:
Magnus Hagander <magnus@hagander.net> writes:
> createuser, dropuser - definitely pollutes the namespace, people do
> sometimes try them for the wrong thing. Unlike the db ones they do add
> value though -- I don't think we have a psql way of in a single command
> doing what --pwprompt on createuser does, do we? But given that we are in
> the process of breaking a lot of other scripts for 10, perhaps we should
> rename it to pg_createuser?

I'm not particularly on board with arguments like "we already broke a ton
of stuff so let's break some more".  Eventually you've managed to create
a daunting barrier to upgrading at all.

I think a more reasonable way to proceed is to install symlinks
pg_createuser -> createuser (or the other direction), mark the older names
as deprecated, and announce that we'll remove the old names a few releases
from now.  That gives people time to adjust.

Maybe we should handle createdb likewise, rather than just kicking it to
the curb.  I know I use it quite often; it's less typing than psql -c
'create database ...' postgres, and still would be with a pg_ prefix.
        regards, tom lane



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
"Joshua D. Drake"
Date:
On 03/18/2017 01:12 PM, Magnus Hagander wrote:
> Magnus Hagander <magnus@hagander.net <mailto:magnus@hagander.net>> writes:
>>> 2017-03-18 14:00 GMT+01:00 Peter Eisentraut <peter.eisentraut@2ndquadrant.com
<mailto:peter.eisentraut@2ndquadrant.com>>:

> createuser, dropuser - definitely pollutes the namespace, people do
> sometimes try them for the wrong thing. Unlike the db ones they do add
> value though -- I don't think we have a psql way of in a single command
> doing what --pwprompt on createuser does, do we? But given that we are
> in the process of breaking a lot of other scripts for 10, perhaps we
> should rename it to pg_createuser?

We have one chance in the near future to shake things up, break things 
for the better and lose a lot of long time issues. Making things 
consistent and declarative (pg_ for example) is a great opportunity.

+1

JD


-- 
Command Prompt, Inc.                  http://the.postgres.company/                        +1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Andreas Karlsson
Date:
On 03/18/2017 09:12 PM, Magnus Hagander wrote:
> createdb, dropdb - also not clear they're about postgres, more likely to
> be used by mistake but not that bad. That said, do they add any *value*
> beyond what you can do with psql -c "CREATE DATABASE"? I don't really
> see one, so I'd suggest dropping these too.

The value they add is that they quote the database name and options 
correctly which makes them easier to use safely and reliably in shell 
scripts. And unless I am missing something obvious I do not think there 
is any easy way for a beginner to do this with just psql.

Andreas



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Magnus Hagander
Date:


On Sun, Mar 19, 2017 at 1:44 PM, Andreas Karlsson <andreas@proxel.se> wrote:
On 03/18/2017 09:12 PM, Magnus Hagander wrote:
createdb, dropdb - also not clear they're about postgres, more likely to
be used by mistake but not that bad. That said, do they add any *value*
beyond what you can do with psql -c "CREATE DATABASE"? I don't really
see one, so I'd suggest dropping these too.

The value they add is that they quote the database name and options correctly which makes them easier to use safely and reliably in shell scripts. And unless I am missing something obvious I do not think there is any easy way for a beginner to do this with just psql.

Good point, I hadn't thought of that. I guess I just generally make sure I don't use names that require quoting myself, but I can definitely see how that's an actual value-add. 

--

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Magnus Hagander
Date:


On Sat, Mar 18, 2017 at 9:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
> createuser, dropuser - definitely pollutes the namespace, people do
> sometimes try them for the wrong thing. Unlike the db ones they do add
> value though -- I don't think we have a psql way of in a single command
> doing what --pwprompt on createuser does, do we? But given that we are in
> the process of breaking a lot of other scripts for 10, perhaps we should
> rename it to pg_createuser?

I'm not particularly on board with arguments like "we already broke a ton
of stuff so let's break some more".  Eventually you've managed to create
a daunting barrier to upgrading at all.

The argument is more that if we are going to break it, now is a good time to do it because we are already forcing people to review their scripts.

If we decide not to break it at all that's one thing. But if we *are* going to break it, it's better to do it in 10 than in 11.

 
I think a more reasonable way to proceed is to install symlinks
pg_createuser -> createuser (or the other direction), mark the older names
as deprecated, and announce that we'll remove the old names a few releases
from now.  That gives people time to adjust.

I'd suggest doing it in the other direction, but yeah, that seems like a softer way to handle it. As long as we clearly document them as such. Perhaps we should even have them output a little "hey you should be using pg_xyz" if they are used by the wrong name, but I wonder if that might break things.


Maybe we should handle createdb likewise, rather than just kicking it to
the curb.  I know I use it quite often; it's less typing than psql -c
'create database ...' postgres, and still would be with a pg_ prefix.

As long as they have a pg_ prefix, I don't see much harm in them being there, they're tiny. It's not like they're a huge maintenance burden. 

--

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Stephen Frost
Date:
* Magnus Hagander (magnus@hagander.net) wrote:
> On Sat, Mar 18, 2017 at 9:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Magnus Hagander <magnus@hagander.net> writes:
> > > createuser, dropuser - definitely pollutes the namespace, people do
> > > sometimes try them for the wrong thing. Unlike the db ones they do add
> > > value though -- I don't think we have a psql way of in a single command
> > > doing what --pwprompt on createuser does, do we? But given that we are in
> > > the process of breaking a lot of other scripts for 10, perhaps we should
> > > rename it to pg_createuser?
> >
> > I'm not particularly on board with arguments like "we already broke a ton
> > of stuff so let's break some more".  Eventually you've managed to create
> > a daunting barrier to upgrading at all.
>
> The argument is more that if we are going to break it, now is a good time
> to do it because we are already forcing people to review their scripts.
>
> If we decide not to break it at all that's one thing. But if we *are* going
> to break it, it's better to do it in 10 than in 11.

I tend to agree with this, but...

> > I think a more reasonable way to proceed is to install symlinks
> > pg_createuser -> createuser (or the other direction), mark the older names
> > as deprecated, and announce that we'll remove the old names a few releases
> > from now.  That gives people time to adjust.
>
> I'd suggest doing it in the other direction, but yeah, that seems like a
> softer way to handle it. As long as we clearly document them as such.
> Perhaps we should even have them output a little "hey you should be using
> pg_xyz" if they are used by the wrong name, but I wonder if that might
> break things.

I don't really agree with these half-measures.  They don't do anything
for the namespace pollution problem and even if we do spit out a warning
or something (which may break things *anyway*, as you suggest), it's
just delaying when things break to some future point where we'll have to
argue, again, about removing the symlinks, and then it'll be in PG11 or
PG12 or some other release which isn't breaking as much.

> > Maybe we should handle createdb likewise, rather than just kicking it to
> > the curb.  I know I use it quite often; it's less typing than psql -c
> > 'create database ...' postgres, and still would be with a pg_ prefix.
>
> As long as they have a pg_ prefix, I don't see much harm in them being
> there, they're tiny. It's not like they're a huge maintenance burden.

I agree with keeping them, with a pg_ prefix.

Thanks!

Stephen

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Tom Lane
Date:
Stephen Frost <sfrost@snowman.net> writes:
> * Magnus Hagander (magnus@hagander.net) wrote:
>> On Sat, Mar 18, 2017 at 9:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> I think a more reasonable way to proceed is to install symlinks
>>> pg_createuser -> createuser (or the other direction), mark the older names
>>> as deprecated, and announce that we'll remove the old names a few releases
>>> from now.  That gives people time to adjust.

>> I'd suggest doing it in the other direction, but yeah, that seems like a
>> softer way to handle it. As long as we clearly document them as such.
>> Perhaps we should even have them output a little "hey you should be using
>> pg_xyz" if they are used by the wrong name, but I wonder if that might
>> break things.

> I don't really agree with these half-measures.

It's not a "half measure", it's providing a sane upgrade path.

If we had not had the deprecation notice in place for createlang, I do
not think Magnus' proposal to remove it would have been accepted so
easily --- or, probably, at all.  There is no such notice in place
for createdb or createuser, so arguing by analogy to that proposal
falls flat.
        regards, tom lane



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Stephen Frost
Date:
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Magnus Hagander (magnus@hagander.net) wrote:
> >> On Sat, Mar 18, 2017 at 9:23 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >>> I think a more reasonable way to proceed is to install symlinks
> >>> pg_createuser -> createuser (or the other direction), mark the older names
> >>> as deprecated, and announce that we'll remove the old names a few releases
> >>> from now.  That gives people time to adjust.
>
> >> I'd suggest doing it in the other direction, but yeah, that seems like a
> >> softer way to handle it. As long as we clearly document them as such.
> >> Perhaps we should even have them output a little "hey you should be using
> >> pg_xyz" if they are used by the wrong name, but I wonder if that might
> >> break things.
>
> > I don't really agree with these half-measures.
>
> It's not a "half measure", it's providing a sane upgrade path.

I really don't see it that way.  We're talking about existing scripts
which will break if the binary is renamed.  That means that, today,
they're using createlang/createdb/createuser.  The options for a user
using such a script with the proposed approach, when PG10 comes out
are:

1) don't change the script, because the old names work
2) adjust the script to work with both X and pg_X values

If option #1 is used, then it's just going to break whenever we do
remove those symlinks, meaning all we're doing is delaying things
without very much point.

If option #2 is used, which I believe is what we would want people to
do, then the symlinks are useless.

I anticipate an argument along the lines of "but we're giving them time
to make the change" but I don't see that as really holding all that much
weight either- we maintain back-branch releases for years to give users
time to adjust to changes that have been made in the latest releases.
The people who are going to rush out to deploy PG10 as soon as it's
released with the expectation that nothing is going to break aren't very
likely going to be the group who will be reading the release notes
carefully and making note of the changes we made that impact their
environment and scripts.  In other words, they're likely to stick with
option #1 above, and complain loudly whenever we do remove the symlinks,
if we ever manage to.

> If we had not had the deprecation notice in place for createlang, I do
> not think Magnus' proposal to remove it would have been accepted so
> easily --- or, probably, at all.  There is no such notice in place
> for createdb or createuser, so arguing by analogy to that proposal
> falls flat.

I wasn't.

Thanks!

Stephen

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Tom Lane
Date:
Stephen Frost <sfrost@snowman.net> writes:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> It's not a "half measure", it's providing a sane upgrade path.

> I really don't see it that way.  We're talking about existing scripts
> which will break if the binary is renamed.  That means that, today,
> they're using createlang/createdb/createuser.  The options for a user
> using such a script with the proposed approach, when PG10 comes out
> are:

> 1) don't change the script, because the old names work
> 2) adjust the script to work with both X and pg_X values

You're neglecting what I think most people would want to do:

3) Wait till they no longer care about supporting versions that have
only the old names, then change their scripts to use pg_X.

> I anticipate an argument along the lines of "but we're giving them time
> to make the change" but I don't see that as really holding all that much
> weight either- we maintain back-branch releases for years to give users
> time to adjust to changes that have been made in the latest releases.

Yes, and that also means that other tooling has to be prepared to work
with multiple releases.  You're proposing to make that harder, and I
do not think there's sufficient reason.

This line of argument means that we probably couldn't remove the old
names until 9.6 is out of support, but so what?  We had the deprecation
notice for createlang in place since 9.1, and I think that that's about
the right timeline for this sort of changeover.  We should not
cavalierly break peoples' scripts for what's fundamentally just a
cosmetic improvement.

(Or in other words, we've been getting along fine with these script names
for circa twenty years, so what's the rush to change them RIGHT NOW?)
        regards, tom lane



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Stephen Frost
Date:
* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Tom Lane (tgl@sss.pgh.pa.us) wrote:
> >> It's not a "half measure", it's providing a sane upgrade path.
>
> > I really don't see it that way.  We're talking about existing scripts
> > which will break if the binary is renamed.  That means that, today,
> > they're using createlang/createdb/createuser.  The options for a user
> > using such a script with the proposed approach, when PG10 comes out
> > are:
>
> > 1) don't change the script, because the old names work
> > 2) adjust the script to work with both X and pg_X values
>
> You're neglecting what I think most people would want to do:
>
> 3) Wait till they no longer care about supporting versions that have
> only the old names, then change their scripts to use pg_X.

Perhaps it's a bit stark, but my experience has been that people tend to
fall into one of the two camps I outlined above.

If we take your approach to its logical conclustion then we should be
planning to maintain all user-facing deprecated features for as long as
there is a version where it exists in a non-deprecated fashion, in other
words for 5 years, if we assume that script authors only care about
supporting as far back as we do, which I'm not entirely sure is a great
assumption to begin with, but I'd say it's at least a minimum.

To that I say: heck no.  I understand that there has been grief due to
the various user-facing changes we've made recently and I completely
understand those complaints because it means that developers have to
deal with version-specific differences, particularly when it comes to
monitoring and now with the pg_xlog -> pg_wal changes, more with backup
too, but trying to deal with multiple sets of interfaces over a long
period of time is going to be incredibly painful and very confusing for
users, especially new users.

In some cases, we would end up with 5 interfaces, all having to be
maintained across all of the back-branches, because we changed something
in every release, deprecating each prior version as we went.

No, I don't feel that this kind of backwards-compatibility is really
something we want, as a project.  We, for very good reason, support 5
years worth of back-branches to allow users the opportunity and time to
migrate at a reasoned pace, possibly even skipping versions as they go
if they don't wish to make changes every year.

I'll admit that this is a bit of a straw-man and goes beyond what you're
suggesting here, specifically, but that just begs the question of "where
do we draw the line", if we are going to draw one.  Are scripts which
use command-line utilities more of an issue to break than those which
query pg_stat_activity?

> > I anticipate an argument along the lines of "but we're giving them time
> > to make the change" but I don't see that as really holding all that much
> > weight either- we maintain back-branch releases for years to give users
> > time to adjust to changes that have been made in the latest releases.
>
> Yes, and that also means that other tooling has to be prepared to work
> with multiple releases.  You're proposing to make that harder, and I
> do not think there's sufficient reason.

I do think, in the general sense, that tools based on PG should be
prepared to deal with differences between major versions, because we
have quite a few of those.  I'm hard-pressed to come up with a major PG
tool which *doesn't* have mechanisms for dealing with differences
between major versions already.

There's certainly no shortage of PG-based applications which also have
code for dealing with different PG major versions either.

> This line of argument means that we probably couldn't remove the old
> names until 9.6 is out of support, but so what?  We had the deprecation
> notice for createlang in place since 9.1, and I think that that's about
> the right timeline for this sort of changeover.  We should not
> cavalierly break peoples' scripts for what's fundamentally just a
> cosmetic improvement.

Per my question above- when is 5 years of deprecated-but-working support
for a feature the right timeline and when isn't it?  My feeling is that
most of the changes we make in this regard are accepted under the "well,
it's a major version change, so you'll need to adjust" but this change
isn't viewed in that same light and I'm not really sure that I see a
good reason for that.

I'll admit that part of that is likely because I don't think I have
*ever* used createdb or createlang or createuser (excepting whatever
regression tests run them), but even so, figuring out some idea where we
draw this line between things that need to be deprecated for 5 years and
things that don't would be useful, though likely a source of ongoing
discussion too.

> (Or in other words, we've been getting along fine with these script names
> for circa twenty years, so what's the rush to change them RIGHT NOW?)

To be clear, I'm not in any particular rush to change them 'RIGHT NOW'.
I tend to agree with Magnus that we're doing a lot of other things in
PG10 and that makes it a bit of a natural point, but I don't hold that
position terribly strongly.  On the other hand, I do not relish the idea
of providing backwards-compatibility for every user-facing change we do
for 5 years and that's where I feel this approach is encouraging us to
go.

Thanks!

Stephen

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Andreas Karlsson
Date:
On 03/19/2017 07:35 PM, Stephen Frost wrote:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> Stephen Frost <sfrost@snowman.net> writes:
>> (Or in other words, we've been getting along fine with these script names
>> for circa twenty years, so what's the rush to change them RIGHT NOW?)
>
> To be clear, I'm not in any particular rush to change them 'RIGHT NOW'.
> I tend to agree with Magnus that we're doing a lot of other things in
> PG10 and that makes it a bit of a natural point, but I don't hold that
> position terribly strongly.  On the other hand, I do not relish the idea
> of providing backwards-compatibility for every user-facing change we do
> for 5 years and that's where I feel this approach is encouraging us to
> go.

I only think that argument is only applicable where the changes are 
closely related, e.g. renaming pg_clog, pg_xlog and pg_log in the same 
release. I do not see any strong connection between createuser and pg_xlog.

As for if we should have backwards compatibility for the old names I am 
leaning weakly for providing it in the case of createuser. I can see end 
users being pissed off that the createuser command is suddenly gone 
without any warning when they upgrade. On the flip side I have no idea 
how much work it would be to maintain those legacy names.

Andreas




Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Stephen Frost
Date:
Greetings,

* Andreas Karlsson (andreas@proxel.se) wrote:
> On 03/19/2017 07:35 PM, Stephen Frost wrote:
> >* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> >>Stephen Frost <sfrost@snowman.net> writes:
> >>(Or in other words, we've been getting along fine with these script names
> >>for circa twenty years, so what's the rush to change them RIGHT NOW?)
> >
> >To be clear, I'm not in any particular rush to change them 'RIGHT NOW'.
> >I tend to agree with Magnus that we're doing a lot of other things in
> >PG10 and that makes it a bit of a natural point, but I don't hold that
> >position terribly strongly.  On the other hand, I do not relish the idea
> >of providing backwards-compatibility for every user-facing change we do
> >for 5 years and that's where I feel this approach is encouraging us to
> >go.
>
> I only think that argument is only applicable where the changes are
> closely related, e.g. renaming pg_clog, pg_xlog and pg_log in the
> same release. I do not see any strong connection between createuser
> and pg_xlog.

Given all that we're doing, it strikes me as pretty likely that people
will realize that PG10 does more and will therefore pay more attention,
in general, to what we tell them in the release notes about changes.

> As for if we should have backwards compatibility for the old names I
> am leaning weakly for providing it in the case of createuser. I can
> see end users being pissed off that the createuser command is
> suddenly gone without any warning when they upgrade. On the flip
> side I have no idea how much work it would be to maintain those
> legacy names.

This particular case might not be that much of a maintenance burden, but
to your example above, if they're going to be annoyed about it going
missing in PG10, it seems likely that they're going to be annoyed when
the symlink goes away in PG15 (or whatever) too.

Either way, we'll obviously document what we're doing in the release
notes, so the whole "without any warning" doesn't really fly, in my
view, either.

Thanks!

Stephen

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Tom Lane
Date:
Andreas Karlsson <andreas@proxel.se> writes:
> As for if we should have backwards compatibility for the old names I am 
> leaning weakly for providing it in the case of createuser. I can see end 
> users being pissed off that the createuser command is suddenly gone 
> without any warning when they upgrade. On the flip side I have no idea 
> how much work it would be to maintain those legacy names.

It seems reasonably trivial to me as far as the code goes --- just
create a symlink during installation.  (On Windows I suppose we'd have
to make a second physical copy, but these files are not so large that
that seems unacceptable.)  It might be more work on the documentation
side, depending on whether you wanted two copies of the man page or
not.  I see that postmaster's man page pretty much just points to
postgres, so it seems like we could do that for these scripts too.
        regards, tom lane



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Andres Freund
Date:
On 2017-03-18 16:23:17 -0400, Tom Lane wrote:
> Magnus Hagander <magnus@hagander.net> writes:
> > createuser, dropuser - definitely pollutes the namespace, people do
> > sometimes try them for the wrong thing. Unlike the db ones they do add
> > value though -- I don't think we have a psql way of in a single command
> > doing what --pwprompt on createuser does, do we? But given that we are in
> > the process of breaking a lot of other scripts for 10, perhaps we should
> > rename it to pg_createuser?
> 
> I'm not particularly on board with arguments like "we already broke a ton
> of stuff so let's break some more".  Eventually you've managed to create
> a daunting barrier to upgrading at all.

+1


> I think a more reasonable way to proceed is to install symlinks
> pg_createuser -> createuser (or the other direction), mark the older names
> as deprecated, and announce that we'll remove the old names a few releases
> from now.  That gives people time to adjust.

+1


> Maybe we should handle createdb likewise, rather than just kicking it to
> the curb.  I know I use it quite often; it's less typing than psql -c
> 'create database ...' postgres, and still would be with a pg_ prefix.

I think we should add pg_createdb, and do a normal deprecation cycle for
removing createdb.


I wish 'pg' wasn't an already used binary name.  It'd be much nicer if
we had a '/usr/bin/pg' wrapper binary in the git style, that we could easily
expand over time, without hitting new conflicts.  I'd even consider a
'/usr/bin/pgsql' that has commands for all our binaries a considerable
improvement in the long term.


Greetings,

Andres Freund



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Tom Lane
Date:
Stephen Frost <sfrost@snowman.net> writes:
> If we take your approach to its logical conclustion then we should be
> planning to maintain all user-facing deprecated features for as long as
> there is a version where it exists in a non-deprecated fashion, in other
> words for 5 years, if we assume that script authors only care about
> supporting as far back as we do, which I'm not entirely sure is a great
> assumption to begin with, but I'd say it's at least a minimum.

Well, that is definitely a straw man (as you admit later).  I don't want
to go there either.  Where to draw the line has to be a case-by-case
decision, though.  In this case the amount of effort involved in providing
backwards compatibility is pretty minimal, and the project has failed to
provide any warning to users that this is something we might choose to
break in future, so it seems to me that we ought to provide compatibility
for awhile.  If somebody bitches that we broke their script in v15, we'll
be in a much better position if we can point to five years' worth of
deprecation notices than if we just have to say "tough, we felt like
breaking your script so we did it".

> I'll admit that part of that is likely because I don't think I have
> *ever* used createdb or createlang or createuser (excepting whatever
> regression tests run them),

Well, personally I use createdb pretty often.  I basically never use
the other two, but I don't fool myself that I'm a representative user.
createuser in particular seems like it has pretty high value-added
if you create new roles often and need to give them passwords.  Yeah,
you can do it in psql, but it's not as convenient, especially if you
want to do proper quoting.
        regards, tom lane



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Tom Lane
Date:
Andres Freund <andres@anarazel.de> writes:
> I wish 'pg' wasn't an already used binary name.  It'd be much nicer if
> we had a '/usr/bin/pg' wrapper binary in the git style, that we could easily
> expand over time, without hitting new conflicts.  I'd even consider a
> '/usr/bin/pgsql' that has commands for all our binaries a considerable
> improvement in the long term.

Meh.  I'm not seeing where "pg command args" is that much superior to
"pg_command args".  Yeah, if we'd done it like that from the beginning
it'd be fine; but people are used to the latter style and I don't feel
a need to make them change over.
        regards, tom lane



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
"Joshua D. Drake"
Date:
On 03/19/2017 01:01 PM, Tom Lane wrote:
> Andreas Karlsson <andreas@proxel.se> writes:
>> As for if we should have backwards compatibility for the old names I am
>> leaning weakly for providing it in the case of createuser. I can see end
>> users being pissed off that the createuser command is suddenly gone
>> without any warning when they upgrade. On the flip side I have no idea
>> how much work it would be to maintain those legacy names.
>
> It seems reasonably trivial to me as far as the code goes --- just
> create a symlink during installation.  (On Windows I suppose we'd have
> to make a second physical copy, but these files are not so large that
> that seems unacceptable.)

Windows supports symlinks:

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365680(v=vs.85).aspx

JD


-- 
Command Prompt, Inc.                  http://the.postgres.company/                        +1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.
Unless otherwise stated, opinions are my own.



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Peter Eisentraut
Date:
On 3/18/17 16:12, Magnus Hagander wrote:
> But given that we are in the process of breaking a lot of other scripts
> for 10,

I don't think the xlog renaming really affects this area.

> perhaps we should rename it to pg_createuser?

I'm not keen on doing that now, but if we were to do it at some point, I
would do it for all programs not already starting with "p" or containing
"pg".

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



[HACKERS] Our feature change policy

From
Bruce Momjian
Date:
On Sun, Mar 19, 2017 at 04:15:09PM -0400, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > If we take your approach to its logical conclusion then we should be
> > planning to maintain all user-facing deprecated features for as long as
> > there is a version where it exists in a non-deprecated fashion, in other
> > words for 5 years, if we assume that script authors only care about
> > supporting as far back as we do, which I'm not entirely sure is a great
> > assumption to begin with, but I'd say it's at least a minimum.
> 
> Well, that is definitely a straw man (as you admit later).  I don't want
> to go there either.  Where to draw the line has to be a case-by-case
> decision, though.  In this case the amount of effort involved in providing
> backwards compatibility is pretty minimal, and the project has failed to
> provide any warning to users that this is something we might choose to
> break in future, so it seems to me that we ought to provide compatibility
> for awhile.  If somebody bitches that we broke their script in v15, we'll
> be in a much better position if we can point to five years' worth of
> deprecation notices than if we just have to say "tough, we felt like
> breaking your script so we did it".

[ I have started a new thread for this topic but kept the above quote.]

We keep re-litigating changes, either with pg_xlog, binaries, or
pg_stat_activity, and at some point we need to settle on a
policy.  The usual "change" options are:

1.  make the change now and mention it in the release notes
2.  #1, but also provide backward compatibility for 5+ years
3.  mark the feature as deprecated and remove/change it in 5+ years
4.  #3, but issue a warning for deprecated usage

What causes us to choose different outcomes?  #1 is usually for
rarely-used features or features that are dangerous.  #2 is usually for
popular features where backward compatibility will not confuse users,
e.g. this is not possible for renaming pg_stat_activity columns because
SELECT * FROM pg_stat_activity will show the old and new columns.  #3 is
for rarely-used features or features we don't want --- sometimes we go
long over five years and still can't remove something because we don't
have a fully-featured replacement, e.g. contrib/xml2.  #4 is for changes
that could affect data validity or are hard to detect, e.g. the
standard-conforming string change.

Is this accurate?  If so we can write this down and avoid the +1/-1
re-litigating every time we want to change something, or at least have a
more focused conversation on the issues related to the change.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +



Re: [HACKERS] Our feature change policy

From
Stephen Frost
Date:
Bruce,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Sun, Mar 19, 2017 at 04:15:09PM -0400, Tom Lane wrote:
> > Stephen Frost <sfrost@snowman.net> writes:
> > > If we take your approach to its logical conclusion then we should be
> > > planning to maintain all user-facing deprecated features for as long as
> > > there is a version where it exists in a non-deprecated fashion, in other
> > > words for 5 years, if we assume that script authors only care about
> > > supporting as far back as we do, which I'm not entirely sure is a great
> > > assumption to begin with, but I'd say it's at least a minimum.
> >
> > Well, that is definitely a straw man (as you admit later).  I don't want
> > to go there either.  Where to draw the line has to be a case-by-case
> > decision, though.  In this case the amount of effort involved in providing
> > backwards compatibility is pretty minimal, and the project has failed to
> > provide any warning to users that this is something we might choose to
> > break in future, so it seems to me that we ought to provide compatibility
> > for awhile.  If somebody bitches that we broke their script in v15, we'll
> > be in a much better position if we can point to five years' worth of
> > deprecation notices than if we just have to say "tough, we felt like
> > breaking your script so we did it".
>
> [ I have started a new thread for this topic but kept the above quote.]
>
> We keep re-litigating changes, either with pg_xlog, binaries, or
> pg_stat_activity, and at some point we need to settle on a
> policy.  The usual "change" options are:

Agreed.

> 1.  make the change now and mention it in the release notes
> 2.  #1, but also provide backward compatibility for 5+ years
> 3.  mark the feature as deprecated and remove/change it in 5+ years
> 4.  #3, but issue a warning for deprecated usage
>
> What causes us to choose different outcomes?  #1 is usually for
> rarely-used features or features that are dangerous.  #2 is usually for
> popular features where backward compatibility will not confuse users,
> e.g. this is not possible for renaming pg_stat_activity columns because
> SELECT * FROM pg_stat_activity will show the old and new columns.  #3 is
> for rarely-used features or features we don't want --- sometimes we go
> long over five years and still can't remove something because we don't
> have a fully-featured replacement, e.g. contrib/xml2.  #4 is for changes
> that could affect data validity or are hard to detect, e.g. the
> standard-conforming string change.
>
> Is this accurate?  If so we can write this down and avoid the +1/-1
> re-litigating every time we want to change something, or at least have a
> more focused conversation on the issues related to the change.

I don't generally feel like #1 is so rarely used (nor do I think it
should be rare that we use it).  With regard to #2, if we're going to do
that, I'd really like to see us decide ahead of time on a point in time
when we will remove the backwards-compatibility, otherwise it seems to
live on forever.  For my 2c, #3 should be reserved for things we are
explicitly removing, not for things we're changing and we should do #4
whenever possible in those cases because we're going to be removing it.

Otherwise, #3 ends up being a case where we're holding up progress for
years because we have to announce that we're going to deprecate
something and then wait before we actually make whatever the change is.

Thanks!

Stephen

Re: [HACKERS] Our feature change policy

From
Tom Lane
Date:
Stephen Frost <sfrost@snowman.net> writes:
> * Bruce Momjian (bruce@momjian.us) wrote:
>> 1.  make the change now and mention it in the release notes
>> 2.  #1, but also provide backward compatibility for 5+ years
>> 3.  mark the feature as deprecated and remove/change it in 5+ years
>> 4.  #3, but issue a warning for deprecated usage

> I don't generally feel like #1 is so rarely used (nor do I think it
> should be rare that we use it).  With regard to #2, if we're going to do
> that, I'd really like to see us decide ahead of time on a point in time
> when we will remove the backwards-compatibility, otherwise it seems to
> live on forever.  For my 2c, #3 should be reserved for things we are
> explicitly removing, not for things we're changing and we should do #4
> whenever possible in those cases because we're going to be removing it.

> Otherwise, #3 ends up being a case where we're holding up progress for
> years because we have to announce that we're going to deprecate
> something and then wait before we actually make whatever the change is.

Well, to what extent are we "holding up progress" in this particular
case?  There is no other development work that's stalled by not renaming
these binaries.  I think there should be some explicit accounting for
the impact of delay if we're going to try to formalize these decisions
better.
        regards, tom lane



Re: [HACKERS] Our feature change policy

From
Bruce Momjian
Date:
On Mon, Mar 20, 2017 at 11:40:34AM -0400, Tom Lane wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Bruce Momjian (bruce@momjian.us) wrote:
> >> 1.  make the change now and mention it in the release notes
> >> 2.  #1, but also provide backward compatibility for 5+ years
> >> 3.  mark the feature as deprecated and remove/change it in 5+ years
> >> 4.  #3, but issue a warning for deprecated usage
> 
> > I don't generally feel like #1 is so rarely used (nor do I think it
> > should be rare that we use it).  With regard to #2, if we're going to do
> > that, I'd really like to see us decide ahead of time on a point in time
> > when we will remove the backwards-compatibility, otherwise it seems to
> > live on forever.  For my 2c, #3 should be reserved for things we are
> > explicitly removing, not for things we're changing and we should do #4
> > whenever possible in those cases because we're going to be removing it.
> 
> > Otherwise, #3 ends up being a case where we're holding up progress for
> > years because we have to announce that we're going to deprecate
> > something and then wait before we actually make whatever the change is.
> 
> Well, to what extent are we "holding up progress" in this particular
> case?  There is no other development work that's stalled by not renaming
> these binaries.  I think there should be some explicit accounting for
> the impact of delay if we're going to try to formalize these decisions
> better.

I mentioned the problem of "confuse users" for #2.  What delay are you
talking about?  #3 and #4?

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +



Re: [HACKERS] Our feature change policy

From
Stephen Frost
Date:
Tom,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Bruce Momjian (bruce@momjian.us) wrote:
> >> 1.  make the change now and mention it in the release notes
> >> 2.  #1, but also provide backward compatibility for 5+ years
> >> 3.  mark the feature as deprecated and remove/change it in 5+ years
> >> 4.  #3, but issue a warning for deprecated usage
>
> > I don't generally feel like #1 is so rarely used (nor do I think it
> > should be rare that we use it).  With regard to #2, if we're going to do
> > that, I'd really like to see us decide ahead of time on a point in time
> > when we will remove the backwards-compatibility, otherwise it seems to
> > live on forever.  For my 2c, #3 should be reserved for things we are
> > explicitly removing, not for things we're changing and we should do #4
> > whenever possible in those cases because we're going to be removing it.
>
> > Otherwise, #3 ends up being a case where we're holding up progress for
> > years because we have to announce that we're going to deprecate
> > something and then wait before we actually make whatever the change is.
>
> Well, to what extent are we "holding up progress" in this particular
> case?  There is no other development work that's stalled by not renaming
> these binaries.  I think there should be some explicit accounting for
> the impact of delay if we're going to try to formalize these decisions
> better.

The rename of the binaries is case #2 above, at least as I was thinking
of it.  I don't believe we are holding up progress in that case.

I was imagining the changes to pg_stat_activity as possibly falling
under #3/change, meaning that we'd wait for 5 years before actually
renaming those columns, which is part of why I was objecting to that
idea.

If we're able to make the change and provide backwards compatibility
then I think the main question is if it's worth it to do so and, if so,
when are we going to remove that backwards compatibility.  If we don't
expect to ever be able to remove it, then that's an issue.

Thanks!

Stephen

Re: [HACKERS] Our feature change policy

From
Bruce Momjian
Date:
On Mon, Mar 20, 2017 at 11:57:13AM -0400, Stephen Frost wrote:
> Tom,
> 
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
> > Stephen Frost <sfrost@snowman.net> writes:
> > > * Bruce Momjian (bruce@momjian.us) wrote:
> > >> 1.  make the change now and mention it in the release notes
> > >> 2.  #1, but also provide backward compatibility for 5+ years
> > >> 3.  mark the feature as deprecated and remove/change it in 5+ years
> > >> 4.  #3, but issue a warning for deprecated usage
...
> > Well, to what extent are we "holding up progress" in this particular
> > case?  There is no other development work that's stalled by not renaming
> > these binaries.  I think there should be some explicit accounting for
> > the impact of delay if we're going to try to formalize these decisions
> > better.
> 
> The rename of the binaries is case #2 above, at least as I was thinking
> of it.  I don't believe we are holding up progress in that case.

Agreed.

> I was imagining the changes to pg_stat_activity as possibly falling
> under #3/change, meaning that we'd wait for 5 years before actually
> renaming those columns, which is part of why I was objecting to that
> idea.

Yes, that is a good point.  Removing contrib/tsearch2 or contrib/xml2 is
really not holding up progress on anything, but not delaying the
addition of wait event reporting to pg_stat_activity is certainly
holding up progress.  #3 and #4 would need to be weighted depending on
whether choosing them would delay progress, e.g. it did delay progress
on standard-conforming strings, but the delay was determined to be
reasonable.

> If we're able to make the change and provide backwards compatibility
> then I think the main question is if it's worth it to do so and, if so,
> when are we going to remove that backwards compatibility.  If we don't
> expect to ever be able to remove it, then that's an issue.

Agreed.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +



Re: [HACKERS] Our feature change policy

From
"David G. Johnston"
Date:
On Mon, Mar 20, 2017 at 8:57 AM, Stephen Frost <sfrost@snowman.net> wrote:
Tom,

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > * Bruce Momjian (bruce@momjian.us) wrote:
> >> 1.  make the change now and mention it in the release notes
> >> 2.  #1, but also provide backward compatibility for 5+ years
> >> 3.  mark the feature as deprecated and remove/change it in 5+ years
> >> 4.  #3, but issue a warning for deprecated usage
>
 
I was imagining the changes to pg_stat_activity as possibly falling
under #3/change, meaning that we'd wait for 5 years before actually
renaming those columns, which is part of why I was objecting to that
idea.

Which ends up boiling down to:

A. Make a change and document it in the release notes

B. If applicable, and desired, provide a 5 year backward​ compatibility deprecation period (i.e., 3 =>[implies] 2). Generally 2 => 3 but the deprecation period is undefined.

C. Optionally, if deprecating, provide explicit warnings when the deprecated feature is used

Guidelines for when to desire the 5 year period would be helpful.  And also acknowledge that we may wish to choose a shorter period of time, or institute immediate removal, at our discretion.

Nothing says we cannot go longer than 5 years but given our support policy I would say that we'd rarely desired to do so intentionally - the burden of proof falling to those who would want to keep something longer.

David J.

Re: [HACKERS] Our feature change policy

From
"David G. Johnston"
Date:
On Mon, Mar 20, 2017 at 9:18 AM, Bruce Momjian <bruce@momjian.us> wrote:
.  #3 and #4 would need to be weighted depending on
whether choosing them would delay progress, e.g. it did delay progress
on standard-conforming strings, but the delay was determined to be
reasonable.

w.r.t. standard-conforming strings to this day we are in indefinite deprecation of the backward compatibility mechanics.  We simply made the choice of whether to use the backward compatibility mode explicit when we changed the GUC default.  For features with that possibility adding an "D. Optionally, when applicable, maintain current behavior during release and later switch the default behavior to the new mode after N years."  Odds are if we are considering instituting item D we'd be content with discussing the specifics of the case and not rely on any kind of rule or guideline to define N.  As evidenced defaults and deprecation are orthogonal concerns.

David J.

Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Robert Haas
Date:
On Sat, Mar 18, 2017 at 4:12 PM, Magnus Hagander <magnus@hagander.net> wrote:
> createdb, dropdb - also not clear they're about postgres, more likely to be
> used by mistake but not that bad. That said, do they add any *value* beyond
> what you can do with psql -c "CREATE DATABASE"? I don't really see one, so
> I'd suggest dropping these too.

That would annoy me, because I use these constantly.  I also think
that they solve a problem for users, which is this:

[rhaas ~]$ psql
psql: FATAL:  database "rhaas" does not exist
[rhaas ~]$ psql -c 'create database rhaas;'
psql: FATAL:  database "rhaas" does not exist
[rhaas ~]$ gosh, i know i need to connect to a database in order to
create the database to which psql tries to connect by default, so
there must be an existing database with some name, but what exactly is
that name, anyway?
-bash: gosh,: command not found

There was an occasion when this exact problem almost caused me to give
up on using PostgreSQL.  Everybody here presumably knows that
template1 and postgres are the magic words you can add to the end of
that command line to make it work, but that is NOT self-evident to
newcomers.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: [HACKERS] Removing binaries (was: createlang/droplang deprecated)

From
Jan de Visser
Date:
On Monday, March 20, 2017 3:30:49 PM EDT Robert Haas wrote:
> On Sat, Mar 18, 2017 at 4:12 PM, Magnus Hagander <magnus@hagander.net> 
wrote:
> > createdb, dropdb - also not clear they're about postgres, more likely to
> > be
> > used by mistake but not that bad. That said, do they add any *value*
> > beyond
> > what you can do with psql -c "CREATE DATABASE"? I don't really see one, so
> > I'd suggest dropping these too.
> 
> That would annoy me, because I use these constantly.  I also think
> that they solve a problem for users, which is this:
> 
> [rhaas ~]$ psql
> psql: FATAL:  database "rhaas" does not exist
> [rhaas ~]$ psql -c 'create database rhaas;'
> psql: FATAL:  database "rhaas" does not exist
> [rhaas ~]$ gosh, i know i need to connect to a database in order to
> create the database to which psql tries to connect by default, so
> there must be an existing database with some name, but what exactly is
> that name, anyway?
> -bash: gosh,: command not found
> 
> There was an occasion when this exact problem almost caused me to give
> up on using PostgreSQL.  Everybody here presumably knows that
> template1 and postgres are the magic words you can add to the end of
> that command line to make it work, but that is NOT self-evident to
> newcomers.

Same here. I worked on a system with a shrink-wrap installer which installed 
pgsql as well and initialized it for use by the system user of our software. 
If a tester or sales engineer wanted to play with the DB, it would be about 30 
minutes before they would end up at my desk, in tears.



Re: [HACKERS] Removing binaries

From
David Steele
Date:
On 3/20/17 3:40 PM, Jan de Visser wrote:
> On Monday, March 20, 2017 3:30:49 PM EDT Robert Haas wrote:
>> On Sat, Mar 18, 2017 at 4:12 PM, Magnus Hagander <magnus@hagander.net>
> wrote:
>>> createdb, dropdb - also not clear they're about postgres, more likely to
>>> be
>>> used by mistake but not that bad. That said, do they add any *value*
>>> beyond
>>> what you can do with psql -c "CREATE DATABASE"? I don't really see one, so
>>> I'd suggest dropping these too.
>>
>> That would annoy me, because I use these constantly.  I also think
>> that they solve a problem for users, which is this:
>>
>> [rhaas ~]$ psql
>> psql: FATAL:  database "rhaas" does not exist
>> [rhaas ~]$ psql -c 'create database rhaas;'
>> psql: FATAL:  database "rhaas" does not exist
>> [rhaas ~]$ gosh, i know i need to connect to a database in order to
>> create the database to which psql tries to connect by default, so
>> there must be an existing database with some name, but what exactly is
>> that name, anyway?
>> -bash: gosh,: command not found
>>
>> There was an occasion when this exact problem almost caused me to give
>> up on using PostgreSQL.  Everybody here presumably knows that
>> template1 and postgres are the magic words you can add to the end of
>> that command line to make it work, but that is NOT self-evident to
>> newcomers.
>
> Same here. I worked on a system with a shrink-wrap installer which installed
> pgsql as well and initialized it for use by the system user of our software.
> If a tester or sales engineer wanted to play with the DB, it would be about 30
> minutes before they would end up at my desk, in tears.

How about adding a hint?

-- 
-David
david@pgmasters.net



Re: [HACKERS] Our feature change policy

From
Robert Haas
Date:
On Mon, Mar 20, 2017 at 10:35 AM, Bruce Momjian <bruce@momjian.us> wrote:
> We keep re-litigating changes, either with pg_xlog, binaries, or
> pg_stat_activity, and at some point we need to settle on a
> policy.  The usual "change" options are:
>
> 1.  make the change now and mention it in the release notes
> 2.  #1, but also provide backward compatibility for 5+ years
> 3.  mark the feature as deprecated and remove/change it in 5+ years
> 4.  #3, but issue a warning for deprecated usage
>
> What causes us to choose different outcomes?

Well, sometimes backward compatibility is more possible than at other
times.  With standard_confirming_strings, we made it a GUC, but that
doesn't work with renaming a column in pg_stat_activity.  Also,
there's a big difference in my mind between changes that affect DBAs
and changes that affect user queries.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



[HACKERS] Implementing delete in columnar store fdw

From
Hari Krishnan
Date:
Hello,
I want to implement delete functionality for a column store fdw in postgres. It is similar to file_fdw. I want to use
the“AddForeignUpdateTargets” function to implement this , but the junk filter shouldn’t be a column present in the
table. Is it possible to add a Expr/Var to the junk filter that is not present in the schema of the table , that i will
generateon the fly ?? 

If yes, how can i do that ?? Thanks in advance.

Regards,
Harikrishnan


Re: [HACKERS] Removing binaries

From
Robert Haas
Date:
On Mon, Mar 20, 2017 at 6:15 PM, David Steele <david@pgmasters.net> wrote:
> On 3/20/17 3:40 PM, Jan de Visser wrote:
>> On Monday, March 20, 2017 3:30:49 PM EDT Robert Haas wrote:
>>> That would annoy me, because I use these constantly.  I also think
>>> that they solve a problem for users, which is this:
>>>
>>> [rhaas ~]$ psql
>>> psql: FATAL:  database "rhaas" does not exist
>>> [rhaas ~]$ psql -c 'create database rhaas;'
>>> psql: FATAL:  database "rhaas" does not exist
>>> [rhaas ~]$ gosh, i know i need to connect to a database in order to
>>> create the database to which psql tries to connect by default, so
>>> there must be an existing database with some name, but what exactly is
>>> that name, anyway?
>>> -bash: gosh,: command not found
>>>
>>> There was an occasion when this exact problem almost caused me to give
>>> up on using PostgreSQL.  Everybody here presumably knows that
>>> template1 and postgres are the magic words you can add to the end of
>>> that command line to make it work, but that is NOT self-evident to
>>> newcomers.
>>
>> Same here. I worked on a system with a shrink-wrap installer which
>> installed
>> pgsql as well and initialized it for use by the system user of our
>> software.
>> If a tester or sales engineer wanted to play with the DB, it would be
>> about 30
>> minutes before they would end up at my desk, in tears.
>
> How about adding a hint?

I think it's tricky to do that, because the error happens in response
to the connection attempt and at that point you don't know that the
command the user plans to send is CREATE DATABASE.  If we could
somehow detect that the user is trying to CREATE DATABASE against a
nonexistent database and hint in that case, that would be *great*, but
I don't see a way to make it work.

Here's another idea: what if we always created the default database at
initdb time?  For example, if I initdb as rhaas, maybe it should
create an "rhaas" database for me, so that this works:

initdb
pg_ctl start
psql

I think a big part of the usability problem here comes from the fact
that the default database for connections is based on the username,
but the default databases that get created have fixed names (postgres,
template1).  So the default configuration is one where you can't
connect.  Why the heck do we do it that way?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: [HACKERS] Removing binaries

From
"David G. Johnston"
Date:
On Tue, Mar 21, 2017 at 5:12 AM, Robert Haas <robertmhaas@gmail.com> wrote:

Here's another idea: what if we always created the default database at
initdb time?  For example, if I initdb as rhaas, maybe it should
create an "rhaas" database for me, so that this works:

initdb
pg_ctl start
psql

I think a big part of the usability problem here comes from the fact
that the default database for connections is based on the username,
but the default databases that get created have fixed names (postgres,
template1).  So the default configuration is one where you can't
connect.  Why the heck do we do it that way?


​I'd be curious to estimate how many users that have difficulties learning how all this works actually run a manual initdb prior to beginning their experimentation.  I suspect the percentage is fairly low.

Doing away with "the default database for psql is one named after the user" seems like it would be more widely applicable.  I for one tend to name things after what they do, or are used for, and thus have never benefited from this particular design decision.

David J.

Re: [HACKERS] Removing binaries

From
David Steele
Date:
On 3/21/17 10:30 AM, David G. Johnston wrote:
> On Tue, Mar 21, 2017 at 5:12 AM, Robert Haas <robertmhaas@gmail.com
> <mailto:robertmhaas@gmail.com>>wrote:
>
>
>     Here's another idea: what if we always created the default database at
>     initdb time?  For example, if I initdb as rhaas, maybe it should
>     create an "rhaas" database for me, so that this works:
>
>     initdb
>     pg_ctl start
>     psql
>
>     I think a big part of the usability problem here comes from the fact
>     that the default database for connections is based on the username,
>     but the default databases that get created have fixed names (postgres,
>     template1).  So the default configuration is one where you can't
>     connect.  Why the heck do we do it that way?
>
>
> ​I'd be curious to estimate how many users that have difficulties
> learning how all this works actually run a manual initdb prior to
> beginning their experimentation.  I suspect the percentage is fairly low.
>
> Doing away with "the default database for psql is one named after the
> user" seems like it would be more widely applicable.  I for one tend to
> name things after what they do, or are used for, and thus have never
> benefited from this particular design decision.

I suppose it would be too big a change to have psql try the username and 
then fallback to postgres on failure?

-- 
-David
david@pgmasters.net



Re: [HACKERS] Removing binaries

From
Robert Haas
Date:
On Tue, Mar 21, 2017 at 10:43 AM, David Steele <david@pgmasters.net> wrote:
> I suppose it would be too big a change to have psql try the username and
> then fallback to postgres on failure?

It's not so much that the change would be too big as that the
resulting semantics would be confusing, at least IMHO.  Imagine:

1. Person A logs into the service account, runs psql, starts creating
stuff in postgres DB without realizing it.
2. Person B logs into the service account, runs psql -l, sees that the
usual DB name is not there, runs createdb.
3. Person A reruns psql and says "where'd all my stuff go?".

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Removing binaries

From
Peter Eisentraut
Date:
On 3/21/17 08:12, Robert Haas wrote:
> I think a big part of the usability problem here comes from the fact
> that the default database for connections is based on the username,
> but the default databases that get created have fixed names (postgres,
> template1).  So the default configuration is one where you can't
> connect.  Why the heck do we do it that way?

Historical, probably.  We could ponder changing the way the default
database is determined, but I don't want to imagine the breakage coming
out of that.

Some time ago I changed my environment to PGDATABASE=postgres by
default, and my experience has been much smoother.  Of course, if we did
that, then people will store a bunch of junk in their postgres database.But maybe that's not a problem.  Maybe just
toleratethe use of the
 
postgres database by default, like using the public schema.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services



Re: Removing binaries

From
Robert Haas
Date:
On Tue, Mar 28, 2017 at 11:44 AM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 3/21/17 08:12, Robert Haas wrote:
>> I think a big part of the usability problem here comes from the fact
>> that the default database for connections is based on the username,
>> but the default databases that get created have fixed names (postgres,
>> template1).  So the default configuration is one where you can't
>> connect.  Why the heck do we do it that way?
>
> Historical, probably.  We could ponder changing the way the default
> database is determined, but I don't want to imagine the breakage coming
> out of that.

What do you think would break?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Removing binaries

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> On Tue, Mar 28, 2017 at 11:44 AM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
>> On 3/21/17 08:12, Robert Haas wrote:
>>> I think a big part of the usability problem here comes from the fact
>>> that the default database for connections is based on the username,
>>> but the default databases that get created have fixed names (postgres,
>>> template1).  So the default configuration is one where you can't
>>> connect.  Why the heck do we do it that way?

>> Historical, probably.  We could ponder changing the way the default
>> database is determined, but I don't want to imagine the breakage coming
>> out of that.

> What do you think would break?

Any configuration depending on the existing default?

The existing behavior here dates from before we had schemas, so that
if users wanted to have private objects they *had* to use separate
databases.  Nowadays a schema-per-user within one database makes a lot
more sense for many environments, and we even have the default value
for search_path set up to make that as painless as possible.  Still,
it's not a solution for everybody, particularly not installations
that want to keep their users well separated.

Perhaps we could satisfy novices by changing the out-of-the-box
behavior, but provide some way to select the old behavior for
installations that are really depending on it.
        regards, tom lane



Re: Removing binaries

From
Robert Haas
Date:
On Tue, Mar 28, 2017 at 12:18 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Tue, Mar 28, 2017 at 11:44 AM, Peter Eisentraut
>> <peter.eisentraut@2ndquadrant.com> wrote:
>>> On 3/21/17 08:12, Robert Haas wrote:
>>>> I think a big part of the usability problem here comes from the fact
>>>> that the default database for connections is based on the username,
>>>> but the default databases that get created have fixed names (postgres,
>>>> template1).  So the default configuration is one where you can't
>>>> connect.  Why the heck do we do it that way?
>
>>> Historical, probably.  We could ponder changing the way the default
>>> database is determined, but I don't want to imagine the breakage coming
>>> out of that.
>
>> What do you think would break?
>
> Any configuration depending on the existing default?
>
> The existing behavior here dates from before we had schemas, so that
> if users wanted to have private objects they *had* to use separate
> databases.  Nowadays a schema-per-user within one database makes a lot
> more sense for many environments, and we even have the default value
> for search_path set up to make that as painless as possible.  Still,
> it's not a solution for everybody, particularly not installations
> that want to keep their users well separated.
>
> Perhaps we could satisfy novices by changing the out-of-the-box
> behavior, but provide some way to select the old behavior for
> installations that are really depending on it.

Hmm.  I guess that would mean that the same connection string would
mean something different depending on how you configured this
behavior, which does not sound like a good idea.  But why not go the
other way and just create the default database by default, either in
addition to or instead of the postgres database?  I mean, most people
probably do this:

initdb
pg_ctl start
createdb

If initdb created the database that you currently have to create as a
separate step by running 'createdb', I bet we'd eliminate a metric
buttload of new user confusion and harm almost nobody.  Anybody who
doesn't want that extra database can just drop it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Removing binaries

From
Tom Lane
Date:
Robert Haas <robertmhaas@gmail.com> writes:
> On Tue, Mar 28, 2017 at 12:18 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Perhaps we could satisfy novices by changing the out-of-the-box
>> behavior, but provide some way to select the old behavior for
>> installations that are really depending on it.

> Hmm.  I guess that would mean that the same connection string would
> mean something different depending on how you configured this
> behavior, which does not sound like a good idea.  But why not go the
> other way and just create the default database by default, either in
> addition to or instead of the postgres database?

What "default database"?  Are you suggesting that createuser should
forcibly create a similarly-named database?  Doesn't sound like a
great idea to me; it's presuming what seems like minority usage.

If we were to do anything about this, my sketch of a desirable solution
would be to invent a GUC named something like "default_database", which
would specify the DB to connect to when the incoming request packet
doesn't name one.  We could have it be "postgres" out of the box, but
allow the value "$user" to select the old behavior.

However ... a little bit of research shows that this behavior is wired
into libpq as well as the backend, and that therefore the backend's
code path that inserts the username as the default DB name is dead
code (at least when processing libpq-driven connection requests;
I don't know what JDBC and other client libraries do).  This means
that there isn't any really centralized way to fix it.  We could
delete that behavior from libpq easily enough, but then we'd have
a situation where the behavior varies depending on which version of
which client library you're using.

I'm afraid now that changing this would be far more pain than it
could possibly be worth.
        regards, tom lane



Re: Removing binaries

From
Magnus Hagander
Date:


On Tue, Mar 28, 2017 at 6:44 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Mar 28, 2017 at 12:18 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Tue, Mar 28, 2017 at 11:44 AM, Peter Eisentraut
>> <peter.eisentraut@2ndquadrant.com> wrote:
>>> On 3/21/17 08:12, Robert Haas wrote:
>>>> I think a big part of the usability problem here comes from the fact
>>>> that the default database for connections is based on the username,
>>>> but the default databases that get created have fixed names (postgres,
>>>> template1).  So the default configuration is one where you can't
>>>> connect.  Why the heck do we do it that way?
>
>>> Historical, probably.  We could ponder changing the way the default
>>> database is determined, but I don't want to imagine the breakage coming
>>> out of that.
>
>> What do you think would break?
>
> Any configuration depending on the existing default?
>
> The existing behavior here dates from before we had schemas, so that
> if users wanted to have private objects they *had* to use separate
> databases.  Nowadays a schema-per-user within one database makes a lot
> more sense for many environments, and we even have the default value
> for search_path set up to make that as painless as possible.  Still,
> it's not a solution for everybody, particularly not installations
> that want to keep their users well separated.
>
> Perhaps we could satisfy novices by changing the out-of-the-box
> behavior, but provide some way to select the old behavior for
> installations that are really depending on it.

Hmm.  I guess that would mean that the same connection string would
mean something different depending on how you configured this
behavior, which does not sound like a good idea.  But why not go the
other way and just create the default database by default, either in
addition to or instead of the postgres database?  I mean, most people
probably do this:

initdb
pg_ctl start
createdb

I would argue that only a very small minority does that. The majority does one of:

yum install postgresql
setup-postgresql <can't remember the switch offhand, but the one that calls initdb for them>
systemctl start postgresql

or
apt-get install postgresql
(which auto-initdb's and autostarts)


or
double-clicks graphical installer on windows
(which auto-initdb's and autostarts)


I would bet each one of those have at least one and probably more than one order of magnitude more users than ever call initdb or pg_ctl manually.


--