Thread: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

On 12/13/16 12:47 PM, Vladimir Rusinov wrote:
> Based on discussion in
> https://www.postgresql.org/message-id/CAE1wr-w%3DLE1cK5uG_rmAh-VBxc4_Bnw-gAE3qSqL-%3DtWwvLvjQ%40mail.gmail.com.
>     
> Tested via regression tests.
> To be applied in master only and to be included in 10.0.

I don't think the idea was to rename just one "xlog" function to "wal".

Personally, I think this is not important, but if you want to do it, I'd
follow the suggestion in the thread to rename all functions and leave
the old names as aliases or wrappers of some kind.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Dec 15, 2016 at 1:20 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
>
> On Wed, Dec 14, 2016 at 4:07 PM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
> Others will follow later in separate patches. Or is it preferred to have one
> huge patch submitted?

Please yes. One change makes little sense.

>> Personally, I think this is not important, but if you want to do it, I'd
>> follow the suggestion in the thread to rename all functions and leave
>> the old names as aliases or wrappers of some kind.
>
> I think I agree with Michael Paquier: "Better to do breakages in a single
> release rather than spreading them across releases.".
> There's going to be a lot of broken scripts following pg_xlog rename, so I
> think it makes sense to just drop functions as well.

For consistency that makes sense in my view. But I won't be too noisy
as well if people think that we should keep aliases for compatibility.

> We don't maintain pg_xlog -> pg_wal symlink, do we?

This part would be a mess for base backups, that's why we didn't do
it. Symlinks are equivalent to empty directories in the pg_basebackup
world. So by having one you would very likely break your backup *and*
restore tools silently.
-- 
Michael

On Thu, Dec 15, 2016 at 8:51 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> On Thu, Dec 15, 2016 at 1:20 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
>>> Personally, I think this is not important, but if you want to do it, I'd
>>> follow the suggestion in the thread to rename all functions and leave
>>> the old names as aliases or wrappers of some kind.
>>
>> I think I agree with Michael Paquier: "Better to do breakages in a single
>> release rather than spreading them across releases.".
>> There's going to be a lot of broken scripts following pg_xlog rename, so I
>> think it makes sense to just drop functions as well.
>
> For consistency that makes sense in my view. But I won't be too noisy
> as well if people think that we should keep aliases for compatibility.

Actually, I am changing my mind on this bit, following Peter's
opinion. Maintaining aliases of the past functions is a no-brainer,
and it has no cost in terms of maintenance. That will save much pain
to many monitoring scripts as well.
-- 
Michael

1) I agree with Michael that we should make this change backward compatible. It would help PostgreSQL image if we did
notbreak everyone's code. It costs businesses money to rewrite code (e.g. middle tier software, backup tools, etc),
testand redeploy to their customers. 
 

2) We decided to rename the pg_xlog directory because people were deleting it when disks were getting full thinking it
wasjust unimportant logging data; I get that. I'm a little unclear why we need to change the functions - it would be
lesspainful to our users and less risky if we just left them as is. Could someone please elaborate why this change is
necessary?I'm just trying to understand that.
 

Thanks,
-Cynthia

Cynthia,

* Cynthia Shang (cynthia.shang@crunchydata.com) wrote:
> 1) I agree with Michael that we should make this change backward compatible. It would help PostgreSQL image if we did
notbreak everyone's code. It costs businesses money to rewrite code (e.g. middle tier software, backup tools, etc),
testand redeploy to their customers.  

While I agree that we don't want to break client code or to make
backwards incompatible changes without good cause, in this case, it's
definitely a good cause and it makes sense to have things be consistent
and that includes changing these functions.

We make backwards-incompatible changes with each major release, which is
part of why we support older versions of PG for as long as we do- to
give PG users time to make any necessary changes for the new version of
PG.  One could argue that we shouldn't ever make a backwards
incompatible change because it will break an existing user's code and
cost users time and effort to rewrite that code, but the flip side of
that is that the extra code and complexity results in its own
maintenance burdens for the code and the project moving forward.

Generally speaking, we've also found that backwards compatibility
'features' end up having a much longer life than they should.
Ultimately, the best way forward tends to be either make the backwards
incompatible change or don't make the change at all.  We've already
agreed on this particular change, and with good reason, so the way
forward is to make the rest of the changes, not to go half-way or to try
and provide some backwards compatibility complexity.

> 2) We decided to rename the pg_xlog directory because people were deleting it when disks were getting full thinking
itwas just unimportant logging data; I get that. I'm a little unclear why we need to change the functions - it would be
lesspainful to our users and less risky if we just left them as is. Could someone please elaborate why this change is
necessary?I'm just trying to understand that. 

It would be inconsistent to change the directory name without also
changing the documentation, functions, and other user-facing pieces.

Thanks!

Stephen

Cynthia,

Please don't top-post on the PG mailing lists but rather write responses
in-line.

* Cynthia Shang (cynthia.shang@crunchydata.com) wrote:
> I have never heard of coding standards where naming conventions required a function/variable name match a directory
orfile name. It seems that would be restrictive.  

We aren't discussing a coding standard, we're talking about changing the
references from 'xlog' to 'wal' to match the change we did to the
directory, to be consistent.

> I'm not trying to pick a fight, I just think the pros should outweigh the cons when choosing a path forward. In this
caseI see lots of cons and one partial pro; partial because renaming only user facing functions and documentation will
createinconsistency within the Postgres code for all of the following. It sounds as if your minds are already made up,
whichsaddens me but I will say nothing further on the matter. 

All backwards incompatible changes are judgement calls and people are
certainly welcome to have different opinions.  I have a pretty strong
feeling about this particular change being worthwhile and also pretty
long overdue.

[... list of references to xlog in the code ...]

No need to include a list, I can find them myself. :)

Nearly all of the references listed are internal functions or references
in the code.  Those don't need to be changed, just the user-facing
pieces, which is quite a bit less.

The doxygen website is generated directly off of the code and isn't
user-facing documentation.

Thanks!

Stephen

On Fri, Dec 30, 2016 at 2:59 AM, Stephen Frost <sfrost@snowman.net> wrote:
> All backwards incompatible changes are judgement calls and people are
> certainly welcome to have different opinions.  I have a pretty strong
> feeling about this particular change being worthwhile and also pretty
> long overdue.

The renaming is definitely worth it to be consistent with the new
pg_wal/. Now as in the case of those functions we have ways a simple
way to avoid useless pain to users we should really take it easy. So
introducing a set of aliases and deprecating them after a couple of
years is a fine plan IMO.
-- 
Michael

On Fri, Dec 30, 2016 at 8:08 PM, Vladimir Rusinov <vrusinov@google.com> wrote:
> Now, I'm not sure whether it is worth maintaining function aliases. Assuming
> these are indeed trivial (can somebody point me to example?) I see roughly
> the same amount of downsides both ways.
> Having aliases raises additional questions:
> - do we keep them documented (probably not?)
> - do we keep them forever or kill in some future version?

The idea here is to keep documented only the new function names, but
mention in the release notes that aliases are kept, and that those
will be dropped in a couple of years (see for example 5d58c07a for
initdb). This will give plenty of time to monitoring script
maintainers to adapt to the new world order.
-- 
Michael

* Michael Paquier (michael.paquier@gmail.com) wrote:
> On Fri, Dec 30, 2016 at 8:08 PM, Vladimir Rusinov <vrusinov@google.com> wrote:
> > Now, I'm not sure whether it is worth maintaining function aliases. Assuming
> > these are indeed trivial (can somebody point me to example?) I see roughly
> > the same amount of downsides both ways.
> > Having aliases raises additional questions:
> > - do we keep them documented (probably not?)
> > - do we keep them forever or kill in some future version?
>
> The idea here is to keep documented only the new function names, but
> mention in the release notes that aliases are kept, and that those
> will be dropped in a couple of years (see for example 5d58c07a for
> initdb). This will give plenty of time to monitoring script
> maintainers to adapt to the new world order.

I don't particularly like this.  Undocumented aliases that make things
keep working are bound to just confuse people who are looking at what's
happening (eg: logging queries) and wondering "what the heck is that
function?!" and then if they're unable to find it in the docs then they
might think it's some local function that they can remove or similar.

Worse, those function aliases will probably continue to persist for
forever because no one can agree on removing them.

We maintain back-branches for years to provide people time to adjust
their code and whatever else they need to, no one is going to be forced
to move to 10 for quite a few years yet.

Additionally, people who are actually using these bits of the system are
almost certainly going to have to adjust things for the directory
change, having to change other bits of related code nearby at the same
time is much better than someone changing just the directory now and
then having to remmeber/realize that they have to change the function
calls in a few years or "whenever the PG people decide to remove the
function aliases."

Let's make this a clean break/change.  Perhaps it will also encourage
people who have their own hacked together scripts to consider using a
well maintained alternative solution.

Thanks!

Stephen

On Fri, Dec 30, 2016 at 09:57:25AM -0500, Stephen Frost wrote:

> Let's make this a clean break/change.

+1 

If there are known management gizmos to notify, it'd be nice to try to
give them some sort of warning, but even for them, the release notes
should spell it out clearly.

That business where people would delete files with "log" in the path,
not infrequently on a system running at capacity, isn't just
theoretical.  I've seen people lose data permanently that way, and I
know I'm not the only one who's seen this in the real world.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On 12/30/2016 06:46 PM, David Fetter wrote:
> On Fri, Dec 30, 2016 at 09:57:25AM -0500, Stephen Frost wrote:
> 
>> Let's make this a clean break/change.
> 
> +1 

+1
-- 
Vik Fearing                                          +33 6 46 75 15 36
http://2ndQuadrant.fr     PostgreSQL : Expertise, Formation et Support

On 12/30/16 9:57 AM, Stephen Frost wrote:
> Additionally, people who are actually using these bits of the system are
> almost certainly going to have to adjust things for the directory
> change,

Some *xlog* functions are commonly used to measure replay lag.  That
usage would not be affected by the directory renaming.  Renaming those
functions would only serve to annoy users and have them delay their
upgrades.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 1/1/17 9:48 AM, Peter Eisentraut wrote:
> On 12/30/16 9:57 AM, Stephen Frost wrote:
>> Additionally, people who are actually using these bits of the system are
>> almost certainly going to have to adjust things for the directory
>> change,
>
> Some *xlog* functions are commonly used to measure replay lag.  That
> usage would not be affected by the directory renaming.  Renaming those
> functions would only serve to annoy users and have them delay their
> upgrades.

Perhaps we should split the difference and do what we did for XML: 
provide a contrib module with alias functions using the old (xlog) names.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

On 1/2/17 12:30 PM, Jim Nasby wrote:
> On 1/1/17 9:48 AM, Peter Eisentraut wrote:
>> On 12/30/16 9:57 AM, Stephen Frost wrote:
>>> Additionally, people who are actually using these bits of the system are
>>> almost certainly going to have to adjust things for the directory
>>> change,
>>
>> Some *xlog* functions are commonly used to measure replay lag.  That
>> usage would not be affected by the directory renaming.  Renaming those
>> functions would only serve to annoy users and have them delay their
>> upgrades.
> 
> Perhaps we should split the difference and do what we did for XML:
> provide a contrib module with alias functions using the old (xlog) names.

-1

Since these functions are normally used by admins and not generally used
in SQL and functions, I'm not convinced the maintenance of the extension
would be worth it.  Admins are free to create whatever aliases they need
to get their work done.

-- 
-David
david@pgmasters.net

On 1/2/17 11:39 AM, David Steele wrote:
>
>
> On 1/2/17 12:30 PM, Jim Nasby wrote:
>> On 1/1/17 9:48 AM, Peter Eisentraut wrote:
>>> On 12/30/16 9:57 AM, Stephen Frost wrote:
>>>> Additionally, people who are actually using these bits of the system are
>>>> almost certainly going to have to adjust things for the directory
>>>> change,
>>>
>>> Some *xlog* functions are commonly used to measure replay lag.  That
>>> usage would not be affected by the directory renaming.  Renaming those
>>> functions would only serve to annoy users and have them delay their
>>> upgrades.
>>
>> Perhaps we should split the difference and do what we did for XML:
>> provide a contrib module with alias functions using the old (xlog) names.
>
> -1
>
> Since these functions are normally used by admins and not generally used
> in SQL and functions, I'm not convinced the maintenance of the extension
> would be worth it.  Admins are free to create whatever aliases they need
> to get their work done.

AIUI several others are arguing that this name change is going to break 
a lot of user monitoring code. I certainly agree with Stephen that some 
of the *xlog* functions are used for monitoring replay lag. So I think a 
backwards compatibility fix is reasonable.

Why would we force users to each come up with their own solution to this 
when we can just provide one?

BTW, I think fears of the maintenance cost of a contrib module are 
pretty overblown... it's not like we change these functions that often. 
We have added quite a few in the last few releases, but we don't need 
backwards compatibility for new stuff.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> On 1/2/17 11:39 AM, David Steele wrote:
>> On 1/2/17 12:30 PM, Jim Nasby wrote:
>>> On 1/1/17 9:48 AM, Peter Eisentraut wrote:
>>> Perhaps we should split the difference and do what we did for XML:
>>> provide a contrib module with alias functions using the old (xlog) names.

>> -1
>> Since these functions are normally used by admins and not generally used
>> in SQL and functions, I'm not convinced the maintenance of the extension
>> would be worth it.  Admins are free to create whatever aliases they need
>> to get their work done.

> BTW, I think fears of the maintenance cost of a contrib module are 
> pretty overblown... it's not like we change these functions that often. 
> We have added quite a few in the last few releases, but we don't need 
> backwards compatibility for new stuff.

I'm also -1 on this idea.  If we're going to provide backwards
compatibility, we should just leave the old names in the core.
Providing an extension is more work for *everybody* --- for us, and
for the users who will have to learn about and install the extension.
I don't see any gain to justify that work, either.
        regards, tom lane

On Tue, Jan 3, 2017 at 4:18 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I'm also -1 on this idea.  If we're going to provide backwards
> compatibility, we should just leave the old names in the core.
> Providing an extension is more work for *everybody* --- for us, and
> for the users who will have to learn about and install the extension.
> I don't see any gain to justify that work, either.

Yeah, let's make the life of users just easier if we can, without any
extension. Some people are likely going to forget to enable it anyway,
and some more don't like installing the package dedicated to contrib
modules.
-- 
Michael

Vladamir, all,

* Vladimir Rusinov (vrusinov@google.com) wrote:
> On Tue, Jan 3, 2017 at 11:56 AM, Michael Paquier <michael.paquier@gmail.com>
> wrote:
>
> > Yeah, let's make the life of users just easier if we can, without any
> > extension. Some people are likely going to forget to enable it anyway,
> > and some more don't like installing the package dedicated to contrib
> > modules.
>
> I think I +1 on this.
> I've did a github search on these function names and there is a lot of code
> that use them. E.g. there is 8.5k hits for pg_last_xlog_location
> <https://github.com/search?q=pg_last_xlog_replay_location&type=Code&utf8=%E2%9C%93>;
> a lot of them a forks and copy-pastes, but still, that's quite a lot. Let's
> keep the aliases around for couple of versions after which hopefully a lot
> of the code will be updated.

And there's 12k hits for pg_xlog.

If we do that, we'll just end up with exactly the same question about
removing them and the same amount of code breakage in a few years.  I
don't see how that is really helping anyone.

> After I'm done with a patch I can take a look if there's a way to mass-fill
> issues to affected projects on github (and maybe other hosting providers)
> and ask them to migrate to new function names. This is of course will not
> fix everything, but it would be a significant chunk. I suspect a lot of
> private tools/scripts borrow snippets from github as well.

If they're maintained, then they'll be updated.  I don't have any
sympathy if they aren't maintained.

If we really feel that this is the only thing between 9.6 and 10 that'll
cause problems for some serious amount of code and we don't expect to
change the function APIs anytime in the near future then perhaps we
could keep aliases, *document* them, and treat them as full functions
just like the regular ones.  Doing that requires a bit of extra
documentation and a few extra entries in pg_proc.  If we ever change
them then we'll need to make sure to let people know that they're both
changing, of course, in release notes and anywhere else.

Personally, I'm not excited about keeping these as documented aliases
because, frankly, I don't think it's the only thing that's going to
break for these projects between 9.6 and 10, and if they're being well
maintained then making the change isn't going to be a big deal, but I
won't object to adding them if that's what it takes to make progress
here.

Thanks!

Stephen

* Vladimir Rusinov (vrusinov@google.com) wrote:
> On Tue, Jan 3, 2017 at 3:37 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > If they're maintained, then they'll be updated.  I don't have any
> >
> sympathy if they aren't maintained.
> >
>
> Updating may be non-trivial effort even if they are maintained. E.g. some
> project may need to support both 9.6 and 10.0, and a lot of them written in
> a way that makes conditionals on function names non-trivial (e.g. there's
> just flat .sql file that is expected to work everywhere).

I don't think we should be concerned with that.  Anything but the
simplest SQL scripts have a good chance of being broken across major
version upgrades, *especially* ones that are using admin functions,
which we change on a regular basis for various reasons.

>  *document* them, and treat them as full functions just like the regular
> > ones.
>
> In my next WIP version of the patch (
> https://github.com/vrusinov/postgres/tree/rename-xlog) I keep references to
> old names in the description for the new names. This makes it
> searchable/greppable and does not encourage their usage. Exact format may
> change, but I'd certainly not like to treat them exactly like full
> functions and I'd like them to be mentioned so it's possible to search.

My point is that we need to document them and make sure to mention them
whenever we change anything about them (release notes, etc), exactly to
make sure that someone reading the release notes will know what to
search their code for.

> Overall, I don't feel super-strong either way. Adding aliases does not seem
> like a lot of effort or burden and I also see arguments for dropping them
> sooner than later. As a compromise I propose keep aliases with hard
> deadline for removal in 11.0.

Ugh, no, I don't want to have this argument again next year.

> This way we are nicer to people who maintain their tools and read release
> notes via giving them more time, and nicer to ourselves via cleaning legacy
> stuff relatively soon.

They have plenty of time.  If we're going to remove them, then let's
just do it and have a clean break.

Thanks!

Stephen

Hi,

On 2017-01-03 10:37:08 -0500, Stephen Frost wrote:
> * Vladimir Rusinov (vrusinov@google.com) wrote:
> > I think I +1 on this.
> > I've did a github search on these function names and there is a lot of code
> > that use them. E.g. there is 8.5k hits for pg_last_xlog_location
> > <https://github.com/search?q=pg_last_xlog_replay_location&type=Code&utf8=%E2%9C%93>;
> > a lot of them a forks and copy-pastes, but still, that's quite a lot. Let's
> > keep the aliases around for couple of versions after which hopefully a lot
> > of the code will be updated.
> 
> And there's 12k hits for pg_xlog.

> If we do that, we'll just end up with exactly the same question about
> removing them and the same amount of code breakage in a few years.  I
> don't see how that is really helping anyone.

Meh^2.  The cost of having pg_xlog was that people lost their
data. Hence their was motivation of changing things. The cost of having
some function aliases is, what, a pg_proc line?  If we end up carrying
them forever, so what?


> If we really feel that this is the only thing between 9.6 and 10 that'll
> cause problems for some serious amount of code and we don't expect to
> change the function APIs anytime in the near future then perhaps we
> could keep aliases, *document* them, and treat them as full functions
> just like the regular ones.

I think we've been far to cavalier lately about unnecessarily breaking
admin and monitoring tools. There's been pg_stat_activity backward
incompat changes in most of the last releases. It's a *PAIN* to develop
monitoring / admin tools that work with a number of releases.  It's fine
to cause that pain if there's some considerable benefit (e.g. not
triggering data loss seems like a case for that, as imo is unifying
configuration), but I don't see how that justifying breaking things
gratuitously.  Just renaming well known functions for a minor bit of
cleanliness seems not to survive a cost/benefit analysis.

Greetings,

Andres Freund

On 4 January 2017 at 07:32, Andres Freund <andres@anarazel.de> wrote:

> I think we've been far to cavalier lately about unnecessarily breaking
> admin and monitoring tools.

> Just renaming well known functions for a minor bit of
> cleanliness seems not to survive a cost/benefit analysis.

+1

This thread is at best a minor cleanup item, yet major work still
remains for 10.0.

-- 
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andres,

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-01-03 10:37:08 -0500, Stephen Frost wrote:
> > * Vladimir Rusinov (vrusinov@google.com) wrote:
> > > I think I +1 on this.
> > > I've did a github search on these function names and there is a lot of code
> > > that use them. E.g. there is 8.5k hits for pg_last_xlog_location
> > > <https://github.com/search?q=pg_last_xlog_replay_location&type=Code&utf8=%E2%9C%93>;
> > > a lot of them a forks and copy-pastes, but still, that's quite a lot. Let's
> > > keep the aliases around for couple of versions after which hopefully a lot
> > > of the code will be updated.
> >
> > And there's 12k hits for pg_xlog.
>
> > If we do that, we'll just end up with exactly the same question about
> > removing them and the same amount of code breakage in a few years.  I
> > don't see how that is really helping anyone.
>
> Meh^2.  The cost of having pg_xlog was that people lost their
> data. Hence their was motivation of changing things. The cost of having
> some function aliases is, what, a pg_proc line?  If we end up carrying
> them forever, so what?

I outlined exactly that in the part you didn't quote.

> > If we really feel that this is the only thing between 9.6 and 10 that'll
> > cause problems for some serious amount of code and we don't expect to
> > change the function APIs anytime in the near future then perhaps we
> > could keep aliases, *document* them, and treat them as full functions
> > just like the regular ones.
>
> I think we've been far to cavalier lately about unnecessarily breaking
> admin and monitoring tools. There's been pg_stat_activity backward
> incompat changes in most of the last releases. It's a *PAIN* to develop
> monitoring / admin tools that work with a number of releases.  It's fine
> to cause that pain if there's some considerable benefit (e.g. not
> triggering data loss seems like a case for that, as imo is unifying
> configuration), but I don't see how that justifying breaking things
> gratuitously.  Just renaming well known functions for a minor bit of
> cleanliness seems not to survive a cost/benefit analysis.

I agree that we have been breaking monitoring tools on the regular for a
while as we continue to add capabilities and improve things, which is
part of the reason that I don't see this particular break as a very big
deal- serious monitoring tools are going to need to be updated anyway.

I don't see that changing any time soon either- we are woefully far
behind in this area compared to other databases and we should be trying
to encourage people to continue improving these areas, not making things
unnecessairly difficult by requiring backwards compatibility hacks.

As I said in what you did quote above- I won't complain if someone wants
the aliases and we include them in the documentation, but I don't agree
with the other suggestions of having undocumented aliases or not making
the change.

Thanks!

Stephen

Stephen Frost <sfrost@snowman.net> writes:
> As I said in what you did quote above- I won't complain if someone wants
> the aliases and we include them in the documentation, but I don't agree
> with the other suggestions of having undocumented aliases or not making
> the change.

FWIW, that position makes sense to me as well.  If we keep aliases,
let's just reconcile ourselves to the probability that they'll be there
indefinitely.  It's not that big a deal, and the other proposals are
bigger warts than having an alias.
        regards, tom lane

On 2017-01-04 09:38:42 -0500, Stephen Frost wrote:
> Andres,
> 
> * Andres Freund (andres@anarazel.de) wrote:
> > On 2017-01-03 10:37:08 -0500, Stephen Frost wrote:
> > > * Vladimir Rusinov (vrusinov@google.com) wrote:
> > > > I think I +1 on this.
> > > > I've did a github search on these function names and there is a lot of code
> > > > that use them. E.g. there is 8.5k hits for pg_last_xlog_location
> > > > <https://github.com/search?q=pg_last_xlog_replay_location&type=Code&utf8=%E2%9C%93>;
> > > > a lot of them a forks and copy-pastes, but still, that's quite a lot. Let's
> > > > keep the aliases around for couple of versions after which hopefully a lot
> > > > of the code will be updated.
> > > 
> > > And there's 12k hits for pg_xlog.
> > 
> > > If we do that, we'll just end up with exactly the same question about
> > > removing them and the same amount of code breakage in a few years.  I
> > > don't see how that is really helping anyone.
> > 
> > Meh^2.  The cost of having pg_xlog was that people lost their
> > data. Hence their was motivation of changing things. The cost of having
> > some function aliases is, what, a pg_proc line?  If we end up carrying
> > them forever, so what?
> 
> I outlined exactly that in the part you didn't quote.

Sure, but my point was that you and several others essentially argue for
breaking things gratuitously. And I think that's bad. That you can live
with backward compatibility doesn't change that point.


> > > If we really feel that this is the only thing between 9.6 and 10 that'll
> > > cause problems for some serious amount of code and we don't expect to
> > > change the function APIs anytime in the near future then perhaps we
> > > could keep aliases, *document* them, and treat them as full functions
> > > just like the regular ones.
> > 
> > I think we've been far to cavalier lately about unnecessarily breaking
> > admin and monitoring tools. There's been pg_stat_activity backward
> > incompat changes in most of the last releases. It's a *PAIN* to develop
> > monitoring / admin tools that work with a number of releases.  It's fine
> > to cause that pain if there's some considerable benefit (e.g. not
> > triggering data loss seems like a case for that, as imo is unifying
> > configuration), but I don't see how that justifying breaking things
> > gratuitously.  Just renaming well known functions for a minor bit of
> > cleanliness seems not to survive a cost/benefit analysis.
> 
> I agree that we have been breaking monitoring tools on the regular for a
> while as we continue to add capabilities and improve things, which is
> part of the reason that I don't see this particular break as a very big
> deal- serious monitoring tools are going to need to be updated anyway.

I have no problem with breaking things when necessary. I do have a
problem when we do so willy-nilly when providing backward-compatibility
would have been easy enough. E.g. Nearly all recent pg_stat_activity
changes would have been doable just as well without breakage.


> I don't see that changing any time soon either- we are woefully far
> behind in this area compared to other databases and we should be trying
> to encourage people to continue improving these areas, not making things
> unnecessairly difficult by requiring backwards compatibility hacks.

By breaking stuff on a regular basis we're also showing that we're
behind. Compatibility also is a big topic. We also force users to stay
behind longer, and to upgrade less frequently.


Andres

Andres,

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-01-04 09:38:42 -0500, Stephen Frost wrote:
> > * Andres Freund (andres@anarazel.de) wrote:
> > > On 2017-01-03 10:37:08 -0500, Stephen Frost wrote:
> > > > * Vladimir Rusinov (vrusinov@google.com) wrote:
> > > > > I think I +1 on this.
> > > > > I've did a github search on these function names and there is a lot of code
> > > > > that use them. E.g. there is 8.5k hits for pg_last_xlog_location
> > > > > <https://github.com/search?q=pg_last_xlog_replay_location&type=Code&utf8=%E2%9C%93>;
> > > > > a lot of them a forks and copy-pastes, but still, that's quite a lot. Let's
> > > > > keep the aliases around for couple of versions after which hopefully a lot
> > > > > of the code will be updated.
> > > >
> > > > And there's 12k hits for pg_xlog.
> > >
> > > > If we do that, we'll just end up with exactly the same question about
> > > > removing them and the same amount of code breakage in a few years.  I
> > > > don't see how that is really helping anyone.
> > >
> > > Meh^2.  The cost of having pg_xlog was that people lost their
> > > data. Hence their was motivation of changing things. The cost of having
> > > some function aliases is, what, a pg_proc line?  If we end up carrying
> > > them forever, so what?
> >
> > I outlined exactly that in the part you didn't quote.
>
> Sure, but my point was that you and several others essentially argue for
> breaking things gratuitously. And I think that's bad. That you can live
> with backward compatibility doesn't change that point.

I do not agree that we are arguing for breaking things gratuitously.
That's saying that there's no justification for such a change and I
think that's wrong- the prior names were driven off the directory name
and were, really, not well suited to what the functions were actually
for.  This change is to align the functions with the agreed-upon
directory name change and because the new name is distinctly more
accurate than the prior name.  As discussed previously, the term
'transaction log' is more closely related to what *clog* is than the
write-ahead log.

My feeling is that people who are arguing for the aliases are doing so
from a position of "it's easy for us to do, so we should," which I am
generally not in favor of when it comes to backwards compatibility
because it leaves things in an inconsistent state.  Tools which monitor
the progression of our write-ahead-log (wal) from the primary to stanbys
will end up using a function that has "xlog" in the name, even though
that's an inconsistent and less than accurate name for what they're
actually doing, simply because "well, that part didn't break, so we
don't have to change it, even though these other things had to be
changed to refer to 'wal'".  That, in my opinion, isn't the right
compromise to be making.

> > > > If we really feel that this is the only thing between 9.6 and 10 that'll
> > > > cause problems for some serious amount of code and we don't expect to
> > > > change the function APIs anytime in the near future then perhaps we
> > > > could keep aliases, *document* them, and treat them as full functions
> > > > just like the regular ones.
> > >
> > > I think we've been far to cavalier lately about unnecessarily breaking
> > > admin and monitoring tools. There's been pg_stat_activity backward
> > > incompat changes in most of the last releases. It's a *PAIN* to develop
> > > monitoring / admin tools that work with a number of releases.  It's fine
> > > to cause that pain if there's some considerable benefit (e.g. not
> > > triggering data loss seems like a case for that, as imo is unifying
> > > configuration), but I don't see how that justifying breaking things
> > > gratuitously.  Just renaming well known functions for a minor bit of
> > > cleanliness seems not to survive a cost/benefit analysis.
> >
> > I agree that we have been breaking monitoring tools on the regular for a
> > while as we continue to add capabilities and improve things, which is
> > part of the reason that I don't see this particular break as a very big
> > deal- serious monitoring tools are going to need to be updated anyway.
>
> I have no problem with breaking things when necessary. I do have a
> problem when we do so willy-nilly when providing backward-compatibility
> would have been easy enough. E.g. Nearly all recent pg_stat_activity
> changes would have been doable just as well without breakage.

Would those changes have resulted in an inconsistent and difficult
system to understand and follow?  For pg_stat_activity, frankly, I don't
recall offhand which probably means they weren't a very big deal for me.

Changes to anything related to the 'pg_xlog' directory or the set of
functions that pgbackrest or our tools care about are a lot closer to me
and changes there represent serious time that I'll need to ask David and
Jeff to spend on updating pgbackrest and our tooling for, yet I've had
no push-back from them on this because there is general agreement that
this is a good change to be making.

> > I don't see that changing any time soon either- we are woefully far
> > behind in this area compared to other databases and we should be trying
> > to encourage people to continue improving these areas, not making things
> > unnecessairly difficult by requiring backwards compatibility hacks.
>
> By breaking stuff on a regular basis we're also showing that we're
> behind. Compatibility also is a big topic. We also force users to stay
> behind longer, and to upgrade less frequently.

I don't mind showing that we're behind, it's a fact and anyone who cares
to will found that out quickly enough.  I do object to putting up
roadblocks or making it more difficult than necessary to making
progress.

I have a pretty hard time believing that users are going to seriously
stay on older versions for very much longer just because of these
changes.  Either they have a strong interest in the new features in
later releases and have cause to upgrade, in which case updating a few
function calls isn't going to be a big deal, or they don't have much
cause to change and therefore they're not going to rush to do so (which
is an entirly acceptable position!).

Lastly, if they have an issue with the tooling around the database that
uses these functions or they have an issue with the maintenance
requirements associated with such relativly mundane changes, that they
instead use tools which are well maintained and are updated consistently
with the changes being made.

Thanks!

Stephen

On Fri, Jan 6, 2017 at 1:31 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
> Attaching a patch that renames all 'xlog' functions, keeping aliases for old
> ones (since it looks like majority vote is for keeping them).

OK.

> - OIDs - where do I get numbers from? I was kinda choosing them at random,
> unaware if there is some process for keeping track of them. Please point me
> if such thing exists and I'll change them.

You can use src/include/catalog/unused_oids to look at the OIDs not
yet assigned. Assigning them logically increasing may be a good idea.
But OID ordering is not that mandatory for the internal functions and
operators.

> - Documentation. I've added a new section with a simple table, keeping it
> somewhat neglected on purpose. We'll have old names in index and searchable
> on page, but nothing more. Is it sufficient?

I find quite clear what you have produced here. The table states
clearly the one-one mapping between the old and new entries.

> - New function names. I've used 'recovery' instead of 'xlog_replay'

I would not have touched this one. But if we go this way, let's
bike-shed and use pg_recovery_ as prefix for consistency.

> and used 'wal_file' instead of 'xlogfile'. Does it make sense?

This one yes.

For simplicity I would just have done a s/xlog/wal/ and keep the names
consistent a maximum with the old ones. But I am just working here.
You clearly put thoughts in the new names.

> - Release notes. I was unable to find a draft for 10.0. How do I make sure
> these renames are not forgotten?

That's part of the job of the release note writer, very likely Tom or
Bruce. The commit message should be explicit enough that the writer
does not have to dig into the code itself for the changes.
-- 
Michael

On Wed, Jan  4, 2017 at 09:38:42AM -0500, Stephen Frost wrote:
> > I think we've been far to cavalier lately about unnecessarily breaking
> > admin and monitoring tools. There's been pg_stat_activity backward
> > incompat changes in most of the last releases. It's a *PAIN* to develop
> > monitoring / admin tools that work with a number of releases.  It's fine
> > to cause that pain if there's some considerable benefit (e.g. not
> > triggering data loss seems like a case for that, as imo is unifying
> > configuration), but I don't see how that justifying breaking things
> > gratuitously.  Just renaming well known functions for a minor bit of
> > cleanliness seems not to survive a cost/benefit analysis.
> 
> I agree that we have been breaking monitoring tools on the regular for a
> while as we continue to add capabilities and improve things, which is
> part of the reason that I don't see this particular break as a very big
> deal- serious monitoring tools are going to need to be updated anyway.
> 
> I don't see that changing any time soon either- we are woefully far
> behind in this area compared to other databases and we should be trying
> to encourage people to continue improving these areas, not making things
> unnecessairly difficult by requiring backwards compatibility hacks.

I agree.  I know these changes are painful, and I will take
responsibility for perhaps the most cavalier breakage of
pg_stat_statements in renaming column procpid to pid.  (I still stand
behind that change.)

As painful as these breakages are *at* the time of the breakage,
long-term, it keeps our user API clean.  Frankly, because changes are
reviewed by so many people, we don't normally make dumb API mistakes
that require changes --- rather new features or complex interactions
require user API changes.  We often feel we have too many users to make
the change, but every few years we double our user base and all those
new users see a nice clean API that previous users had to adjust to.

If we don't make changes like this, our API becomes nonintuitive very
quickly, and let's face it, we expose a lot of internal interfaces to
our users, so clarity is extra important.

I don't think anyone is arguing that these API breakages are cost-free,
but I think long-term, the costs are minor compared to the clean API we
provide to users.

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

On 1/6/17 7:21 PM, Bruce Momjian wrote:
> I don't think anyone is arguing that these API breakages are cost-free,
> but I think long-term, the costs are minor compared to the clean API we
> provide to users.

Except in this case we can provide a clean new API without gratuitously 
breaking the old one.

I absolutely do agree that we have to have a way of making incompatible 
changes from time to time (as I've been arguing over and over in the 
plpgsql2 thread). I also think a one-size-fits-all approach to when a 
break is allowed would be good. That said, the change to 
pg_stat_activity caused a lot of needless user pain, and this one will 
as well.

What makes this even more frustrating (to me at least) is that the 
attitude that's been demonstrated around plpgsql is that there can 
absolutely NEVER be a compatibility break of any kind.

So part of the project thinks it's perfectly OK to just break things 
without any warning or support, while another part of the project 
adamantly refuses any kind of a break at all, even what's breaking has 
never officially been supported.

I don't think that dichotomy is good for the community or for our users.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

On 1/9/17 10:50 AM, Vladimir Rusinov wrote:
> pg_is_xlog_replay_paused        | pg_is_recovery_paused

All the other xlog_replay names have been changed to wal_replay, so
making this one different is probably not so good.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Tue, Jan 10, 2017 at 1:53 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
>
> On Mon, Jan 9, 2017 at 4:14 PM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
>>
>> > pg_is_xlog_replay_paused        | pg_is_recovery_paused
>>
>> All the other xlog_replay names have been changed to wal_replay, so
>> making this one different is probably not so good.
>
> Oops, forgot about that one. Attached patch with this addressed as well.

-                errhint("pg_xlogfile_name_offset() cannot be executed
during recovery.")));
+                errhint(
+                    "pg_wal_file_name_offset() cannot be executed
during recovery.")));
I am not sure that there is any need to reformat this error hint.

Your patch includes useless diff noise in pg_proc.h. By that I mean
the lines of pg_start_backup, pg_stop_backup, etc.

+/*  Next OID: 6016 */
+
I don't think you need that.

src/test/perl/PostgresNode.pm has added recently a new method called
lsn() that uses some of the WAL position functions. Their update is
necessary as well for this patch.

As there are two school of thoughts on this thread, keeping your patch
with the compatibility table is the best move for now. Even if we end
up by having a version without aliases, that will be just code to
remove in the final version.
-- 
Michael

On Wed, Jan 4, 2017 at 4:32 PM, Andres Freund <andres@anarazel.de> wrote:
> Hi,
>
> On 2017-01-03 10:37:08 -0500, Stephen Frost wrote:
>> * Vladimir Rusinov (vrusinov@google.com) wrote:
>> > I think I +1 on this.
>> > I've did a github search on these function names and there is a lot of code
>> > that use them. E.g. there is 8.5k hits for pg_last_xlog_location
>> > <https://github.com/search?q=pg_last_xlog_replay_location&type=Code&utf8=%E2%9C%93>;
>> > a lot of them a forks and copy-pastes, but still, that's quite a lot. Let's
>> > keep the aliases around for couple of versions after which hopefully a lot
>> > of the code will be updated.
>>
>> And there's 12k hits for pg_xlog.
>
>> If we do that, we'll just end up with exactly the same question about
>> removing them and the same amount of code breakage in a few years.  I
>> don't see how that is really helping anyone.
>
> Meh^2.  The cost of having pg_xlog was that people lost their
> data. Hence their was motivation of changing things. The cost of having
> some function aliases is, what, a pg_proc line?  If we end up carrying
> them forever, so what?
>
>
>> If we really feel that this is the only thing between 9.6 and 10 that'll
>> cause problems for some serious amount of code and we don't expect to
>> change the function APIs anytime in the near future then perhaps we
>> could keep aliases, *document* them, and treat them as full functions
>> just like the regular ones.
>
> I think we've been far to cavalier lately about unnecessarily breaking
> admin and monitoring tools. There's been pg_stat_activity backward
> incompat changes in most of the last releases. It's a *PAIN* to develop
> monitoring / admin tools that work with a number of releases.  It's fine
> to cause that pain if there's some considerable benefit (e.g. not
> triggering data loss seems like a case for that, as imo is unifying
> configuration), but I don't see how that justifying breaking things
> gratuitously.  Just renaming well known functions for a minor bit of
> cleanliness seems not to survive a cost/benefit analysis.

+1

Regards,

-- 
Fujii Masao

On Thu, Jan 12, 2017 at 2:00 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
> On Tue, Jan 10, 2017 at 5:24 AM, Michael Paquier <michael.paquier@gmail.com>
> wrote:
>> As there are two school of thoughts on this thread, keeping your patch
>> with the compatibility table is the best move for now. Even if we end
>> up by having a version without aliases, that will be just code to
>> remove in the final version.
>
> Indeed, it is trivial to kill aliases.
>
> New version of the patch attached.

The patch still updates a bunch of .po files. Those are normally
refreshed with the translation updates, so they had better be removed.
Other than that, the patch looks in good shape to me so switch to
"Ready for committer".

So, to sum up things on this thread, here are the votes about the use
of aliases or a pure breakage:
- No to aliases, shake the world: Stephen, Tom, David F, Vik, Bruce M => 5
- Yes to aliases: Michael P, Andres, Peter E., Cynthia S, Jim N,
Vladimir, Simon R, Fujii-san => 8
If I misunderstood things, please feel free to speak up.
-- 
Michael

On Thu, Jan 12, 2017 at 2:49 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> On Thu, Jan 12, 2017 at 2:00 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
>> On Tue, Jan 10, 2017 at 5:24 AM, Michael Paquier <michael.paquier@gmail.com>
>> wrote:
>>> As there are two school of thoughts on this thread, keeping your patch
>>> with the compatibility table is the best move for now. Even if we end
>>> up by having a version without aliases, that will be just code to
>>> remove in the final version.
>>
>> Indeed, it is trivial to kill aliases.
>>
>> New version of the patch attached.
>
> The patch still updates a bunch of .po files. Those are normally
> refreshed with the translation updates, so they had better be removed.
> Other than that, the patch looks in good shape to me so switch to
> "Ready for committer".
>
> So, to sum up things on this thread, here are the votes about the use
> of aliases or a pure breakage:
> - No to aliases, shake the world: Stephen, Tom, David F, Vik, Bruce M => 5
> - Yes to aliases: Michael P, Andres, Peter E., Cynthia S, Jim N,
> Vladimir, Simon R, Fujii-san => 8
> If I misunderstood things, please feel free to speak up.

To be pricise, I'd like to avoid the renaming of the functions at all rather
than using aliases.

Regards,

-- 
Fujii Masao

On 1/12/17 12:49 AM, Michael Paquier wrote:
> On Thu, Jan 12, 2017 at 2:00 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
>> On Tue, Jan 10, 2017 at 5:24 AM, Michael Paquier <michael.paquier@gmail.com>
>> wrote:
>>> As there are two school of thoughts on this thread, keeping your patch
>>> with the compatibility table is the best move for now. Even if we end
>>> up by having a version without aliases, that will be just code to
>>> remove in the final version.
>>
>> Indeed, it is trivial to kill aliases.
>>
>> New version of the patch attached.
> 
> The patch still updates a bunch of .po files. Those are normally
> refreshed with the translation updates, so they had better be removed.
> Other than that, the patch looks in good shape to me so switch to
> "Ready for committer".
> 
> So, to sum up things on this thread, here are the votes about the use
> of aliases or a pure breakage:
> - No to aliases, shake the world: Stephen, Tom, David F, Vik, Bruce M => 5
> - Yes to aliases: Michael P, Andres, Peter E., Cynthia S, Jim N,
> Vladimir, Simon R, Fujii-san => 8
> If I misunderstood things, please feel free to speak up.

I'm also in the "no to aliases" camp.

-- 
-David
david@pgmasters.net

On Thu, Jan 12, 2017 at 8:56 AM, David Steele <david@pgmasters.net> wrote:
>> So, to sum up things on this thread, here are the votes about the use
>> of aliases or a pure breakage:
>> - No to aliases, shake the world: Stephen, Tom, David F, Vik, Bruce M => 5
>> - Yes to aliases: Michael P, Andres, Peter E., Cynthia S, Jim N,
>> Vladimir, Simon R, Fujii-san => 8
>> If I misunderstood things, please feel free to speak up.
>
> I'm also in the "no to aliases" camp.

Me too.

I appreciate what people are saying about the pain that is involved
when we change administration interfaces.  However, I suspect that
most of that pain falls on tool authors, who are disproportionately
represented in this forum, and on DBAs, who may not like it much but
can probably cope with it.  It's not really going to affect
applications, because applications generally don't care about the
transaction log directly.  What is really bad is when we break syntax
that might be used in queries, or in PL/pgsql functions.  That's why
the 8.3 casting changes were so painful, and why the discussion about
changing PL/pgsql semantics is so dominated by people who don't want
to break what's worth now.  When we change things in those areas,
people who know nothing and care nothing about PostgreSQL other than
that it's the underlying database have to update their SQL, and that
hurts adoption.  Anyone who is affected by these proposed changes is
already deeply invested in PostgreSQL.

Also, I think it is absolutely wrong to suppose that there's no pain
involved in having these aliases around forever.  I thought the idea
of stuffing the aliases in a contrib module was a rather good one,
actually, because then the people who need them can install them
easily and yet the burden on core to maintain them is largely removed.
Also, there's still positive pressure for tools that haven't been
updated to get themselves updated to work without that compatibility
extension, so eventually the need for it should go away.  I have not
abandoned all hope of eventually removing contrib/tsearch2 from the
core distribution.  When we keep things in the core system catalogs,
they just accumulate forever.  There will not only be old code that
continues to use the old names; there will be new code that uses the
old names.  There will be people who ask why we have two copies of the
function, or aren't sure which one to use.  Somebody will have a
tab-completion fail that results from an irrelevant decision between
two names for the same thing.  Individually, these things don't matter
very much, but over time they lead to a user experience that's full of
warts and a developer experience that is enslaved to historical
garbage that takes time and energy away from new feature development.

So, my order of preference is:

1. Rename everything and add an extension to create aliases for the old names.
2. Rename everything without adding a backward-compatibility extension.
3. Don't rename anything.
4. Stomach flu.
5. Put a second name for everything in pg_proc.h.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 12-01-2017 10:56, David Steele wrote:
>> So, to sum up things on this thread, here are the votes about the use
>> of aliases or a pure breakage:
>> - No to aliases, shake the world: Stephen, Tom, David F, Vik, Bruce M => 5
>> - Yes to aliases: Michael P, Andres, Peter E., Cynthia S, Jim N,
>> Vladimir, Simon R, Fujii-san => 8
>> If I misunderstood things, please feel free to speak up.
> 
> I'm also in the "no to aliases" camp.
> 
+1. Even if we don't document aliases, one day some applications will
break when someone decided to clean up this code. Let's do it smoothly.
As Robert suggested in the other email: extension to create old names.


--   Euler Taveira                   Timbira - http://www.timbira.com.br/  PostgreSQL: Consultoria, Desenvolvimento,
Suporte24x7 e Treinamento
 

Vladimir Rusinov <vrusinov@google.com> writes:
> On Thu, Jan 12, 2017 at 4:57 PM, Euler Taveira <euler@timbira.com.br> wrote:
>> As Robert suggested in the other email: extension to create old names.

> I don't follow the reasoning for the extension. It seem to have downsides
> of both alternatives combined: we still break people's code, and we add
> even more maintenance burden than just keeping aliases.

Yeah, I'm not terribly for the extension idea.  Robert cited the precedent
of contrib/tsearch2, but I think the history of that is a good argument
against this: tsearch2 is still there 9 years later and there's no
indication that we'll ever get rid of it.  We can let things rot
indefinitely in core too.  If we do ever agree to get rid of the aliases,
stripping them out of pg_proc.h will not be any harder than removing
a contrib directory would be.
        regards, tom lane

On 1/12/17 10:12 AM, Tom Lane wrote:
> Yeah, I'm not terribly for the extension idea.  Robert cited the precedent
> of contrib/tsearch2, but I think the history of that is a good argument
> against this: tsearch2 is still there 9 years later and there's no
> indication that we'll ever get rid of it.  We can let things rot
> indefinitely in core too.  If we do ever agree to get rid of the aliases,
> stripping them out of pg_proc.h will not be any harder than removing
> a contrib directory would be.

Is there any burden to carrying tsearch2 around? Have we formally marked 
it as deprecated?

The way I see it, either one person can spend an hour or whatever 
creating an extension once, or every postgres install that's using any 
of these functions now has yet another hurdle to upgrading.

If PGXN was better supported by the community maybe the answer would be 
to just throw an extension up there. But that's certainly not the case.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

On Thu, Jan 12, 2017 at 12:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Vladimir Rusinov <vrusinov@google.com> writes:
>> On Thu, Jan 12, 2017 at 4:57 PM, Euler Taveira <euler@timbira.com.br> wrote:
>>> As Robert suggested in the other email: extension to create old names.
>
>> I don't follow the reasoning for the extension. It seem to have downsides
>> of both alternatives combined: we still break people's code, and we add
>> even more maintenance burden than just keeping aliases.
>
> Yeah, I'm not terribly for the extension idea.  Robert cited the precedent
> of contrib/tsearch2, but I think the history of that is a good argument
> against this: tsearch2 is still there 9 years later and there's no
> indication that we'll ever get rid of it.  We can let things rot
> indefinitely in core too.  If we do ever agree to get rid of the aliases,
> stripping them out of pg_proc.h will not be any harder than removing
> a contrib directory would be.

How about just leaving it to someone who cares about the aliases to
post such an extension at pgxn.org (or anywhere else they like).
It can be around as long as someone cares enough to maintain it.

I guess that makes my vote -1 on aliases by the project.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Jim,

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> The way I see it, either one person can spend an hour or whatever
> creating an extension once, or every postgres install that's using
> any of these functions now has yet another hurdle to upgrading.

I just don't buy this argument, at all.  These functions names are
certainly not the only things we're changing with PG10 and serious
monitoring/backup/administration tools are almost certainly going to
have quite a bit to adjust to with the new release, and that isn't news
to anyone who works with PG.

Thanks!

Stephen

On 2017-01-12 13:40:50 -0500, Stephen Frost wrote:
> Jim,
> 
> * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> > The way I see it, either one person can spend an hour or whatever
> > creating an extension once, or every postgres install that's using
> > any of these functions now has yet another hurdle to upgrading.
> 
> I just don't buy this argument, at all.  These functions names are
> certainly not the only things we're changing with PG10 and serious
> monitoring/backup/administration tools are almost certainly going to
> have quite a bit to adjust to with the new release, and that isn't news
> to anyone who works with PG.

By that argument we can just do arbitrary backward incompat changes.  We
should aspire to be better than we've been in the past, not use that
past as an excuse for not even trying.

Greetings,

Andres Freund

Stephen Frost <sfrost@snowman.net> writes:
> I just don't buy this argument, at all.  These functions names are
> certainly not the only things we're changing with PG10 and serious
> monitoring/backup/administration tools are almost certainly going to
> have quite a bit to adjust to with the new release, and that isn't news
> to anyone who works with PG.

Hmm --- we've been conducting this argument in a vacuum, but you're right,
we should consider what else is changing in v10.  If you can point to
already-committed changes that mean that code using these functions will
almost certainly need changes anyway for v10, then that would greatly
weaken the argument for providing aliases.
        regards, tom lane

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-01-12 13:40:50 -0500, Stephen Frost wrote:
> > * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> > > The way I see it, either one person can spend an hour or whatever
> > > creating an extension once, or every postgres install that's using
> > > any of these functions now has yet another hurdle to upgrading.
> >
> > I just don't buy this argument, at all.  These functions names are
> > certainly not the only things we're changing with PG10 and serious
> > monitoring/backup/administration tools are almost certainly going to
> > have quite a bit to adjust to with the new release, and that isn't news
> > to anyone who works with PG.
>
> By that argument we can just do arbitrary backward incompat changes.  We
> should aspire to be better than we've been in the past, not use that
> past as an excuse for not even trying.

When they're changes that are primairly going to affect
monitoring/backup/administration tools, yes, I do think we can make just
about arbitrary backward-incompatible changes.

As Robert mentioned, and I agree with, changing things which will impact
regular application usage of PG is a different story and one we should
be more cautious about.

Thanks!

Stephen

On January 12, 2017 10:50:18 AM PST, Stephen Frost <sfrost@snowman.net> wrote:
>* Andres Freund (andres@anarazel.de) wrote:
>> On 2017-01-12 13:40:50 -0500, Stephen Frost wrote:
>> > * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
>> > > The way I see it, either one person can spend an hour or whatever
>> > > creating an extension once, or every postgres install that's
>using
>> > > any of these functions now has yet another hurdle to upgrading.
>> >
>> > I just don't buy this argument, at all.  These functions names are
>> > certainly not the only things we're changing with PG10 and serious
>> > monitoring/backup/administration tools are almost certainly going
>to
>> > have quite a bit to adjust to with the new release, and that isn't
>news
>> > to anyone who works with PG.
>>
>> By that argument we can just do arbitrary backward incompat changes.
>We
>> should aspire to be better than we've been in the past, not use that
>> past as an excuse for not even trying.
>
>When they're changes that are primairly going to affect
>monitoring/backup/administration tools, yes, I do think we can make
>just
>about arbitrary backward-incompatible changes.
>
>As Robert mentioned, and I agree with, changing things which will
>impact
>regular application usage of PG is a different story and one we should
>be more cautious about.

I find it very hard to understand the justification for that, besides that it's strengthening external projects
providingmonitoring, backup, etc in a compatible way. 

Andres

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Stephen Frost <sfrost@snowman.net> writes:
> > I just don't buy this argument, at all.  These functions names are
> > certainly not the only things we're changing with PG10 and serious
> > monitoring/backup/administration tools are almost certainly going to
> > have quite a bit to adjust to with the new release, and that isn't news
> > to anyone who works with PG.
>
> Hmm --- we've been conducting this argument in a vacuum, but you're right,
> we should consider what else is changing in v10.  If you can point to
> already-committed changes that mean that code using these functions will
> almost certainly need changes anyway for v10, then that would greatly
> weaken the argument for providing aliases.

We changed the pg_xlog directory to be pg_wal, that's certainly going to
have an impact on monitoring and backup tools.

We've also made changes in, at least, what's reported in
pg_stat_activity and there's been discussions about changing it further
(list background workers or not, etc), and in pg_stat_replication (with
the addition of quorum-based sync rep).  In fact, these kinds of changes
are almost certainly going to require more work for tool authors to deal
with than just a simple function name change.

And that's only with a few minutes of looking at the commit log and I
don't doubt that there's more and that we're going to have even more
before feature freeze that a serious monitoring tool will have to be
updated for.

As long as we properly include these changes in the release notes for
tool authors, I really don't see any of them as a big deal.

Thanks!

Stephen

Andres,

* Andres Freund (andres@anarazel.de) wrote:
> On January 12, 2017 10:50:18 AM PST, Stephen Frost <sfrost@snowman.net> wrote:
> >* Andres Freund (andres@anarazel.de) wrote:
> >> On 2017-01-12 13:40:50 -0500, Stephen Frost wrote:
> >> > * Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> >> > > The way I see it, either one person can spend an hour or whatever
> >> > > creating an extension once, or every postgres install that's
> >using
> >> > > any of these functions now has yet another hurdle to upgrading.
> >> >
> >> > I just don't buy this argument, at all.  These functions names are
> >> > certainly not the only things we're changing with PG10 and serious
> >> > monitoring/backup/administration tools are almost certainly going
> >to
> >> > have quite a bit to adjust to with the new release, and that isn't
> >news
> >> > to anyone who works with PG.
> >>
> >> By that argument we can just do arbitrary backward incompat changes.
> >We
> >> should aspire to be better than we've been in the past, not use that
> >> past as an excuse for not even trying.
> >
> >When they're changes that are primairly going to affect
> >monitoring/backup/administration tools, yes, I do think we can make
> >just
> >about arbitrary backward-incompatible changes.
> >
> >As Robert mentioned, and I agree with, changing things which will
> >impact
> >regular application usage of PG is a different story and one we should
> >be more cautious about.
>
> I find it very hard to understand the justification for that, besides that it's strengthening external projects
providingmonitoring, backup, etc in a compatible way. 

That might be understandable if these were entirely arbitrary changes
(which isn't actually what I'm talking about above), but they aren't.
These changes aren't being made just for the sake of making them and are
not entirely arbitrary, as I've already pointed out up-thread.

The point I was making above is that the only reason to not make such
changes is if they really are entirely arbitrary, but I don't think
we'd even be having this discussion if that was the case or that we'd
be split about the question.  We already moved forward with the real
change here- pg_xlog -> pg_wal, it really astounds me that we're having
to argue about what is primairly just clean-up from that change, at
least in my view.

Also, frankly, I don't agree with the notion that this is seriously
going to "strengthen" external projects any more than the regular set of
changes that we're making as we move forward.  Changing these functions
certainly isn't likely to change pgbackrest or barman or check_postgres
uptake any more than the pg_xlog -> pg_wal rename is going to.  Maybe if
we *reverted* that change and then didn't make any other changes then
various hand-written tools would be able to work with PG10 without being
changed, but we might as well call it PG 9.6.2 then.

Thanks!

Stephen

On 1/12/17 1:40 PM, Stephen Frost wrote:
> I just don't buy this argument, at all.  These functions names are
> certainly not the only things we're changing with PG10 and serious
> monitoring/backup/administration tools are almost certainly going to
> have quite a bit to adjust to with the new release, and that isn't news
> to anyone who works with PG.

I in turn don't buy this argument. ;-)

I have checked a variety of WAL-related monitoring scripts,
graphing/trending scripts, switchover/failover scripts, and the like,
and of course they all make ample use of a variety of *xlog* functions,
but as far as I can tell, they don't care about the pg_xlog renaming and
would continue to work just fine if the functions were not renamed.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 1/12/17 1:50 PM, Stephen Frost wrote:
> When they're changes that are primairly going to affect
> monitoring/backup/administration tools, yes, I do think we can make just
> about arbitrary backward-incompatible changes.

I don't agree with that.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 1/12/17 2:22 PM, Stephen Frost wrote:
> The point I was making above is that the only reason to not make such
> changes is if they really are entirely arbitrary, but I don't think
> we'd even be having this discussion if that was the case or that we'd
> be split about the question.  We already moved forward with the real
> change here- pg_xlog -> pg_wal, it really astounds me that we're having
> to argue about what is primairly just clean-up from that change, at
> least in my view.

I don't agree with the reasoning.  The change of the directory name was
because some people erroneously delete the directory.  There is little
to no risk that people accidentally delete built-in functions.  It was
not agreed that because we rename the directory that all other uses of
"xlog" have to be removed as well.  All those subsequent changes need to
stand on their own.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

* Peter Eisentraut (peter.eisentraut@2ndquadrant.com) wrote:
> On 1/12/17 1:40 PM, Stephen Frost wrote:
> > I just don't buy this argument, at all.  These functions names are
> > certainly not the only things we're changing with PG10 and serious
> > monitoring/backup/administration tools are almost certainly going to
> > have quite a bit to adjust to with the new release, and that isn't news
> > to anyone who works with PG.
>
> I in turn don't buy this argument. ;-)
>
> I have checked a variety of WAL-related monitoring scripts,
> graphing/trending scripts, switchover/failover scripts, and the like,
> and of course they all make ample use of a variety of *xlog* functions,
> but as far as I can tell, they don't care about the pg_xlog renaming and
> would continue to work just fine if the functions were not renamed.

The point I was making was that serious montioring systems would have to
be changed and I stand by that.  Sure, there are simple scripts out
there that could, individually, continue to work fine using the old
function names (unless and until we change some behavior of those- which
I'm sure will end up happening at some point), but the argument being
presented is that we shouldn't make people have to change and the only
way to do that would be to revert the pg_xlog -> pg_wal change.

Anything beyond that and we're into arguing about how *much* we should
ask people to change, and for my 2c, that's barely even useful to
discuss because once they have to make any change at all then the
question will come up if it's "worth it" to move to the next version or
the complaint about PG being too difficult to live with will be raised.
I'm not terribly worried about such arguments because, put simply, PG10
is going to have enough awesome features that it's an easy question to
answer.

The argument being presented here is that users won't mind that we
changed the name of a critical directory for every PG system in
existence, but they're going to be upset that we change a few function
names to go along with that change.  Really?

Certainly, check_postgres is going to have to be changed to address this
and, unsurprisingly, it's already had to address a variety of major
version differences that have been introduced over the years.

Thanks!

Stephen

* Peter Eisentraut (peter.eisentraut@2ndquadrant.com) wrote:
> On 1/12/17 2:22 PM, Stephen Frost wrote:
> > The point I was making above is that the only reason to not make such
> > changes is if they really are entirely arbitrary, but I don't think
> > we'd even be having this discussion if that was the case or that we'd
> > be split about the question.  We already moved forward with the real
> > change here- pg_xlog -> pg_wal, it really astounds me that we're having
> > to argue about what is primairly just clean-up from that change, at
> > least in my view.
>
> I don't agree with the reasoning.  The change of the directory name was
> because some people erroneously delete the directory.  There is little
> to no risk that people accidentally delete built-in functions.  It was
> not agreed that because we rename the directory that all other uses of
> "xlog" have to be removed as well.  All those subsequent changes need to
> stand on their own.

The subsequent discussion about what to call the directory lead to a
pretty clear winner that a better name is "wal", which fits in pretty
nicely with things like "wal_level", "wal_sync_method", "wal_log_hints",
section 19.5 of our documentation.  Try finding 'xlog' or even
'transaction log' anywhere in our sample postgresql.conf, in fact.

Look at our docs too, we talk about WAL a *great* deal more than we talk
about 'xlog'.  Almost all utilization of 'xlog' that I see is due to
some function or program name which was derived from the directory name,
and half of those end up, confusingly, going back and forth between
"WAL" and "transaction log" (the description of pg_receivexlog is a
great example of this).

In the end, while I don't agree that they really need to, it seems
pretty clear to me that these changes can certainly stand on their own.
The only reason they were ever called 'xlog' was because of the
directory name and without that link, there's really no reason to keep
mis-calling these functions 'xlog' *except* this claim that we don't
want to break things, even though we're changing the directory name.

Thanks!

Stephen

On Thu, Jan 12, 2017 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Vladimir Rusinov <vrusinov@google.com> writes:
>> On Thu, Jan 12, 2017 at 4:57 PM, Euler Taveira <euler@timbira.com.br> wrote:
>>> As Robert suggested in the other email: extension to create old names.
>
>> I don't follow the reasoning for the extension. It seem to have downsides
>> of both alternatives combined: we still break people's code, and we add
>> even more maintenance burden than just keeping aliases.
>
> Yeah, I'm not terribly for the extension idea.  Robert cited the precedent
> of contrib/tsearch2, but I think the history of that is a good argument
> against this: tsearch2 is still there 9 years later and there's no
> indication that we'll ever get rid of it.  We can let things rot
> indefinitely in core too.  If we do ever agree to get rid of the aliases,
> stripping them out of pg_proc.h will not be any harder than removing
> a contrib directory would be.

From a technical point of view, that is pretty much true.  But from a
user perspective, I don't think it is.  If the old names live in an
extension, then they will not be there by default.  Tools and scripts
will break.  If that's a problem for a particular user, they an
install the extension to unbreak those things, but tool authors will
probably be motivated to upgrade their tools so that the extension is
no longer needed.  Therefore, if we eventually drop the extension,
there will probably be few users relying on it at that point.  On the
other hand, if we install aliases that are present in every install -
and that are not even optionally removable - a lot of people are not
going to bother using the new names at all.  They'll just keep on
using the old ones so that their tools work with both old and new
versions, and if we eventually rip the stuff out of pg_proc.h it will
cause almost as much howling as if we'd done it right at the
beginning.

In other words, adding an extension is the only option that I see
which will give tool authors a strong incentive to support the new
names without making it a hard requirement on day one.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas wrote:

> From a technical point of view, that is pretty much true.  But from a
> user perspective, I don't think it is.  If the old names live in an
> extension, then they will not be there by default.  Tools and scripts
> will break.  If that's a problem for a particular user, they an
> install the extension to unbreak those things, but tool authors will
> probably be motivated to upgrade their tools so that the extension is
> no longer needed.  Therefore, if we eventually drop the extension,
> there will probably be few users relying on it at that point.  On the
> other hand, if we install aliases that are present in every install -
> and that are not even optionally removable - a lot of people are not
> going to bother using the new names at all.  They'll just keep on
> using the old ones so that their tools work with both old and new
> versions, and if we eventually rip the stuff out of pg_proc.h it will
> cause almost as much howling as if we'd done it right at the
> beginning.
> 
> In other words, adding an extension is the only option that I see
> which will give tool authors a strong incentive to support the new
> names without making it a hard requirement on day one.

A compatibility extension sounds like a pretty decent idea to me,
considering this point.  I think breakage without a workaround will
cause people to defer migration more than they otherwise would; the
extension gives them an easy way forward while they set up their
internal code to the new reality, to test the other new things in pg10.

If we keep the extension forever, so what?  It's not like tsearch2
causes us too much of a maintenance pain.  It's been patched a few
times, but only to change things that wouldn't need changes in a
xlog->wal aliases-only module.  Anyway, I would mark the module as
deprecated with an intent of removing it in half a dozen releases or so.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sat, Jan 14, 2017 at 3:10 AM, Vladimir Rusinov <vrusinov@google.com> wrote:
> Attached are two new version of the patch: one keeps aliases, one don't.
> Also, remove stray reference to xlog function in one of the tests.
>
> I've lost vote count. Should we create a form to calculate which one of the
> patches should be commited?

If we do that, we should vote on all the "renaming" stuff, i.e., not only
function names but also program names like pg_receivexlog, directory names
like clog, option names like xlogdir option of initdb, return value names of
the functions like xlog_position in pg_create_physical_replication_slot, etc.

Regards,

-- 
Fujii Masao

On 1/13/17 10:18 AM, Stephen Frost wrote:
> The point I was making was that serious montioring systems would have to
> be changed and I stand by that.

I don't think my monitoring systems are any less serious than yours.

> Certainly, check_postgres is going to have to be changed to address this
> and, unsurprisingly, it's already had to address a variety of major
> version differences that have been introduced over the years.

check_postgres will not need to be changed except for the actions that
check the disk, which you don't need unless you are using archiving.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter,

* Peter Eisentraut (peter.eisentraut@2ndquadrant.com) wrote:
> On 1/13/17 10:18 AM, Stephen Frost wrote:
> > Certainly, check_postgres is going to have to be changed to address this
> > and, unsurprisingly, it's already had to address a variety of major
> > version differences that have been introduced over the years.
>
> check_postgres will not need to be changed except for the actions that
> check the disk, which you don't need unless you are using archiving.

That isn't really the point though, is it?  check_postgres will need to
be changed because there are actions which check the pg_xlog directory.
That'll cause a new release, which will be the "release that works with
PG10."

Perhaps if you're following along with -hackers and know how
check_postgres works then you'll realize that you might not *have* to
upgrade your check_postgres installation if only the directory is
changed and nothing else is, but I've got a pretty hard time seeing that
as a very common user use-case.

The implication here seems to be that because the older version of
check_postgres might appear to continue working for *some* set of
actions (but not all) that we should encouarge users to keep using that
older version with PG10.  That doesn't make any sense to me and I
certainly don't agree with it.

Thanks!

Stephen

On Mon, Jan 16, 2017 at 10:48 PM, Fujii Masao <masao.fujii@gmail.com> wrote:
> If we do that, we should vote on all the "renaming" stuff, i.e., not only
> function names but also program names like pg_receivexlog, directory names
> like clog, option names like xlogdir option of initdb, return value names of
> the functions like xlog_position in pg_create_physical_replication_slot, etc.

Right.  I think a lot of that stuff should also be changed.  If we
weren't OK with breaking compatibility, why'd we change pg_xlog ->
pg_wal?  If we're not willing to change other things to match, let's
revert that change and be done with it.  It's undeniable that there's
going to be some pain here and I'm not committed to incurring that
pain, but I don't really understand the the theory that saying that a
half-renaming will save us pain vs. a more through renaming.  If we
change some things and not others, people will have to try to remember
in which cases they now have to say xlog instead of wal and in which
cases they still have to say xlog and (perhaps) in which cases they
can choose either.  I think "xlog" is terrible naming; there is no
universe in which "x" is a reasonable short-hand for either
"transaction" or "write-ahead".

Q: OK, where is my WAL stored?
A: pg_wal
Q: How do I reset it?
A: pg_resetxlog
Q: Why is it called pg_resetxlog?
A: Because we call the transaction log "xlog".
Q: Evidently you don't.
A: Well, it used to be called pg_xlog prior to version 10, but then we
renamed it, but the tool has still got the old name.
Q: Are you kidding me?
A: No.
Q: Do you guys suck at picking names for things?
A: Yes.
Q: Wouldn't it at least be better to settle on ONE incomprehensible
abbreviation for any given concept instead of having TWO DIFFERENT
ONES?
A: Hey, look at the time.

If we don't make this all consistent, we'll be having that
conversation -- or some variant of it -- forever.  There will be
constant arguments about whether to give in and rename some more
things.  The people who are now saying that we shouldn't break
compatibility for this change (as if we haven't already) will be even
less happy when they keep having to argue against breaking
compatibility for the same thing a second time.

I'm OK with continuing to use "xlog" as the user-facing name for the
write-ahead log, and I am OK with switching to wal.  But leaving
things in the halfway in-between state where they are right now seems
like a mess.  It conveniences the people who happen to care about the
names of the parts we haven't renamed yet but not the part we already
did rename at the price of a permanent inconsistency in naming
conventions that can't ever eliminate.  Yuck.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

* Robert Haas (robertmhaas@gmail.com) wrote:
> I'm OK with continuing to use "xlog" as the user-facing name for the
> write-ahead log, and I am OK with switching to wal.  But leaving
> things in the halfway in-between state where they are right now seems
> like a mess.  It conveniences the people who happen to care about the
> names of the parts we haven't renamed yet but not the part we already
> did rename at the price of a permanent inconsistency in naming
> conventions that can't ever eliminate.  Yuck.

I'm more of a '-1' when it comes to keeping everything called "xlog",
and a '+1' on renaming everything to be WAL, since that's what we
usually refer to this thing as, but I agree with Robert that only doing
a half-way renaming is about a -1000.

Thanks!

Stephen

On Wed, Jan 18, 2017 at 8:15 PM, Vladimir Rusinov <vrusinov@google.com> wrote:
> On the topic of binaries, there's going to be another patch renaming them.
> Those will have no aliases as it's trivial to work-around (symlinks, shell
> scripts, etc) and not so trivial to implement in a portable way.

On Windows that would be a pain... So let's not use symlinks for binaries.

> I'm used to workflows with lots of small commits, so I was not able to force
> myself to include all of the renames in one diff.

That's a good habit IMO.
-- 
Michael

On Mon, Jan 23, 2017 at 2:06 PM, Venkata B Nagothi <nag1010@gmail.com> wrote:
>
> On Sat, Jan 14, 2017 at 5:10 AM, Vladimir Rusinov <vrusinov@google.com>
> wrote:
>>
>> Attached are two new version of the patch: one keeps aliases, one don't.
>
>
> Both the patches (with and without aliases) are not getting applied to the
> latest master. Below is the error -
>
> Perhaps you used the wrong -p or --strip option?
> The text leading up to this was:
> --------------------------
> |diff --git a/src/include/access/xlog_fn.h b/src/include/access/xlog_fn.h
> |index a013d047a2..a46b2f995c 100644
> |--- a/src/include/access/xlog_fn.h
> |+++ b/src/include/access/xlog_fn.h
> --------------------------
> File to patch:

xlog_fn.h is not necessary anymore as all the declarations of
functions defined in pg_proc.h are now generated automatically, so the
conflict is simple enough to resolve.
-- 
Michael

On 1/17/17 5:03 PM, Robert Haas wrote:
> Right.  I think a lot of that stuff should also be changed.  If we
> weren't OK with breaking compatibility, why'd we change pg_xlog ->
> pg_wal?  If we're not willing to change other things to match, let's
> revert that change and be done with it.

For the record, I don't like the name "xlog" either.  It would be nice
if we could have more consistent and intuitive naming.

But I don't see any proposals to actually change all uses of "xlog" to
"wal".  What about program names, command line options, etc.?  If the
argument is, we changed one thing, we should change the rest, then let's
see that.  I think that argument itself is flawed, but if that's what
we're going with, let's see the whole plan.

Moreover, I see we still have the pg_clog directory.  I thought that was
supposed to be renamed as well, to avoid confusing it with a "log"
directory.  Surely, we should at least conclude that original chapter
before going further.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Peter,

* Peter Eisentraut (peter.eisentraut@2ndquadrant.com) wrote:
> On 1/17/17 5:03 PM, Robert Haas wrote:
> > Right.  I think a lot of that stuff should also be changed.  If we
> > weren't OK with breaking compatibility, why'd we change pg_xlog ->
> > pg_wal?  If we're not willing to change other things to match, let's
> > revert that change and be done with it.
>
> For the record, I don't like the name "xlog" either.  It would be nice
> if we could have more consistent and intuitive naming.
>
> But I don't see any proposals to actually change all uses of "xlog" to
> "wal".  What about program names, command line options, etc.?  If the
> argument is, we changed one thing, we should change the rest, then let's
> see that.  I think that argument itself is flawed, but if that's what
> we're going with, let's see the whole plan.

That is the proposal and what Vladimir is working towards, as I
understand it, but one piece at a time rather than one big huge patch,
as he's already stated elsewhere on this thread.

I don't have any problem with asking for a summary of the exact set of
changes that he's planning to make though.  My understanding is that it
includes changing program names, command line options, etc.

> Moreover, I see we still have the pg_clog directory.  I thought that was
> supposed to be renamed as well, to avoid confusing it with a "log"
> directory.  Surely, we should at least conclude that original chapter
> before going further.

My understanding is that it is planned to be changed also, but it's
backed up behind the sudden hang-up with making progress on the xlog ->
WAL changes.

I agree that we could probably just go ahead and switch over to starting
on the clog changes (there was agreement somewhere about the new name
for that too), but, well, if I was someone watching all of this
discussion, I have to admit I probably wouldn't be too excited starting
on another set of name changes with all of this going on.  Admittedly,
the clog rename is a lot less user-facing and perhaps we should have
started with it, but this is where we're at now.

Thanks!

Stephen

On Tue, Jan 24, 2017 at 3:59 AM, Stephen Frost <sfrost@snowman.net> wrote:
> I agree that we could probably just go ahead and switch over to starting
> on the clog changes (there was agreement somewhere about the new name
> for that too), but, well, if I was someone watching all of this
> discussion, I have to admit I probably wouldn't be too excited starting
> on another set of name changes with all of this going on.  Admittedly,
> the clog rename is a lot less user-facing and perhaps we should have
> started with it, but this is where we're at now.

There are no SQL-level functions and no binaries using the naming clog
or subtrans, so that's a good anti-complain start.
-- 
Michael

* Vladimir Rusinov (vrusinov@google.com) wrote:
> On Mon, Jan 23, 2017 at 6:59 PM, Stephen Frost <sfrost@snowman.net> wrote:
> > I don't have any problem with asking for a summary of the exact set of
> > changes that he's planning to make though.  My understanding is that it
> > includes changing program names, command line options, etc.
>
> Here's what I currently have in mind:
>
> - sql function names (current patch)
> - binaries in bin: pg_receivexlog, pg_xlogdump, pg_resetxlog

Command-line options for other binaries in bin should also be changed
(eg: initdb's --xlogdir).

Looking at the other binaries in src/bin:

pg_archivecleanup - seems ok as-is (I'll note that it's Usage mentioned                   'OLDESTKEPTWALFILE' ...)
pg_basebackup - --xlog/--xlog-method/--xlogdir
pgbench - looks fine
pg_config - looks fine
pg_controldata - looks fine
pg_ctl - looks fine
pgevent - DLL, not a binary, looks ok
pg_rewind - looks fine
pg_test_fsync - looks fine
pg_test_timing - looks fine
pg_upgrade - looks fine
psql - looks fine

> - (maybe) public/exported symbols from libpq,
> e.g. libpqGetCurrentXlogInsertLocation

Hm..?  That function is just a local function in pg_rewind.  I'd be kind
of surprised if we actually had anything in libpq that needed changing
due to this and I don't see anything there in a quick look.

> - at this time I don't target internal function and filenames, but that
> could also be done if there's desire for this.

No, I don't think that'll be necessary.

Thanks!

Stephen

On Mon, Jan 23, 2017 at 1:55 PM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> For the record, I don't like the name "xlog" either.  It would be nice
> if we could have more consistent and intuitive naming.

Great!

> But I don't see any proposals to actually change all uses of "xlog" to
> "wal".  What about program names, command line options, etc.?  If the
> argument is, we changed one thing, we should change the rest, then let's
> see that.  I think that argument itself is flawed, but if that's what
> we're going with, let's see the whole plan.

I'm happy to go change every last bit of it.  I was expecting after I
committed the initial rename that somebody would provide a follow-on
patch to do the rest of it in short order.  Instead, months went by
and we still don't have a complete patch.  But I don't see why that
has to take more than a day's work, probably just a few hours.  I'd
like to do that and move on.

> Moreover, I see we still have the pg_clog directory.  I thought that was
> supposed to be renamed as well, to avoid confusing it with a "log"
> directory.  Surely, we should at least conclude that original chapter
> before going further.

I'm not excited about starting to change pg_clog before we finish with
xlog -> wal.  Then we just have two half-done things, IMO.  But I'm
also not the only one with a commit bit.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Tue, Jan 24, 2017 at 4:47 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> But I don't see any proposals to actually change all uses of "xlog" to
>> "wal".  What about program names, command line options, etc.?  If the
>> argument is, we changed one thing, we should change the rest, then let's
>> see that.  I think that argument itself is flawed, but if that's what
>> we're going with, let's see the whole plan.
>
> I'm happy to go change every last bit of it.  I was expecting after I
> committed the initial rename that somebody would provide a follow-on
> patch to do the rest of it in short order.  Instead, months went by
> and we still don't have a complete patch.  But I don't see why that
> has to take more than a day's work, probably just a few hours.  I'd
> like to do that and move on.

And here are patches for that.

0001 renames everything that contains "xlog" in pg_proc.h to refer to "wal".

0002 renames programs whose names contains "xlog".

0003 renames the dtrace probes whose names contain "xlog".

0004 renames command line options which contain "xlog".

There are probably a few more things that could be done afterwards to
clean up other odds and ends, but I think this gets the vast bulk of
the user-visible references to xlog.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Robert Haas wrote:

> 0002 renames programs whose names contains "xlog".

There have been complaints that pg_receivexlog's name is not consistent
with pg_recvlogical, and I seem to recall there were some votes for
renaming pg_receivexlog to match.  We could make it "pg_recvwal" now.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Thu, Jan 26, 2017 at 12:27 PM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Robert Haas wrote:
>
>> 0002 renames programs whose names contains "xlog".
>
> There have been complaints that pg_receivexlog's name is not consistent
> with pg_recvlogical, and I seem to recall there were some votes for
> renaming pg_receivexlog to match.  We could make it "pg_recvwal" now.

Or prw.  :-)

Personally, I generally push to avoid abbreviations like receive ->
recv.  Yeah, it's a few extra characters of typing, but you can use
tab completion so it's not really a big deal I think.  And the problem
with abbreviating things is that people then have to remember whether
it was abbreviated and in exactly which way.  "Wait, was it recv or
rcv this time?"  Moreover, some of the abbreviations we've used are
potentially opaque to non-English-speakers, or in some cases even to
English speakers.  So I would prefer not to go there.  But if the
consensus is otherwise, so be it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Jan 26, 2017 at 12:27 PM, Alvaro Herrera
> <alvherre@2ndquadrant.com> wrote:
>> There have been complaints that pg_receivexlog's name is not consistent
>> with pg_recvlogical, and I seem to recall there were some votes for
>> renaming pg_receivexlog to match.  We could make it "pg_recvwal" now.

> ... I would prefer not to go there.

I agree.  "pg_recvlogical" was a badly chosen name; let's not double
down on the error.

What I think might be worth considering is inserting underscores,
eg "pg_receive_wal", anywhere that we are running the abbreviation
directly against another word.  We won't get another chance.
        regards, tom lane

* Tom Lane (tgl@sss.pgh.pa.us) wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Thu, Jan 26, 2017 at 12:27 PM, Alvaro Herrera
> > <alvherre@2ndquadrant.com> wrote:
> >> There have been complaints that pg_receivexlog's name is not consistent
> >> with pg_recvlogical, and I seem to recall there were some votes for
> >> renaming pg_receivexlog to match.  We could make it "pg_recvwal" now.
>
> > ... I would prefer not to go there.
>
> I agree.  "pg_recvlogical" was a badly chosen name; let's not double
> down on the error.
>
> What I think might be worth considering is inserting underscores,
> eg "pg_receive_wal", anywhere that we are running the abbreviation
> directly against another word.  We won't get another chance.

Wouldn't that make it 'pg_recv_wal'?  Or were you referring to the 'wal'
as being the abbreviation?  Or did you mean underscores between words in
general?

If we want to switch to using underscores to seperate words, then that
implies a whole lot of *additional* renaming that I'm not sure I can
really get behind...

initdb
pg_archivecleanup
pg_basebackup
pgbench
pg_controldata
pgevent (eh, not really a binary, but it's in src/bin)
pg_dumpall
pg_isready

All of those, to me at least seem fine, and I'd further be fine with
"pg_receivewal".

I haven't got any great answers wrt pg_recvlogical, though I don't
particullarly like the existing name, but that's more because I have
to ask myself how the heck one can receive a "logical"..

Then again 'pg_controldata' only works if you realize it's really
"pg_print_pg_control" or something.  It's hardly controlling any data.

Thanks!

Stephen

On Thu, Jan 26, 2017 at 12:41 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Thu, Jan 26, 2017 at 12:27 PM, Alvaro Herrera
>> <alvherre@2ndquadrant.com> wrote:
>>> There have been complaints that pg_receivexlog's name is not consistent
>>> with pg_recvlogical, and I seem to recall there were some votes for
>>> renaming pg_receivexlog to match.  We could make it "pg_recvwal" now.
>
>> ... I would prefer not to go there.
>
> I agree.  "pg_recvlogical" was a badly chosen name; let's not double
> down on the error.
>
> What I think might be worth considering is inserting underscores,
> eg "pg_receive_wal", anywhere that we are running the abbreviation
> directly against another word.  We won't get another chance.

Yeah, I thought about that, too, but it doesn't really seem worth it.
If we had pg_receive_wal and pg_receive_logical, they'd be nicely
consistent with each other, but inconsistent with practically every
other utility we have: pg_basebackup, pg_archivecleanup,
pg_controldata, etc.  I'm not prepared to endorse renaming all of that
stuff just to add underscores, and frankly I don't think the style
pg_foobarbaz is really a problem.  It's a lot easier to remember "the
only underscore is after the initial pg" than it is to remember
exactly how each word was abbreviated in each context.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Stephen Frost <sfrost@snowman.net> writes:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> What I think might be worth considering is inserting underscores,
>> eg "pg_receive_wal", anywhere that we are running the abbreviation
>> directly against another word.  We won't get another chance.

> Wouldn't that make it 'pg_recv_wal'?  Or were you referring to the 'wal'
> as being the abbreviation?

The latter.  As far as the programs go, that would be

pg_receive_wal
pg_reset_wal
pg_wal_dump

The other cases you mention are, for the most part, words that we're
running together ("db" is the only exception) so they're not committing
double sins against readability.  Anyway, I'm not suggesting that we
should rename anything this patch isn't touching already.
        regards, tom lane

PS: I'm trying hard not to open the can of worms labeled "pg_dump_wal".

Robert Haas <robertmhaas@gmail.com> writes:
> Yeah, I thought about that, too, but it doesn't really seem worth it.
> If we had pg_receive_wal and pg_receive_logical, they'd be nicely
> consistent with each other, but inconsistent with practically every
> other utility we have: pg_basebackup, pg_archivecleanup,
> pg_controldata, etc.

Mmm, good point.  I was looking at pg_test_fsync and pg_test_timing,
but those are the only exceptions, and they're only quasi-user-facing
anyway.
        regards, tom lane

On 2017-01-24 16:47:29 -0500, Robert Haas wrote:
> I'm happy to go change every last bit of it.

I quite regret not aggressively opining against the renaming of pg_xlog
to pg_wal. I think the few users deleting their data don't weigh against
renaming a bunch of tools and function for some sense of
consistency. This will cause pain for a couple years - maybe it'll be
worth it 6-7 releases down the line, but I doubt before that.  But then
the original discussion was about renaming pg_xlog, not about a general
s/xlog/$something/.

- Andres

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-01-24 16:47:29 -0500, Robert Haas wrote:
> > I'm happy to go change every last bit of it.
>
> I quite regret not aggressively opining against the renaming of pg_xlog
> to pg_wal. I think the few users deleting their data don't weigh against
> renaming a bunch of tools and function for some sense of
> consistency. This will cause pain for a couple years - maybe it'll be
> worth it 6-7 releases down the line, but I doubt before that.  But then
> the original discussion was about renaming pg_xlog, not about a general
> s/xlog/$something/.

For my 2c, I'm actually quite happy that we're finally going to start
reducing the confusion between xlog and WAL.

Let's not forget to go update the documentation to be sensible and
consistent as we're making these mechanical changes to the program
names.

Thanks!

Stephen

On Thu, Jan 26, 2017 at 1:21 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-01-24 16:47:29 -0500, Robert Haas wrote:
>> I'm happy to go change every last bit of it.
>
> I quite regret not aggressively opining against the renaming of pg_xlog
> to pg_wal. I think the few users deleting their data don't weigh against
> renaming a bunch of tools and function for some sense of
> consistency. This will cause pain for a couple years - maybe it'll be
> worth it 6-7 releases down the line, but I doubt before that.  But then
> the original discussion was about renaming pg_xlog, not about a general
> s/xlog/$something/.

I completely understand that position.  I have always been doubtful of
the value of renaming pg_xlog to pg_wal, and I'm not any more
dedicated to the idea now than I was when I committed that patch.  But
there was overwhelming support for it, consensus on a level rarely
seen here.

I do not think it can be right to rename the directory and not
anything else.  I stand by what I wrote in

https://www.postgresql.org/message-id/CA+TgmobeHP2qbtMvYxG2x8Pm_9utjRya-rom5XL4QuyA26c1Gg@mail.gmail.com

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2017-01-26 14:05:43 -0500, Robert Haas wrote:
> I completely understand that position.  I have always been doubtful of
> the value of renaming pg_xlog to pg_wal, and I'm not any more
> dedicated to the idea now than I was when I committed that patch.  But
> there was overwhelming support for it, consensus on a level rarely
> seen here.

I think that consistency was based on the change being a narrow
proposition, not a license to run around and change a lot of stuff
including the names of binary.


> I do not think it can be right to rename the directory and not
> anything else.  I stand by what I wrote in
> 
> https://www.postgresql.org/message-id/CA+TgmobeHP2qbtMvYxG2x8Pm_9utjRya-rom5XL4QuyA26c1Gg@mail.gmail.com

I'm tempted to quote Emerson ;).  I don't think the naming of pg_xlog
vs. pg_wal doesn't actually have that large an impact, to change the
dynamics of the wal vs xlog dichotomy.  Sure it's nothing you'd do in a
new program, but neither is it very bad.

Andres

On 2017-01-26 12:23:24 -0700, David G. Johnston wrote:
> On Thu, Jan 26, 2017 at 12:13 PM, Andres Freund <andres@anarazel.de> wrote:
> 
> > On 2017-01-26 14:05:43 -0500, Robert Haas wrote:
> > > I completely understand that position.  I have always been doubtful of
> > > the value of renaming pg_xlog to pg_wal, and I'm not any more
> > > dedicated to the idea now than I was when I committed that patch.  But
> > > there was overwhelming support for it, consensus on a level rarely
> > > seen here.
> >
> > I think that consistency was based on the change being a narrow
> > proposition, not a license to run around and change a lot of stuff
> > including the names of binary.
> >
> >
> ​Whether the voters recognized that fact at the time I would have to concur
> that if we are going to change from xlog to wal we​ should be all-in.  If
> you want to vote to reject putting the whole camel in the tent I would say
> its a vote for reverting the change that put the camel's nose in there in
> the first place.

WTF.

On Thu, Jan 26, 2017 at 2:13 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-01-26 14:05:43 -0500, Robert Haas wrote:
>> I completely understand that position.  I have always been doubtful of
>> the value of renaming pg_xlog to pg_wal, and I'm not any more
>> dedicated to the idea now than I was when I committed that patch.  But
>> there was overwhelming support for it, consensus on a level rarely
>> seen here.
>
> I think that consistency was based on the change being a narrow
> proposition, not a license to run around and change a lot of stuff
> including the names of binary.

I'm not so sure about that.  There are a lot of people who have
supported the idea of making this consistent on THIS thread.  It's not
clear how much of a majority there is, but it's certainly no worse
than 50-50.  It's got far more affirmative votes than most patches
that go in though, I will grant, also far more negative votes.

>> I do not think it can be right to rename the directory and not
>> anything else.  I stand by what I wrote in
>>
>> https://www.postgresql.org/message-id/CA+TgmobeHP2qbtMvYxG2x8Pm_9utjRya-rom5XL4QuyA26c1Gg@mail.gmail.com
>
> I'm tempted to quote Emerson ;).  I don't think the naming of pg_xlog
> vs. pg_wal doesn't actually have that large an impact, to change the
> dynamics of the wal vs xlog dichotomy.  Sure it's nothing you'd do in a
> new program, but neither is it very bad.

Gee, I can't imagine what Emerson quote you might be thinking about.  :-)

I think it's just never going to work to imagine that we can
indefinitely go on having a pg_resetxlog binary to reset a thing that
is no longer called xlog.  Sure, for a few years that will seem like
it makes sense, but if we didn't change it now, eventually there would
be a push to do it later.  And if that one gets shut down, there will
eventually be another push.  We'll repeatedly relitigate this whole
debate, and maybe eventually the result will be one or two more things
get changed ... and then later we'll do it again for what's left.  I
am not entirely excited about the backward-compatibility pain we're
incurring here, but I think if we don't do it all at once it's just
going to get spread out over time.  Maybe it in a universe where
PostgreSQL was controlled by a small number of people acting as one
you could hold the line, but in the actual universe people just keep
calling a vote on this sort of thing every two or three until they
win.  I don't think there's anything you or I or anyone else can say
or do that will prevent that from happening, so I'd as soon just be
done with it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Jan 26, 2017 at 2:24 PM, Andres Freund <andres@anarazel.de> wrote:
>> Whether the voters recognized that fact at the time I would have to concur
>> that if we are going to change from xlog to wal we should be all-in.  If
>> you want to vote to reject putting the whole camel in the tent I would say
>> its a vote for reverting the change that put the camel's nose in there in
>> the first place.
>
> WTF.

I think that response is unwarranted.  I happen to agree entirely with
his position.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 2017-01-26 14:28:01 -0500, Robert Haas wrote:
> On Thu, Jan 26, 2017 at 2:24 PM, Andres Freund <andres@anarazel.de> wrote:
> >> Whether the voters recognized that fact at the time I would have to concur
> >> that if we are going to change from xlog to wal we should be all-in.  If
> >> you want to vote to reject putting the whole camel in the tent I would say
> >> its a vote for reverting the change that put the camel's nose in there in
> >> the first place.
> >
> > WTF.
> 
> I think that response is unwarranted.  I happen to agree entirely with
> his position.

I don't. Considering intent imo is important. David (and you?) is
basically saying "screw it, you voted for that person, you aren't
allowed to have an opinion anymore", and that's way outside of what I
consider acceptable.  So, because you think it doesn't make sense to
view renaming pg_xlog vs pg_wal as separate from a global s/xlog/wal/,
nobody else can have that position.  And on top of that David's
underlying that argument with a metaphor that basically implies the
other party is getting screwed over.  Sorry, that's not the way I want
decisions to be made here.

Andres

Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Jan 26, 2017 at 2:24 PM, Andres Freund <andres@anarazel.de> wrote:
>>> Whether the voters recognized that fact at the time I would have to concur
>>> that if we are going to change from xlog to wal we should be all-in.  If
>>> you want to vote to reject putting the whole camel in the tent I would say
>>> its a vote for reverting the change that put the camel's nose in there in
>>> the first place.

>> WTF.

> I think that response is unwarranted.  I happen to agree entirely with
> his position.

Me three.  If we're changing this at all, we should try to eliminate
"xlog" from user-visible names and docs altogether.  I was okay with the
proposal to have some redundant names in place for awhile, and I'm equally
okay with not having any; but not with being permanently schizophrenic
on the point.
        regards, tom lane

-Hackers,
From the field. I do not care what you chose, I care that:

1. It is consistent
2. It is readable/understandable
3. It is documented
4. It is done wholesale (because of usability)

That's it. So whatever meets that criteria, let's go for it. That may 
mean that certain commands look a little off but with the goal of 
consistency that is o.k.. It becomes explainable.

JD

-- 
Command Prompt, Inc.                  http://the.postgres.company/                        +1-503-667-4564
PostgreSQL Centered full stack support, consulting and development.
Everyone appreciates your honesty, until you are honest with them.

On Thu, Jan 26, 2017 at 2:37 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2017-01-26 14:28:01 -0500, Robert Haas wrote:
>> On Thu, Jan 26, 2017 at 2:24 PM, Andres Freund <andres@anarazel.de> wrote:
>> >> Whether the voters recognized that fact at the time I would have to concur
>> >> that if we are going to change from xlog to wal we should be all-in.  If
>> >> you want to vote to reject putting the whole camel in the tent I would say
>> >> its a vote for reverting the change that put the camel's nose in there in
>> >> the first place.
>> >
>> > WTF.
>>
>> I think that response is unwarranted.  I happen to agree entirely with
>> his position.
>
> I don't. Considering intent imo is important. David (and you?) is
> basically saying "screw it, you voted for that person, you aren't
> allowed to have an opinion anymore", and that's way outside of what I
> consider acceptable.  So, because you think it doesn't make sense to
> view renaming pg_xlog vs pg_wal as separate from a global s/xlog/wal/,
> nobody else can have that position.  And on top of that David's
> underlying that argument with a metaphor that basically implies the
> other party is getting screwed over.  Sorry, that's not the way I want
> decisions to be made here.

I'm not saying that people aren't allowed to have an opinion any more.
I'm saying that when somebody has an opinion that is different than
yours, you should politely disagree with it rather than saying "WTF",
which just as a reminder expands to "What The Fuck".  Frankly, I think
WTF is generally not a particularly useful contribution to most
discussions, but at the very least I think it should be used with some
kind of context.  Sending an email that says "WTF" and nothing else
conveys nothing other than that you don't respect the author of the
email to which you are replying, and David's response was not so
outlandish as to deserve that.  You might as well send an email that
says "go dire in a fire".

The substantive issue here is whether we should go forward with this
change, back out the change we already did, or leave things as they
are.  Tom, David, and I seem to be in lock step on at least the
following conclusion: halfway in between is bad.  So I have every
intention of continuing to push very hard for us to go either forward
or backward.  I hope to do that politely and respectfully, but I am
not prepared to give up on that basic point unless there are a WHOLE
LOT of contrary votes, and that is just not where we are at present.
While a variety of opinions have been expressed on the patch
originally posted and while many of those people took subtly different
positions which I'm unable to summarize concisely, there was ZERO
pushback against my email volunteering to go make all of this
consistent and pushing for it to be made consistent until you showed
up to complain.  Now there is one person objecting and several people
in favor, and I think it is pretty fair to say that many of the people
who were arguing for Vladimir's original patch are still in favor of
proceeding.  Even Peter, who wasn't super-excited about Vladimir's
patch in isolation, conceded that the xlog terminology sucked and
asked for a complete patch set.  I don't think he's unequivocally in
favor of this, but he seemed to be willing to sit still for it.  After
waiting two days for further opinions, I went and wrote the patches as
promised.

And I think that's all pretty reasonable.  I don't consider this a
done deal yet.  I don't consider your -1 irrelevant.  But I don't
think it's fair to present this as if I am somehow running roughshod
over community process, either.  If a large crew of people show up to
insist that we should rename only the directories and nothing else, I
guess I'll have to live with that.  But I think that's a bad decision
that will never survive the passage of time, and there seem to be
several people who agree with me.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

* Robert Haas (robertmhaas@gmail.com) wrote:
> And I think that's all pretty reasonable.  I don't consider this a
> done deal yet.  I don't consider your -1 irrelevant.  But I don't
> think it's fair to present this as if I am somehow running roughshod
> over community process, either.  If a large crew of people show up to
> insist that we should rename only the directories and nothing else, I
> guess I'll have to live with that.  But I think that's a bad decision
> that will never survive the passage of time, and there seem to be
> several people who agree with me.

+1 (to the above, and the rest of Robert's comments on that email).

For my 2c, I'm also in the crowd of forward or back, but don't leave it
half-done.  My preference is certainly to move forward with these
changes, but I understand the process and if there's enough agreement
that we just shouldn't make these changes, then we really shouldn't make
any of them.

Thanks!

Stephen

On 1/26/17 5:07 PM, Stephen Frost wrote:
> * Robert Haas (robertmhaas@gmail.com) wrote:
>> And I think that's all pretty reasonable.  I don't consider this a
>> done deal yet.  I don't consider your -1 irrelevant.  But I don't
>> think it's fair to present this as if I am somehow running roughshod
>> over community process, either.  If a large crew of people show up to
>> insist that we should rename only the directories and nothing else, I
>> guess I'll have to live with that.  But I think that's a bad decision
>> that will never survive the passage of time, and there seem to be
>> several people who agree with me.
>
> +1 (to the above, and the rest of Robert's comments on that email).
>
> For my 2c, I'm also in the crowd of forward or back, but don't leave it
> half-done.  My preference is certainly to move forward with these
> changes, but I understand the process and if there's enough agreement
> that we just shouldn't make these changes, then we really shouldn't make
> any of them.

+1 for forward or back.  I prefer forward but would take back over the
state we're in now.

--
-David
david@pgmasters.net

On 2017-01-26 16:55:37 -0500, Robert Haas wrote:
> On Thu, Jan 26, 2017 at 2:37 PM, Andres Freund <andres@anarazel.de> wrote:
> > On 2017-01-26 14:28:01 -0500, Robert Haas wrote:
> >> On Thu, Jan 26, 2017 at 2:24 PM, Andres Freund <andres@anarazel.de> wrote:
> >> >> Whether the voters recognized that fact at the time I would have to concur
> >> >> that if we are going to change from xlog to wal we should be all-in.  If
> >> >> you want to vote to reject putting the whole camel in the tent I would say
> >> >> its a vote for reverting the change that put the camel's nose in there in
> >> >> the first place.
> >> >
> >> > WTF.
> >>
> >> I think that response is unwarranted.  I happen to agree entirely with
> >> his position.
> >
> > I don't. Considering intent imo is important. David (and you?) is
> > basically saying "screw it, you voted for that person, you aren't
> > allowed to have an opinion anymore", and that's way outside of what I
> > consider acceptable.  So, because you think it doesn't make sense to
> > view renaming pg_xlog vs pg_wal as separate from a global s/xlog/wal/,
> > nobody else can have that position.  And on top of that David's
> > underlying that argument with a metaphor that basically implies the
> > other party is getting screwed over.  Sorry, that's not the way I want
> > decisions to be made here.
>
> I'm not saying that people aren't allowed to have an opinion any more.
> I'm saying that when somebody has an opinion that is different than
> yours, you should politely disagree with it rather than saying "WTF",
> which just as a reminder expands to "What The Fuck".  Frankly, I think
> WTF is generally not a particularly useful contribution to most
> discussions, but at the very least I think it should be used with some
> kind of context.  Sending an email that says "WTF" and nothing else
> conveys nothing other than that you don't respect the author of the
> email to which you are replying, and David's response was not so
> outlandish as to deserve that.  You might as well send an email that
> says "go dire in a fire".

The WTF wasn't about David wanting to go all in or not, but the way he
framed the general discussion.  And I do find it outlandish enough to
deserve that.  To me, especially with that methaphor and link, it still
reads as "you voted for it, even if phrased a lot more narrowly, so you
now get to eat all of it". *Especially* as it's a reply to me saying
that I'm concerned about my tepid yes to s/pg_xlog/pg_wal/ being used
for a larger change.

Using the normal ~concensus type of process for a mildly controversial
breaking change is fine with me.  I *personally* don't think it's worth
changing all this without taking more care about backward compat than
we're apparently willing to do.  I'm ok with loosing that argument.  I
just don't think the previous concensus for a narrower change can be
used for the wider one.

Andres

On 2017-01-26 15:45:15 -0700, David G. Johnston wrote:
> On Thu, Jan 26, 2017 at 3:28 PM, Andres Freund <andres@anarazel.de> wrote:
>
> > I *personally* don't think it's worth
> > changing all this without taking more care about backward compat than
> > we're apparently willing to do.  I'm ok with loosing that argument.  I
> > just don't think the previous concensus for a narrower change can be
> > used for the wider one
> > ​.
> >
>
> ​It would probably help others if you could brain dump a bit as to the
> benefit of the status-quo compared to both the 9.6 behavior and the all-in
> one.

Well, I think I've lost that argument, so I'm not sure how much effort
we should spent on litigating my opinion on it.  But here we go:

As I said somewhere upthread (when discussing the function names), I
think we're playing it too fast and loose with unnecessary changes.

We decided s/pg_xlog/pg_wal/ was necessary because people lost their
data, and we couldn't come up with a reasonable way to change it without
the name. The tradeoff is dataloss vs. dealing with directory renaming
in a few lowlevel tools.  So we decided to change the name.  It seems
breakage was unavoidable.

For me the renaming of functions, binaries, etc, is different because
the benefit is long-term avoidance of a bit of confusion, with the
shorter term price being tool breakages and confusion because
documentation/guides/... for different versions don't apply anymore, and
the search terms don't help anymore.  But we're still changing this,
even though breakage seems avoidable.


And that fits into the bigger topic of us doing minor cleanups without a
lot of concern for backward compatibility, e.g. like us whacking things
around in pg_stat_activity for most of the last releases - nearly all of
those could have been done in a compatible manner, without even smelling
that badly.

I hear these complaints about postgres most frequently: 1) replication
sucks. 2) way too slow on analytics queries. 3) existing admin tools
suck. 4) self written admin tools (required due to 3)) constantly break.

There's a lot being done on 1) and 2). There's very little in-core
progress about 3). We're getting worse on 4).


> I actually took your response as: "why the f**k is he talking about
> camels?" and started laughing...

That's good then, and let's raise a $beverage_of_choice tonight to the
fallability of e-mail conversation ;)


Andres

Andres,

* Andres Freund (andres@anarazel.de) wrote:
> I hear these complaints about postgres most frequently: 1) replication
> sucks. 2) way too slow on analytics queries. 3) existing admin tools
> suck. 4) self written admin tools (required due to 3)) constantly break.
>
> There's a lot being done on 1) and 2). There's very little in-core
> progress about 3). We're getting worse on 4).

I certainly hear some of these complaints also, and I'd love it if '3'
were something that the project was focused on, but, well, I really
don't see pgAdmin ever being in core, and 99% of the time that I'm
talking to end users, that's really what they're looking for (or, well,
something like it).  I don't recall, off-hand at least, ever running
into a user complaining that their pgAdmin-like admin tool broke.

I have doubts that we could even get something like check_postgres or
another monitoring type of tool into core, though it would be sooooo
nice to have tight integration between montioring and the core code.
That would require some kind of pluggable system which could support the
montioring-of-the-day and an easy way to extend it and a way to set up
what should be monitored and what shouldn't along with something to
actually gather that info while the system is running (eg: a background
worker of some kind), or at least, those are my general thoughts.

Maybe we'll get to the point where we'd have a fully-baked, featureful
backup solution that works with large-scale systems in core; I've at
least got some hopes of that happening, but not any particular timeline
or expectation of it happening soon.

Having a good way to do logging of different levels to different
locations would also be really nice, because our current logging
situation is really rather terrible.

> > I actually took your response as: "why the f**k is he talking about
> > camels?" and started laughing...
>
> That's good then, and let's raise a $beverage_of_choice tonight to the
> fallability of e-mail conversation ;)

Hah, now that I can certainly agree with..

Thanks!

Stephen

On 2017-01-26 19:01:54 -0500, Stephen Frost wrote:
> Andres,
> 
> * Andres Freund (andres@anarazel.de) wrote:
> > I hear these complaints about postgres most frequently: 1) replication
> > sucks. 2) way too slow on analytics queries. 3) existing admin tools
> > suck. 4) self written admin tools (required due to 3)) constantly break.
> > 
> > There's a lot being done on 1) and 2). There's very little in-core
> > progress about 3). We're getting worse on 4).
> 
> I certainly hear some of these complaints also, and I'd love it if '3'
> were something that the project was focused on, but, well, I really
> don't see pgAdmin ever being in core, and 99% of the time that I'm
> talking to end users, that's really what they're looking for (or, well,
> something like it).  I don't recall, off-hand at least, ever running
> into a user complaining that their pgAdmin-like admin tool broke.

For me it's it's not a pgadmin alike (or rather a more featureful
version) that's breaking the camel's back.  By far the biggest one I
hear is auto-HA.  There's no in-core support, and pretty much all of the
stuff out there is immature as hell. And even if it weren't people for
good reason don't trust such core aspects to $random_project.  Good,
easy to set up, monitoring is another thing - and I don't thing
check_postgres is a significant portion of that, it doesn't have much in
the way of guidelines, and actually pointing people to where problems
are.

Greetings,

Andres Freund

Andres,

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-01-26 19:01:54 -0500, Stephen Frost wrote:
> > * Andres Freund (andres@anarazel.de) wrote:
> > > I hear these complaints about postgres most frequently: 1) replication
> > > sucks. 2) way too slow on analytics queries. 3) existing admin tools
> > > suck. 4) self written admin tools (required due to 3)) constantly break.
> > >
> > > There's a lot being done on 1) and 2). There's very little in-core
> > > progress about 3). We're getting worse on 4).
> >
> > I certainly hear some of these complaints also, and I'd love it if '3'
> > were something that the project was focused on, but, well, I really
> > don't see pgAdmin ever being in core, and 99% of the time that I'm
> > talking to end users, that's really what they're looking for (or, well,
> > something like it).  I don't recall, off-hand at least, ever running
> > into a user complaining that their pgAdmin-like admin tool broke.
>
> For me it's it's not a pgadmin alike (or rather a more featureful
> version) that's breaking the camel's back.  By far the biggest one I
> hear is auto-HA.  There's no in-core support, and pretty much all of the
> stuff out there is immature as hell.

Oh, that's not what I tend to think of as 'admin tool'.  I completely
agree with you on #1 and also that we need something better for HA.

> And even if it weren't people for
> good reason don't trust such core aspects to $random_project.  Good,
> easy to set up, monitoring is another thing - and I don't thing
> check_postgres is a significant portion of that, it doesn't have much in
> the way of guidelines, and actually pointing people to where problems
> are.

My comment with check_postgres was more of a 'it seems unlikely we would
get in something as simple as this', not to suggest in any way that it's
the end-all, be-all of monitoring.

And now to, unashamedly, push my own agenda- any ideas about something
that would help with any of this that could be accomplished over a
summer by a student?  Even if you don't want to mentor, having a good
description, what skills are needed, the difficulty level, and the
expected outcomes might allow someone else to mentor...

Thanks!

Stephen

On Fri, Jan 27, 2017 at 2:21 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Tue, Jan 24, 2017 at 4:47 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>>> But I don't see any proposals to actually change all uses of "xlog" to
>>> "wal".  What about program names, command line options, etc.?  If the
>>> argument is, we changed one thing, we should change the rest, then let's
>>> see that.  I think that argument itself is flawed, but if that's what
>>> we're going with, let's see the whole plan.
>>
>> I'm happy to go change every last bit of it.  I was expecting after I
>> committed the initial rename that somebody would provide a follow-on
>> patch to do the rest of it in short order.  Instead, months went by
>> and we still don't have a complete patch.  But I don't see why that
>> has to take more than a day's work, probably just a few hours.  I'd
>> like to do that and move on.
>
> And here are patches for that.
> 0001 renames everything that contains "xlog" in pg_proc.h to refer to "wal".
> 0002 renames programs whose names contains "xlog".
> 0003 renames the dtrace probes whose names contain "xlog".
> 0004 renames command line options which contain "xlog".
>
> There are probably a few more things that could be done afterwards to
> clean up other odds and ends, but I think this gets the vast bulk of
> the user-visible references to xlog.

Thanks for doing this work. Jumping on the train, here is a review for
those patches. All of them are very mechanical changes. Just:
-<!ENTITY pgxlogdump         SYSTEM "pg_xlogdump.sgml">
+<!ENTITY pgwaldump         SYSTEM "pg_waldump.sgml"><!ENTITY postgres           SYSTEM "postgres-ref.sgml">
A nit about the number of spaces here.

In protocol.sgml:     <varlistentry>     <term>      <literal>xlogpos</literal> (<type>text</type>)     </term>
<listitem>    <para>      Current xlog flush location. Useful to get a known location in the      transaction log where
streamingcan start.     </para>     </listitem>     </varlistentry>
 
You want to say WAL here instead of xlog.

In storage.sgml, similar thing: <row>  <entry>pd_lsn</entry>  <entry>PageXLogRecPtr</entry>  <entry>8 bytes</entry>
<entry>LSN:next byte after last byte of xlog record for last change  to this page</entry> </row>
 
s/xlog/WAL/

In pg_standby.c:
pg_standby.c:   printf("  %s [OPTION]... ARCHIVELOCATION NEXTWALFILE
XLOGFILEPATH [RESTARTWALFILE]\n", progname);
s/XLOGFILEPATH/WALFILEPATH/.

All the other references to "xlog" are in the code comments, for the
user-facing changes your set of patches is enough.
-- 
Michael

On 1/26/17 2:05 PM, Robert Haas wrote:
> I do not think it can be right to rename the directory and not
> anything else.

I think this is the root of the confusion.

A lot of people apparently consented to renaming pg_xlog with the
understanding that that's it, whereas other people understood it as
consensus for renaming everything.

You really have (at least) three options here:

1. Rename nothing
2. Rename directory only
3. Rename everything

and you need to count the votes of each pair separately.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 1/26/17 1:10 PM, Tom Lane wrote:
> pg_reset_wal

It's really more pg_reset_controldata, isn't it?

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On 1/24/17 4:47 PM, Robert Haas wrote:
> I'm not excited about starting to change pg_clog before we finish with
> xlog -> wal.  Then we just have two half-done things, IMO.  But I'm
> also not the only one with a commit bit.

I think that depends on which way you slice the overall mission.  You
appear to think the mission is to have everything named consistently.  I
thought the mission was to have no directories named *log*.

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

On Sat, Jan 28, 2017 at 8:06 AM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 1/24/17 4:47 PM, Robert Haas wrote:
>> I'm not excited about starting to change pg_clog before we finish with
>> xlog -> wal.  Then we just have two half-done things, IMO.  But I'm
>> also not the only one with a commit bit.
>
> I think that depends on which way you slice the overall mission.  You
> appear to think the mission is to have everything named consistently.  I
> thought the mission was to have no directories named *log*.

Those are two different missions. Deciding what to do for the renaming
of pg_xlog does not prevent pg_clog and pg_subtrans to be renamed as
no functions and no binaries use those terms.
-- 
Michael

On 2017-01-27 18:06:11 -0500, Peter Eisentraut wrote:
> On 1/24/17 4:47 PM, Robert Haas wrote:
> > I'm not excited about starting to change pg_clog before we finish with
> > xlog -> wal.  Then we just have two half-done things, IMO.  But I'm
> > also not the only one with a commit bit.
> 
> I think that depends on which way you slice the overall mission.  You
> appear to think the mission is to have everything named consistently.  I
> thought the mission was to have no directories named *log*.

+1

On Sat, Jan 28, 2017 at 8:03 AM, Peter Eisentraut
<peter.eisentraut@2ndquadrant.com> wrote:
> On 1/26/17 2:05 PM, Robert Haas wrote:
>> I do not think it can be right to rename the directory and not
>> anything else.
>
> I think this is the root of the confusion.
>
> A lot of people apparently consented to renaming pg_xlog with the
> understanding that that's it, whereas other people understood it as
> consensus for renaming everything.
>
> You really have (at least) three options here:
>
> 1. Rename nothing
> 2. Rename directory only
> 3. Rename everything
>
> and you need to count the votes of each pair separately.

4. Rename everything with aliases.

I would vote for 4., or 3. for consistency if 4. is not an eligible choice.
-- 
Michael

On 1/27/17 6:47 PM, Michael Paquier wrote:
> On Sat, Jan 28, 2017 at 8:03 AM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
>> On 1/26/17 2:05 PM, Robert Haas wrote:
>>> I do not think it can be right to rename the directory and not
>>> anything else.
>>
>> I think this is the root of the confusion.
>>
>> A lot of people apparently consented to renaming pg_xlog with the
>> understanding that that's it, whereas other people understood it as
>> consensus for renaming everything.
>>
>> You really have (at least) three options here:
>>
>> 1. Rename nothing
>> 2. Rename directory only
>> 3. Rename everything
>>
>> and you need to count the votes of each pair separately.
>
> 4. Rename everything with aliases.
>
> I would vote for 4., or 3. for consistency if 4. is not an eligible choice.

I vote for 3.

The problem I have with aliases is that they would need to be done 
across the board.  At the least, we would need function aliases, 
symlinks for the binaries (which would rneed to be done by the 
packagers), aliases for the command line options, and lots of notations 
in the documentation.

All of this would need to be preserved more or less indefinitely, 
because if we can't decide on a break now it's not likely to happen later.

-- 
-David
david@pgmasters.net

On Sat, Jan 28, 2017 at 12:43 PM, David Steele <david@pgmasters.net> wrote:
> The problem I have with aliases is that they would need to be done across
> the board.  At the least, we would need function aliases, symlinks for the
> binaries (which would rneed to be done by the packagers), aliases for the
> command line options, and lots of notations in the documentation.

I was not precise enough here, I am fine with only the function
aliases, because they just have as cost an extra line in pg_proc.h and
because binary-level aliases on Windows are a pain.

> All of this would need to be preserved more or less indefinitely, because if
> we can't decide on a break now it's not likely to happen later.

That's true as well.
-- 
Michael

On 2017-01-28 08:47:03 +0900, Michael Paquier wrote:
> On Sat, Jan 28, 2017 at 8:03 AM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
> > On 1/26/17 2:05 PM, Robert Haas wrote:
> >> I do not think it can be right to rename the directory and not
> >> anything else.
> >
> > I think this is the root of the confusion.
> >
> > A lot of people apparently consented to renaming pg_xlog with the
> > understanding that that's it, whereas other people understood it as
> > consensus for renaming everything.
> >
> > You really have (at least) three options here:
> >
> > 1. Rename nothing
> > 2. Rename directory only
> > 3. Rename everything
> >
> > and you need to count the votes of each pair separately.
> 
> 4. Rename everything with aliases.
> 
> I would vote for 4., or 3. for consistency if 4. is not an eligible choice.

2, 4, 1, 3

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-01-28 08:47:03 +0900, Michael Paquier wrote:
> > On Sat, Jan 28, 2017 at 8:03 AM, Peter Eisentraut
> > <peter.eisentraut@2ndquadrant.com> wrote:
> > > On 1/26/17 2:05 PM, Robert Haas wrote:
> > >> I do not think it can be right to rename the directory and not
> > >> anything else.
> > >
> > > I think this is the root of the confusion.
> > >
> > > A lot of people apparently consented to renaming pg_xlog with the
> > > understanding that that's it, whereas other people understood it as
> > > consensus for renaming everything.
> > >
> > > You really have (at least) three options here:
> > >
> > > 1. Rename nothing
> > > 2. Rename directory only
> > > 3. Rename everything
> > >
> > > and you need to count the votes of each pair separately.
> >
> > 4. Rename everything with aliases.
> >
> > I would vote for 4., or 3. for consistency if 4. is not an eligible choice.
>
> 2, 4, 1, 3

For me, it's: 3, 4, 1, 2.

I tend to agree with the other discussion, which points out that aliases
for everything isn't really practical..

Of course, I'd also like to drop things like pg_user and pg_shadow one
day...  Never thought they'd last anywhere near this long when I first
wrote those views, 12 years ago...

Thanks!

Stephen

On Mon, Jan 30, 2017 at 8:01 PM, Vladimir Rusinov <vrusinov@google.com> wrote:
>
> On Fri, Jan 27, 2017 at 11:03 PM, Peter Eisentraut
> <peter.eisentraut@2ndquadrant.com> wrote:
>>
>> 1. Rename nothing
>> 2. Rename directory only
>> 3. Rename everything
>
>
> 3 or 1 (with a slight preference for 3).
>
> Not sure if my vote counts, but for me as ex-DBA consistency mattered a lot.

Any voice counts :)

> This is one of the reasons PostgreSQL is actually nicer to work with than
> many other databases. I remember 'wal' vs 'xlog' was actually one of a very
> few confusing things. I think it is important to stick to just one term,
> whether it be wal or xlog. I'd prefer wal since it's a term used in many
> other systems. 'xlog' may be confused with likes of binary log in MySQL
> which is not what it is.

The first time I worked on Postgres, I was not able to do the mapping
between both terms as well.

> Now, I've rebased my patches to rename functions (with and without aliases)
> on top of current master, but there's probably no point of posting them now
> given there are patches above that rename other things as well.

OK, I have moved this patch to CF 2017-03, let's make the discussion
continue with more votes coming in.
-- 
Michael

    Peter Eisentraut wrote:

> You really have (at least) three options here:
>
> 1. Rename nothing
> 2. Rename directory only
> 3. Rename everything

I vote for 1) as I believe the renaming will create more confusion
than it's worth, not even considering the renaming of functions
and views.

What if we look at the change from the pessimistic angle?
An example of confusion that the change would create:
a lot of users currently choose pg_wal for the destination
directory of their archive command. Less-informed users
that set up archiving and/or log shipping in PG10 based on
advice online from previous versions will be fairly
confused about the missing pg_xlog, and the fact that the
pg_wal directory they're supposed to create already exists.

Also googling for pg_wal, I'm finding food for thought like this
IBM technote:
http://www-01.ibm.com/support/docview.wss?uid=isg3T1015637
which recommends to
"Remove all files under /var/lib/pgsql/9.0/data/pg_wal/"
and also calls that directory the "write-ahead log directory"
which is quite confusing because apparently it's the destination of
their archive command.

This brings the question: what about the people who will delete
their pg_wal (ex-pg_xlog) when they face a disk-full condition and
they mix up in their mind the archive directory and the WAL directory?
It's hard to guess how many could make that mistake but I don't see
why it would be less probable than confusing pg_xlog and pg_log.
At least the contents of the latter directories look totally
different, contrary to the wal directory versus wal archive.

Also, what about the users who are helped currently by
the fact that "pg_xlog" is associated at the top of google results
to good articles that explain what it is, why it fills up and what
to do about it? By burying the name "pg_xlog" we're also loosing that
connection, and in the worst case making worse the problem we
wanted to help with.

There's also the disruption in existing backup scripts that directly
reference pg_xlog. Obviously these scripts are critical, and there's
something weird in breaking that intentionally. The motivation of the
breakage is likely to be felt as frivolous and unfair to those people who
are adversely impacted by the change, even if the part of not
reading the release notes is their fault.


Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite

On Fri, Feb 3, 2017 at 8:50 AM, Daniel Verite <daniel@manitou-mail.org> wrote:
> What if we look at the change from the pessimistic angle?
> An example of confusion that the change would create:
> a lot of users currently choose pg_wal for the destination
> directory of their archive command.

Really? I find that surprising to say "a lot". Perhaps there are some,
but first I would not suspect that there are many people to use this
repository name, and even less crazy enough to store them directly in
PGDATA itself. And of course that's not on a different partition :)

> Also googling for pg_wal, I'm finding food for thought like this
> IBM technote:
> http://www-01.ibm.com/support/docview.wss?uid=isg3T1015637
> which recommends to
> "Remove all files under /var/lib/pgsql/9.0/data/pg_wal/"
> and also calls that directory the "write-ahead log directory"
> which is quite confusing because apparently it's the destination of
> their archive command.

Well this product of IBM is one.

> There's also the disruption in existing backup scripts that directly
> reference pg_xlog. Obviously these scripts are critical, and there's
> something weird in breaking that intentionally. The motivation of the
> breakage is likely to be felt as frivolous and unfair to those people who
> are adversely impacted by the change, even if the part of not
> reading the release notes is their fault.

Those are not complicated to fix because they are hard failures.
Sufficient tests need to be done so as backup scripts don't show in
red on live systems before deploying them. The original reason to do
the rename is that there are folks as well thinking that removing
pg_xlog is fine on full disk because, you know, it contains just
*logs* so they are not critical for the system.
-- 
Michael

Daniel,

* Daniel Verite (daniel@manitou-mail.org) wrote:
> What if we look at the change from the pessimistic angle?
> An example of confusion that the change would create:
> a lot of users currently choose pg_wal for the destination
> directory of their archive command. Less-informed users
> that set up archiving and/or log shipping in PG10 based on
> advice online from previous versions will be fairly
> confused about the missing pg_xlog, and the fact that the
> pg_wal directory they're supposed to create already exists.

One would hope that they would realize that's not going to work
when they set up PG10.  If they aren't paying attention sufficient
to realize that then it seems entirely likely that they would feel
equally safe removing the contents of a directory named 'pg_xlog'.

> Also googling for pg_wal, I'm finding food for thought like this
> IBM technote:
> http://www-01.ibm.com/support/docview.wss?uid=isg3T1015637
> which recommends to
> "Remove all files under /var/lib/pgsql/9.0/data/pg_wal/"
> and also calls that directory the "write-ahead log directory"
> which is quite confusing because apparently it's the destination of
> their archive command.

It's certainly unfortunate that people have thought that they can create
arbitrary directories under the PG data directory.  That's never going
to be safe, witness that we've created new directories under PGDATA in
the last few releases and I don't see any reason why that would change
moving forward.  Perhaps we should check for the existance of such a
directory during pg_upgrade and throw an error, and we should go back
and do the same for other directories which have been added over
releases, but I'm not sure I can see an argument for doing much more
than that.

> This brings the question: what about the people who will delete
> their pg_wal (ex-pg_xlog) when they face a disk-full condition and
> they mix up in their mind the archive directory and the WAL directory?

In my experience, at least, that's less likely to happen than someone
nuke'ing pg_xlog.

> It's hard to guess how many could make that mistake but I don't see
> why it would be less probable than confusing pg_xlog and pg_log.

Based on my experience, at least, that seems quite a bit less likely to
happen.

> At least the contents of the latter directories look totally
> different, contrary to the wal directory versus wal archive.

Sadly, that doesn't seem to have always made much of a difference.

> Also, what about the users who are helped currently by
> the fact that "pg_xlog" is associated at the top of google results
> to good articles that explain what it is, why it fills up and what
> to do about it? By burying the name "pg_xlog" we're also loosing that
> connection, and in the worst case making worse the problem we
> wanted to help with.

We'll have new articles about pg_wal which will be picked up by Google
also.  I'm not quite sure how we would be making the problem worse
though- previously when it filled, a group of users believed it to be a
log directory which could be wiped out.  With the new name, it's at
least clearer that it's not simple a directory of not-necessary log
files.

> There's also the disruption in existing backup scripts that directly
> reference pg_xlog. Obviously these scripts are critical, and there's
> something weird in breaking that intentionally. The motivation of the
> breakage is likely to be felt as frivolous and unfair to those people who
> are adversely impacted by the change, even if the part of not
> reading the release notes is their fault.

While these scripts are critical, they're also likely to not be safe,
which is the unfortunate reality.  If they're well written then they'll
break cleanly, quickly, and be easily fixed.

We will break things across major version upgrades.  That's part of
the reason that we have major version upgrades, and why we support
released versions for years with bug fixes.

Thanks!

Stephen

On Fri, Feb 3, 2017 at 5:21 AM, Stephen Frost <sfrost@snowman.net> wrote:
> Daniel,
>
> * Daniel Verite (daniel@manitou-mail.org) wrote:
>> What if we look at the change from the pessimistic angle?
>> An example of confusion that the change would create:
>> a lot of users currently choose pg_wal for the destination
>> directory of their archive command. Less-informed users
>> that set up archiving and/or log shipping in PG10 based on
>> advice online from previous versions will be fairly
>> confused about the missing pg_xlog, and the fact that the
>> pg_wal directory they're supposed to create already exists.
>
> One would hope that they would realize that's not going to work
> when they set up PG10.  If they aren't paying attention sufficient
> to realize that then it seems entirely likely that they would feel
> equally safe removing the contents of a directory named 'pg_xlog'.

So... somebody want to tally up the votes here?

And... was this discussed at the FOSDEM developer meeting?

(Please say yes.)

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Thu, Jan 26, 2017 at 3:55 PM, Robert Haas <robertmhaas@gmail.com> wrote:

> The substantive issue here is whether we should go forward with this
> change, back out the change we already did, or leave things as they
> are.  Tom, David, and I seem to be in lock step on at least the
> following conclusion: halfway in between is bad.

I agree.

> So I have every
> intention of continuing to push very hard for us to go either forward
> or backward.

+1

Given the number of times I've known of people deleting files from
pg_xlog because the name made the contents seem unimportant, I
think finishing the job is better.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Sat, Feb 4, 2017 at 6:39 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Fri, Feb 3, 2017 at 5:21 AM, Stephen Frost <sfrost@snowman.net> wrote:
>> Daniel,
>>
>> * Daniel Verite (daniel@manitou-mail.org) wrote:
>>> What if we look at the change from the pessimistic angle?
>>> An example of confusion that the change would create:
>>> a lot of users currently choose pg_wal for the destination
>>> directory of their archive command. Less-informed users
>>> that set up archiving and/or log shipping in PG10 based on
>>> advice online from previous versions will be fairly
>>> confused about the missing pg_xlog, and the fact that the
>>> pg_wal directory they're supposed to create already exists.
>>
>> One would hope that they would realize that's not going to work
>> when they set up PG10.  If they aren't paying attention sufficient
>> to realize that then it seems entirely likely that they would feel
>> equally safe removing the contents of a directory named 'pg_xlog'.
>
> So... somebody want to tally up the votes here?

Here is what I have, 6 votes clearly stated:
1. Rename nothing: Daniel,
2. Rename directory only: Andres
3. Rename everything: Stephen, Vladimir, David S, Michael P (with
aliases for functions, I could live without at this point...)

> And... was this discussed at the FOSDEM developer meeting?
>
> (Please say yes.)

Looking only at the minutes, the answer is no:
https://wiki.postgresql.org/wiki/FOSDEM/PGDay_2017_Developer_Meeting
-- 
Michael

On Sun, Feb 5, 2017 at 10:24 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:

>> So... somebody want to tally up the votes here?
>
> Here is what I have, 6 votes clearly stated:
> 1. Rename nothing: Daniel,
> 2. Rename directory only: Andres
> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
> aliases for functions, I could live without at this point...)

I vote for 3 as well, with 1 as the only sane alternative.

-- 
Kevin Grittner

On Mon, Feb 6, 2017 at 1:24 PM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> On Sat, Feb 4, 2017 at 6:39 AM, Robert Haas <robertmhaas@gmail.com> wrote:
>> On Fri, Feb 3, 2017 at 5:21 AM, Stephen Frost <sfrost@snowman.net> wrote:
>>> Daniel,
>>>
>>> * Daniel Verite (daniel@manitou-mail.org) wrote:
>>>> What if we look at the change from the pessimistic angle?
>>>> An example of confusion that the change would create:
>>>> a lot of users currently choose pg_wal for the destination
>>>> directory of their archive command. Less-informed users
>>>> that set up archiving and/or log shipping in PG10 based on
>>>> advice online from previous versions will be fairly
>>>> confused about the missing pg_xlog, and the fact that the
>>>> pg_wal directory they're supposed to create already exists.
>>>
>>> One would hope that they would realize that's not going to work
>>> when they set up PG10.  If they aren't paying attention sufficient
>>> to realize that then it seems entirely likely that they would feel
>>> equally safe removing the contents of a directory named 'pg_xlog'.
>>
>> So... somebody want to tally up the votes here?
>
> Here is what I have, 6 votes clearly stated:
> 1. Rename nothing: Daniel,
> 2. Rename directory only: Andres
> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
> aliases for functions, I could live without at this point...)

I vote for 1.
I still wonder how much the renaming of pg_xlog actually helps very careless
people who remove pg_xlog becase its name includes "log". I'm afraid that
they would make another serious mistake (e.g., remove pg_wal because it has
many files and it occupies large amount of disk space) even after renaming
to pg_wal. The crazy idea, making initdb create the empty file with the name
"DONT_REMOVE_pg_xlog_IF_YOU_DONT_WANT_TO_LOSE_YOUR_IMPORTANT_DATA"
in $PGDATA seems more helpful. Anyway I'm afraid that the renaming would
cause more pain than gain.

Regards,

-- 
Fujii Masao

On Mon, Feb 6, 2017 at 12:29 PM, Magnus Hagander <magnus@hagander.net> wrote:
>> Here is what I have, 6 votes clearly stated:
>> 1. Rename nothing: Daniel,
>> 2. Rename directory only: Andres
>> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
>> aliases for functions, I could live without at this point...)
>
> Put my vote down for 2.

I think there is a very strong consensus for going either forward and
renaming everything or going backward and renaming nothing.  That
position has been endorsed by me, David Johnston, Tom Lane, Stephen
Frost, Kevin Grittner, Vladimir Rusinov, David Steele, probably
Michael Paquier, and possibly JD.  The only people who have explicitly
voted against that position are Andres and Magnus, who prefer renaming
only the directory.  I think that's approximately a 7-2 vote in favor
of not leaving things as they are (#2).

The vote on whether to go forward (#3) or backward (#1) is closer.
All of the people mentioned above as wanting consistency - except for
JD whose actual vote wasn't entirely clear - indicated a preference
for #3 over #1, but a number of them prefer it only weakly.  Moreover,
Fujii Masao and Daniel Verite prefer #1.  But I still think that the
vote is in favor of #3.  There are 7 clear votes for that position and
no more than 2 votes for any other position.  Even regarding every
vote that isn't for #3 as a vote for #1, which is probably not
entirely accurate, it's still 7-4 in favor of #3.

Looking back at older emails before things came quite so sharply into
focus, I found various other opinions.  But I don't think they change
the overall picture very much.  Cynthia Shang seemed to favor a more
limited renaming, but her point was that we have lots of internal
stuff that uses the xlog terminology, which isn't quite the same
question as whether the user-visible stuff should all match.  David
Fetter favored not adding aliases when we did the renaming, but didn't
clearly spell out that he favored the renaming.  Similarly, Euler
Taveira favored aliases in an extension, but likewise didn't clearly
spell out his position on the renaming itself.  (I would tend to count
those as votes in favor of the renaming itself but you could argue
that.)  Bruce Momjian wanted to go forward to keep things clean.
Simon Riggs wanted to leave things as they are, but the reason given
was not so much about the merits of the issue but about not wanting to
spend more time on it.  You can come up with different counts
depending on exactly how you interpret what all of those people said,
but not even the least favorable allocation of those votes ends up
with anything other than #3 as the most popular option.

Therefore, I plan to go ahead and do #3.  Somebody's probably going to
jump in now with another opinion but I think this thread's gone on
long enough.  We're going to take some backward-compatibility pain
here as a result of these changes and some people are going to be
unhappy about that, but I think we've allowed enough time for people
to weigh in with opinions and this seems to be where we're at.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Feb 6, 2017 at 12:29 PM, Magnus Hagander <magnus@hagander.net> wrote:
>>> Here is what I have, 6 votes clearly stated:
>>> 1. Rename nothing: Daniel,
>>> 2. Rename directory only: Andres
>>> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
>>> aliases for functions, I could live without at this point...)

> [ vote-counting ]

> Therefore, I plan to go ahead and do #3.  Somebody's probably going to
> jump in now with another opinion but I think this thread's gone on
> long enough.

Agreed, let's just get it done.

Although this doesn't really settle whether we ought to do 3a (with
backwards-compatibility function aliases in core) or 3b (without 'em).
Do people want to re-vote, understanding that those are the remaining
choices?
        regards, tom lane

On 2/9/17 2:14 PM, David G. Johnston wrote:
> On Thu, Feb 9, 2017 at 12:08 PM, Tom Lane <tgl@sss.pgh.pa.us
> <mailto:tgl@sss.pgh.pa.us>>wrote:
> 
>     Robert Haas <robertmhaas@gmail.com <mailto:robertmhaas@gmail.com>>
>     writes:
>     > On Mon, Feb 6, 2017 at 12:29 PM, Magnus Hagander <magnus@hagander.net <mailto:magnus@hagander.net>> wrote:
>     >>> Here is what I have, 6 votes clearly stated:
>     >>> 1. Rename nothing: Daniel,
>     >>> 2. Rename directory only: Andres
>     >>> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
>     >>> aliases for functions, I could live without at this point...)
> 
>     > [ vote-counting ]
> 
>     > Therefore, I plan to go ahead and do #3.  Somebody's probably going to
>     > jump in now with another opinion but I think this thread's gone on
>     > long enough.
> 
>     Agreed, let's just get it done.
> 
>     Although this doesn't really settle whether we ought to do 3a (with
>     backwards-compatibility function aliases in core) or 3b (without 'em).
>     Do people want to re-vote, understanding that those are the remaining
>     choices?

-1 on aliases in core (so to be clear my vote is for 3b).

> ​I wouldn't oppose:
> 
> CREATE EXTENSION give_me_my_xlog_back;
> 
> but my prior thoughts lead me toward not including such functions in the
> bootstrap catalog.

I'm not very excited about an extension, can we just provide a link to a
script in the release notes, or simply note that wrappers can be created?

-- 
-David
david@pgmasters.net

On 2017-02-09 14:08:14 -0500, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Mon, Feb 6, 2017 at 12:29 PM, Magnus Hagander <magnus@hagander.net> wrote:
> >>> Here is what I have, 6 votes clearly stated:
> >>> 1. Rename nothing: Daniel,
> >>> 2. Rename directory only: Andres
> >>> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
> >>> aliases for functions, I could live without at this point...)
> 
> > [ vote-counting ]
> 
> > Therefore, I plan to go ahead and do #3.  Somebody's probably going to
> > jump in now with another opinion but I think this thread's gone on
> > long enough.
> 
> Agreed, let's just get it done.
> 
> Although this doesn't really settle whether we ought to do 3a (with
> backwards-compatibility function aliases in core) or 3b (without 'em).
> Do people want to re-vote, understanding that those are the remaining
> choices?

3a)

On Thu, Feb 9, 2017 at 2:08 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Mon, Feb 6, 2017 at 12:29 PM, Magnus Hagander <magnus@hagander.net> wrote:
>>>> Here is what I have, 6 votes clearly stated:
>>>> 1. Rename nothing: Daniel,
>>>> 2. Rename directory only: Andres
>>>> 3. Rename everything: Stephen, Vladimir, David S, Michael P (with
>>>> aliases for functions, I could live without at this point...)
>
>> [ vote-counting ]
>
>> Therefore, I plan to go ahead and do #3.  Somebody's probably going to
>> jump in now with another opinion but I think this thread's gone on
>> long enough.
>
> Agreed, let's just get it done.
>
> Although this doesn't really settle whether we ought to do 3a (with
> backwards-compatibility function aliases in core) or 3b (without 'em).
> Do people want to re-vote, understanding that those are the remaining
> choices?

I prefer (3c) put them in an extension and let people that need 'em
install 'em, but not have them available by default.  If the only
choices are (3a) and (3b) then I guess I pick (3b), but I think an
extension doesn't cost much and will ease the migration pain for users
so I'm in favor of it.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:
> On Thu, Feb 9, 2017 at 2:08 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Although this doesn't really settle whether we ought to do 3a (with
>> backwards-compatibility function aliases in core) or 3b (without 'em).
>> Do people want to re-vote, understanding that those are the remaining
>> choices?

> I prefer (3c) put them in an extension and let people that need 'em
> install 'em, but not have them available by default.

As far as the core code is concerned, 3b and 3c are the same thing.
IOW, somebody can write the extension later.
        regards, tom lane

On Thu, Feb 9, 2017 at 1:44 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Thu, Feb 9, 2017 at 2:08 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Although this doesn't really settle whether we ought to do 3a (with
>>> backwards-compatibility function aliases in core) or 3b (without 'em).
>>> Do people want to re-vote, understanding that those are the remaining
>>> choices?
>
>> I prefer (3c) put them in an extension and let people that need 'em
>> install 'em, but not have them available by default.
>
> As far as the core code is concerned, 3b and 3c are the same thing.

I think so, too, if we're talking about an extension in core.

My vote is for 3b.  If someone wants to write the alias extension and
make it available outside of core, fine -- though they don't need anyone's
vote to do so.

-- 
Kevin Grittner

On 02/09/2017 11:08 AM, Tom Lane wrote:
> Agreed, let's just get it done.
> 
> Although this doesn't really settle whether we ought to do 3a (with
> backwards-compatibility function aliases in core) or 3b (without 'em).
> Do people want to re-vote, understanding that those are the remaining
> choices?

Does 3a) mean keeping the aliases more-or-less forever?

If not, I vote for 3b.  If we're going to need to break stuff, let's
just do it.

If we can keep the aliases for 6-10 years, then I see no reason not to
have them (3a).  They're not exactly likely to conflict with user-chosen
names.

-- 
Josh Berkus
Containers & Databases Oh My!

* Josh Berkus (josh@berkus.org) wrote:
> On 02/09/2017 11:08 AM, Tom Lane wrote:
> > Agreed, let's just get it done.
> >
> > Although this doesn't really settle whether we ought to do 3a (with
> > backwards-compatibility function aliases in core) or 3b (without 'em).
> > Do people want to re-vote, understanding that those are the remaining
> > choices?
>
> Does 3a) mean keeping the aliases more-or-less forever?
>
> If not, I vote for 3b.  If we're going to need to break stuff, let's
> just do it.
>
> If we can keep the aliases for 6-10 years, then I see no reason not to
> have them (3a).  They're not exactly likely to conflict with user-chosen
> names.

When we remove pg_shadow, then I'll be willing to agree that maybe we
can start having things in PG for a couple releases that are just for
backwards-compatibility and will actually be removed later.

History has shown that's next to impossible, however.

Thanks!

Stephen

On 02/09/2017 12:42 PM, Stephen Frost wrote:
> * Josh Berkus (josh@berkus.org) wrote:
>> On 02/09/2017 11:08 AM, Tom Lane wrote:
>>> Agreed, let's just get it done.
>>>
>>> Although this doesn't really settle whether we ought to do 3a (with
>>> backwards-compatibility function aliases in core) or 3b (without 'em).
>>> Do people want to re-vote, understanding that those are the remaining
>>> choices?
>>
>> Does 3a) mean keeping the aliases more-or-less forever?
>>
>> If not, I vote for 3b.  If we're going to need to break stuff, let's
>> just do it.
>>
>> If we can keep the aliases for 6-10 years, then I see no reason not to
>> have them (3a).  They're not exactly likely to conflict with user-chosen
>> names.
> 
> When we remove pg_shadow, then I'll be willing to agree that maybe we
> can start having things in PG for a couple releases that are just for
> backwards-compatibility and will actually be removed later.
> 
> History has shown that's next to impossible, however.

That's why I said 6-10 years.  If we're doing 3a, realistically we're
supporting it until PostgreSQL 16, at least, and more likely 20.  I'm OK
with that.

What I'm voting against is the idea that we'll have aliases in core, but
remove them in two releases.  Either that's unrealistic, or it's just
prolonging the pain.

-- 
Josh Berkus
Containers & Databases Oh My!

* Josh Berkus (josh@berkus.org) wrote:
> On 02/09/2017 12:42 PM, Stephen Frost wrote:
> > * Josh Berkus (josh@berkus.org) wrote:
> >> On 02/09/2017 11:08 AM, Tom Lane wrote:
> >>> Agreed, let's just get it done.
> >>>
> >>> Although this doesn't really settle whether we ought to do 3a (with
> >>> backwards-compatibility function aliases in core) or 3b (without 'em).
> >>> Do people want to re-vote, understanding that those are the remaining
> >>> choices?
> >>
> >> Does 3a) mean keeping the aliases more-or-less forever?
> >>
> >> If not, I vote for 3b.  If we're going to need to break stuff, let's
> >> just do it.
> >>
> >> If we can keep the aliases for 6-10 years, then I see no reason not to
> >> have them (3a).  They're not exactly likely to conflict with user-chosen
> >> names.
> >
> > When we remove pg_shadow, then I'll be willing to agree that maybe we
> > can start having things in PG for a couple releases that are just for
> > backwards-compatibility and will actually be removed later.
> >
> > History has shown that's next to impossible, however.
>
> That's why I said 6-10 years.  If we're doing 3a, realistically we're
> supporting it until PostgreSQL 16, at least, and more likely 20.  I'm OK
> with that.

Uh, to be clear, I think it's an entirely bad thing that we've had those
views and various other cruft hang around for over 10 years.

And removing them today will probably still have people crying about how
pgAdmin3 and other things still use them.

> What I'm voting against is the idea that we'll have aliases in core, but
> remove them in two releases.  Either that's unrealistic, or it's just
> prolonging the pain.

Waiting 10+ years doesn't make the pain go away when it comes to
removing things like that.

Let's not go there.

Thanks!

Stephen

On 2017-02-09 15:53:47 -0500, Stephen Frost wrote:
> * Josh Berkus (josh@berkus.org) wrote:
> > What I'm voting against is the idea that we'll have aliases in core, but
> > remove them in two releases.  Either that's unrealistic, or it's just
> > prolonging the pain.
> 
> Waiting 10+ years doesn't make the pain go away when it comes to
> removing things like that.

That's ridiculous. I think we can fairly argue whether backward compat
is worth it, but claiming that migrations to something new aren't easier
if there's a number of versions that support both the old and the new
names/syntax/whatnot is obviously wrong.

On 02/09/2017 12:53 PM, Stephen Frost wrote:
> * Josh Berkus (josh@berkus.org) wrote:
>> On 02/09/2017 12:42 PM, Stephen Frost wrote:
>>> * Josh Berkus (josh@berkus.org) wrote:
>>>> On 02/09/2017 11:08 AM, Tom Lane wrote:
>>>>> Agreed, let's just get it done.
>>>>>
>>>>> Although this doesn't really settle whether we ought to do 3a (with
>>>>> backwards-compatibility function aliases in core) or 3b (without 'em).
>>>>> Do people want to re-vote, understanding that those are the remaining
>>>>> choices?
>>>>
>>>> Does 3a) mean keeping the aliases more-or-less forever?
>>>>
>>>> If not, I vote for 3b.  If we're going to need to break stuff, let's
>>>> just do it.
>>>>
>>>> If we can keep the aliases for 6-10 years, then I see no reason not to
>>>> have them (3a).  They're not exactly likely to conflict with user-chosen
>>>> names.
>>>
>>> When we remove pg_shadow, then I'll be willing to agree that maybe we
>>> can start having things in PG for a couple releases that are just for
>>> backwards-compatibility and will actually be removed later.
>>>
>>> History has shown that's next to impossible, however.
>>
>> That's why I said 6-10 years.  If we're doing 3a, realistically we're
>> supporting it until PostgreSQL 16, at least, and more likely 20.  I'm OK
>> with that.
> 
> Uh, to be clear, I think it's an entirely bad thing that we've had those
> views and various other cruft hang around for over 10 years.
> 
> And removing them today will probably still have people crying about how
> pgAdmin3 and other things still use them.
> 
>> What I'm voting against is the idea that we'll have aliases in core, but
>> remove them in two releases.  Either that's unrealistic, or it's just
>> prolonging the pain.
> 
> Waiting 10+ years doesn't make the pain go away when it comes to
> removing things like that.

Sure it does.  That's two whole generations of client tools.  For
example, at that point, pgAdmin3 won't reliably run on any supported
platform, so it won't be a problem if we break it.

If we clearly mark the old function names as deprecated aliases, client
tools will gradually move to the new names.

Counter-argument: moving the directory is going to break many tools
anyway, so why bother with function aliases?

-- 
Josh Berkus
Containers & Databases Oh My!

Andres,

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-02-09 15:53:47 -0500, Stephen Frost wrote:
> > * Josh Berkus (josh@berkus.org) wrote:
> > > What I'm voting against is the idea that we'll have aliases in core, but
> > > remove them in two releases.  Either that's unrealistic, or it's just
> > > prolonging the pain.
> >
> > Waiting 10+ years doesn't make the pain go away when it comes to
> > removing things like that.
>
> That's ridiculous. I think we can fairly argue whether backward compat
> is worth it, but claiming that migrations to something new aren't easier
> if there's a number of versions that support both the old and the new
> names/syntax/whatnot is obviously wrong.

We do provide a number of years of overlap- 5 years in fact.  I believe
that's an entirely good thing and gives our users a chance to manage
their upgrade paths from one major release to the next.

Increasing that overlap to 10 or 15 years, however, doesn't make a
hill-of-beans worth of difference, in my opinion.  Either they're
tracking the releases that we're doing and making changes and adapting
as we change things, or they aren't and won't ever unless they're forced
to, instead preferring to make us carry the burden of maintenance on the
backwards-compat pieces.

Thanks!

Stephen

On 2/9/17 4:03 PM, Josh Berkus wrote:

> Counter-argument: moving the directory is going to break many tools
> anyway, so why bother with function aliases?

+1.

-- 
-David
david@pgmasters.net

* Josh Berkus (josh@berkus.org) wrote:
> On 02/09/2017 12:53 PM, Stephen Frost wrote:
> > Waiting 10+ years doesn't make the pain go away when it comes to
> > removing things like that.
>
> Sure it does.  That's two whole generations of client tools.  For
> example, at that point, pgAdmin3 won't reliably run on any supported
> platform, so it won't be a problem if we break it.

That's not true if OpenSCG has their way as they're continuing to
maintain it, from my understanding.

And, I'm sorry, but counting on projects to essentially die off to be
the point where we can drop certain bits of backwards-compatibility just
isn't a winning strategy.

> If we clearly mark the old function names as deprecated aliases, client
> tools will gradually move to the new names.

No, they won't.  They haven't.  Look at pg_shadow- it was clearly marked
as deprecated in *8.1*.

> Counter-argument: moving the directory is going to break many tools
> anyway, so why bother with function aliases?

You're going to have to explain exactly the argument you're making
there, because I don't see the point you're trying to get at with that
question.

Thanks!

Stephen

On 2017-02-09 13:03:41 -0800, Josh Berkus wrote:
> Counter-argument: moving the directory is going to break many tools
> anyway, so why bother with function aliases?

There's not actually that many tools affected by renaming pg_xlog,
i.e. there are tools that aren't affected by the rename at all, but do
calle *xlog* functions.

Stephen Frost <sfrost@snowman.net> writes:
> * Josh Berkus (josh@berkus.org) wrote:
>> If we clearly mark the old function names as deprecated aliases, client
>> tools will gradually move to the new names.

> No, they won't.  They haven't.  Look at pg_shadow- it was clearly marked
> as deprecated in *8.1*.

Back in 8.1 we didn't have extensions.  Now that we do, I think the
terms of discussion are a bit different.  In particular, if we relegate
the aliases to an extension, the pain of getting/installing that extension
will provide a forcing function encouraging users to fix their code so
they don't need it anymore.

Also, our experience with contrib/tsearch2 suggests that the extension
shouldn't be part of contrib, because we have zero track record of getting
rid of stuff in contrib, no matter how dead it is.

So I'm entirely happy with somebody who feels a need for this developing
an extension that we don't ship with the core system, and maintaining it
for as long as they continue to feel the need for it.  We couldn't stop
that from happening anyway.  But let's not buy into maintaining it as
part of the core distribution.  We don't have a mechanism for getting
rid of stuff once it's in the distro.
        regards, tom lane

On Thu, Feb 9, 2017 at 4:24 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Also, our experience with contrib/tsearch2 suggests that the extension
> shouldn't be part of contrib, because we have zero track record of getting
> rid of stuff in contrib, no matter how dead it is.

Let's nuke tsearch2 to remove this adverse precedent, and then add the
new thing.

Anybody who still wants tsearch2 can go get it from an old version, or
somebody can maintain a fork on github.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

* Robert Haas (robertmhaas@gmail.com) wrote:
> On Thu, Feb 9, 2017 at 4:24 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Also, our experience with contrib/tsearch2 suggests that the extension
> > shouldn't be part of contrib, because we have zero track record of getting
> > rid of stuff in contrib, no matter how dead it is.
>
> Let's nuke tsearch2 to remove this adverse precedent, and then add the
> new thing.
>
> Anybody who still wants tsearch2 can go get it from an old version, or
> somebody can maintain a fork on github.

Works for me.  I'd love to nuke pg_shadow and all the other
not-really-maintained backwards-compat things from when roles were
added too.

Thanks!

Stephen

On 2017-02-09 19:19:21 -0500, Stephen Frost wrote:
> * Robert Haas (robertmhaas@gmail.com) wrote:
> > On Thu, Feb 9, 2017 at 4:24 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > > Also, our experience with contrib/tsearch2 suggests that the extension
> > > shouldn't be part of contrib, because we have zero track record of getting
> > > rid of stuff in contrib, no matter how dead it is.
> > 
> > Let's nuke tsearch2 to remove this adverse precedent, and then add the
> > new thing.
> > 
> > Anybody who still wants tsearch2 can go get it from an old version, or
> > somebody can maintain a fork on github.
> 
> Works for me.

+1


> I'd love to nuke pg_shadow and all the other
> not-really-maintained backwards-compat things from when roles were
> added too.

Not sure if it's worth the work to rip out and such, but I'm mildly
supportive of this one too.  Depends a bit on what all the other things
are ;)

Andres

* Andres Freund (andres@anarazel.de) wrote:
> On 2017-02-09 19:19:21 -0500, Stephen Frost wrote:
> > I'd love to nuke pg_shadow and all the other
> > not-really-maintained backwards-compat things from when roles were
> > added too.
>
> Not sure if it's worth the work to rip out and such, but I'm mildly
> supportive of this one too.  Depends a bit on what all the other things
> are ;)

Reviewing 7762619e95272974f90a38d8d85aafbe0e94add5 where roles were
added, I find:

pg_user   - use pg_roles instead, which actually includes all of the role           attributes, unlike pg_user

pg_shadow - use pg_authid instead, which, again, actually includes all           of the role attributes, unlike
pg_shadow.

pg_group  - use pg_auth_members instead, which includes the info about           the admin option and the grantor

I don't think we should remove things like CREATE USER, that's a
perfectly reasonable and maintained interface, unlike the above views,
which missed out on things like the 'createrole' role attribute.

Thanks!

Stephen

On 2/9/17 6:37 PM, Andres Freund wrote:
>>> Anybody who still wants tsearch2 can go get it from an old version, or
>>> somebody can maintain a fork on github.
>> Works for me.
> +1

+1

>> I'd love to nuke pg_shadow and all the other
>> not-really-maintained backwards-compat things from when roles were
>> added too.
> Not sure if it's worth the work to rip out and such, but I'm mildly
> supportive of this one too.  Depends a bit on what all the other things
> are ;)

The problem with pg_shadow is unless you specifically looked at it in 
the docs after 8.1, you had no idea it was deprecated. I don't really 
think of it as deprecated.

As someone mentioned, forcing a user to install an extension makes the 
deprecation visible. Another option would be to have the backend spit 
out a WARNING the first time you access anything that's deprecated. Both 
of those are pertinent reminders to people that they need to change 
their tools.
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

Jim,

* Jim Nasby (Jim.Nasby@BlueTreble.com) wrote:
> On 2/9/17 6:37 PM, Andres Freund wrote:
> >>I'd love to nuke pg_shadow and all the other
> >>not-really-maintained backwards-compat things from when roles were
> >>added too.
> >Not sure if it's worth the work to rip out and such, but I'm mildly
> >supportive of this one too.  Depends a bit on what all the other things
> >are ;)
>
> The problem with pg_shadow is unless you specifically looked at it
> in the docs after 8.1, you had no idea it was deprecated. I don't
> really think of it as deprecated.

It's not even maintained properly, I hardly see how it couldn't be
anything but deprecated, and the docs certainly are the right place, if
anywhere, to say that something is deprecated.

> As someone mentioned, forcing a user to install an extension makes
> the deprecation visible. Another option would be to have the backend
> spit out a WARNING the first time you access anything that's
> deprecated. Both of those are pertinent reminders to people that
> they need to change their tools.

Ugh.  Please, no.  Hacking up the backend to recognize that a given
query is referring to a deprecated view and then throwing a warning on
it is just plain ugly.

Let's go one step further, and throw an ERROR if someone tries to query
these views instead.

Thanks!

Stephen

On Fri, Feb 10, 2017 at 10:16 AM, Stephen Frost <sfrost@snowman.net> wrote:
>> As someone mentioned, forcing a user to install an extension makes
>> the deprecation visible. Another option would be to have the backend
>> spit out a WARNING the first time you access anything that's
>> deprecated. Both of those are pertinent reminders to people that
>> they need to change their tools.
>
> Ugh.  Please, no.  Hacking up the backend to recognize that a given
> query is referring to a deprecated view and then throwing a warning on
> it is just plain ugly.
>
> Let's go one step further, and throw an ERROR if someone tries to query
> these views instead.

FWIW, I am of the opinion to just nuke them as the "soft of"
deprecation period has been very long. Applications should have
switched to pg_authid and pg_roles long ago already.
-- 
Michael

On 2017-02-09 20:02:54 -0500, Stephen Frost wrote:
> * Andres Freund (andres@anarazel.de) wrote:
> > On 2017-02-09 19:19:21 -0500, Stephen Frost wrote:
> > > I'd love to nuke pg_shadow and all the other
> > > not-really-maintained backwards-compat things from when roles were
> > > added too.
> > 
> > Not sure if it's worth the work to rip out and such, but I'm mildly
> > supportive of this one too.  Depends a bit on what all the other things
> > are ;)
> 
> Reviewing 7762619e95272974f90a38d8d85aafbe0e94add5 where roles were
> added, I find:
> 
> pg_user   - use pg_roles instead, which actually includes all of the role
>             attributes, unlike pg_user

Hm, I presume this is the most used one.


> pg_group  - use pg_auth_members instead, which includes the info about
>             the admin option and the grantor

then this.

> pg_shadow - use pg_authid instead, which, again, actually includes all
>             of the role attributes, unlike pg_shadow.

That's probably fine.


I'm fine with dropping now, alternatively we could, and that seems like
it'd institute a good practice, name them to be removed in 10+1 in the
10 release notes. "Upcoming removal of deprecated features" or such. And
schedule stuff for that regularly.  Like e.g. dropping psql support for
< 9.0 (randomly chosen version), pg_dump support for very old versions,
etc, ...
While not everyone will be saved by that (by virtue of not reading /
reacting) it helps those that actually read the notes.  Obviously
there'd still some incompatibilities that do not go through that
mechanism.


> I don't think we should remove things like CREATE USER, that's a
> perfectly reasonable and maintained interface, unlike the above views,
> which missed out on things like the 'createrole' role attribute.

Yea, that'd be a bad plan.


Greetings,

Andres Freund

On 02/09/2017 05:19 PM, Michael Paquier wrote:
> On Fri, Feb 10, 2017 at 10:16 AM, Stephen Frost <sfrost@snowman.net> wrote:
>>> As someone mentioned, forcing a user to install an extension makes
>>> the deprecation visible. Another option would be to have the backend
>>> spit out a WARNING the first time you access anything that's
>>> deprecated. Both of those are pertinent reminders to people that
>>> they need to change their tools.
>>
>> Ugh.  Please, no.  Hacking up the backend to recognize that a given
>> query is referring to a deprecated view and then throwing a warning on
>> it is just plain ugly.
>>
>> Let's go one step further, and throw an ERROR if someone tries to query
>> these views instead.
> 
> FWIW, I am of the opinion to just nuke them as the "soft of"
> deprecation period has been very long. Applications should have
> switched to pg_authid and pg_roles long ago already.
> 

We will definitely break a lot of client code by removing these -- I
know that, deprecated or not, a lot of infrequently-updated
driver/orm/GUI code still refers to pg_shadow/pg_user.

I think Postgres 10 is the right time to break that code (I mean, we
have to do it someday, and we're already telling people about breakage
in 10), but be aware that there will be shouting and carrying on.

-1 on a warning.  Very little code today which references the deprecated
code is interactive, so who's going to see the warning?

-- 
Josh Berkus
Containers & Databases Oh My!

* Josh Berkus (josh@berkus.org) wrote:
> On 02/09/2017 05:19 PM, Michael Paquier wrote:
> > On Fri, Feb 10, 2017 at 10:16 AM, Stephen Frost <sfrost@snowman.net> wrote:
> >>> As someone mentioned, forcing a user to install an extension makes
> >>> the deprecation visible. Another option would be to have the backend
> >>> spit out a WARNING the first time you access anything that's
> >>> deprecated. Both of those are pertinent reminders to people that
> >>> they need to change their tools.
> >>
> >> Ugh.  Please, no.  Hacking up the backend to recognize that a given
> >> query is referring to a deprecated view and then throwing a warning on
> >> it is just plain ugly.
> >>
> >> Let's go one step further, and throw an ERROR if someone tries to query
> >> these views instead.
> >
> > FWIW, I am of the opinion to just nuke them as the "soft of"
> > deprecation period has been very long. Applications should have
> > switched to pg_authid and pg_roles long ago already.
>
> We will definitely break a lot of client code by removing these -- I
> know that, deprecated or not, a lot of infrequently-updated
> driver/orm/GUI code still refers to pg_shadow/pg_user.

Any that we're aware of, I'd suggest we reach out to the maintainers of
and let them know.

> I think Postgres 10 is the right time to break that code (I mean, we
> have to do it someday, and we're already telling people about breakage
> in 10), but be aware that there will be shouting and carrying on.

Agreed.

> -1 on a warning.  Very little code today which references the deprecated
> code is interactive, so who's going to see the warning?

Indeed, I was thinking of this also.  The warnings would just end up in
the server logs, amongst tons of often not-terribly-interesting
information that far too many users ignore already, and those that don't
probably actually read the release notes.

Thanks!

Stephen

On Fri, Feb  3, 2017 at 05:21:28AM -0500, Stephen Frost wrote:
> > Also googling for pg_wal, I'm finding food for thought like this
> > IBM technote:
> > http://www-01.ibm.com/support/docview.wss?uid=isg3T1015637
> > which recommends to 
> > "Remove all files under /var/lib/pgsql/9.0/data/pg_wal/"
> > and also calls that directory the "write-ahead log directory"
> > which is quite confusing because apparently it's the destination of
> > their archive command.
> 
> It's certainly unfortunate that people have thought that they can create
> arbitrary directories under the PG data directory.  That's never going
> to be safe, witness that we've created new directories under PGDATA in
> the last few releases and I don't see any reason why that would change
> moving forward.  Perhaps we should check for the existance of such a
> directory during pg_upgrade and throw an error, and we should go back
> and do the same for other directories which have been added over
> releases, but I'm not sure I can see an argument for doing much more
> than that.

Actually, pg_upgrade already checks for some odd directories stored
inside of PGDATA:
WARNING:  new data directory should not be inside theold data directory, e.g. %s\n", old_cluster_pgdata);
WARNING:  user-defined tablespace locations shouldnot be inside the data directory, e.g. %s\n", old_tablespace_dir);

--  Bruce Momjian  <bruce@momjian.us>        http://momjian.us EnterpriseDB
http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +