Thread: Release Notes Archive Patch

Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
Hi,

Attached is a proposed patch for archiving the release notes on the
website. The patch builds the release note archive without modifying the
schema and works with old and new releases.

It works as such:

- Find all release notes that have been loaded into pgweb (a process
that occurs each release when we load the documentation)
- Find the last modified release notes for each version. This accounts
for the old docs, e.g. there is a copy of the 9.2.8 notes in the 9.3.9
release, so we will render those
- There is a list of all release notes. If you click into it, you will
see the version specific release notes

Known adjustments to be made:

- The URLs on each release note page are broken. I have to figure out
how the best way to change the relative path in the URLs that are on the
page as it is going from the page root (/docs/release) instead of the
site root. This _should_ be easy
- Even with above fix, some URLs will be broken in them, i.e. references
to other release notes
- PostgreSQL v1 + v0: I need to relabel this for Postgres95
- Also, some of the earlier versions do not have the proper release
numbering (e.g. 1.03)

I'm hoping we can put this out with the upcoming release, but am ok with
doing this shortly thereafter.

Comments on patch and approach welcome.

Thanks,

Jonathan

Attachment

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/12/19 8:12 PM, Jonathan S. Katz wrote:
> Hi,
>
> Attached is a proposed patch for archiving the release notes on the
> website. The patch builds the release note archive without modifying the
> schema and works with old and new releases.
>
> Known adjustments to be made:
>
> - The URLs on each release note page are broken. I have to figure out
> how the best way to change the relative path in the URLs that are on the
> page as it is going from the page root (/docs/release) instead of the
> site root. This _should_ be easy
> - Even with above fix, some URLs will be broken in them, i.e. references
> to other release notes
> - PostgreSQL v1 + v0: I need to relabel this for Postgres95
> - Also, some of the earlier versions do not have the proper release
> numbering (e.g. 1.03)

All of the above are now fixed. Attached is v2 of the patch. I've also
attached a couple of screenshots so you can see how it looks.

IMV this is ready to go, but would like some review and/or additional
testing prior to pushing.

Thanks,

Jonathan

Attachment

Re: Release Notes Archive Patch

From
Tom Lane
Date:
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> All of the above are now fixed. Attached is v2 of the patch. I've also
> attached a couple of screenshots so you can see how it looks.
> IMV this is ready to go, but would like some review and/or additional
> testing prior to pushing.

The patch doesn't convey a lot to my mind, so I have to ask: exactly
what URLs would this present?  Does it do anything to respond to
Andres' complaint that existing relnote pages will vanish?

            regards, tom lane


Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/13/19 10:13 AM, Tom Lane wrote:
> "Jonathan S. Katz" <jkatz@postgresql.org> writes:
>> All of the above are now fixed. Attached is v2 of the patch. I've also
>> attached a couple of screenshots so you can see how it looks.
>> IMV this is ready to go, but would like some review and/or additional
>> testing prior to pushing.
>
> The patch doesn't convey a lot to my mind, so I have to ask: exactly
> what URLs would this present?

I'm a bit confuse by the question, so I'll answer it from two angles.

WRT loading the release notes themselves, basically it is using the data
found in the current doc imports. Often in the release notes, there are
URLs either to older release notes (like in the top line) or various
features/sections of the documentation (e.g. a GUC, a SQL command, etc.)

Within these notes themselves, all the URLs should work.

The patch also introduces a new set of URLs to present the release
notes, of the format:

/docs/release/11.2/
/docs/release/11.1/
/docs/release/11.0/

...

/docs/release/9.6.2/
/docs/release/9.6.1/
/docs/release/9.6.0/

...

(and some ugliness in the old ones, which I could maybe adjust, but for
now):

/docs/release/1.9/
/docs/release/1.2/
/docs/release/1.1/
/docs/release/1.0/
/docs/release/0.3/
/docs/release/0.1/


> Does it do anything to respond to
> Andres' complaint that existing relnote pages will vanish?

Yes. All the relnotes are now in this archive.

Please let me know if any of the above is unclear.

Thanks,

Jonathan


Attachment

Re: Release Notes Archive Patch

From
Tom Lane
Date:
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 2/13/19 10:13 AM, Tom Lane wrote:
>> The patch doesn't convey a lot to my mind, so I have to ask: exactly
>> what URLs would this present?

> I'm a bit confuse by the question, so I'll answer it from two angles.

Neither of those are quite what I meant.  As I understood Andres'
complaint, it was that right now we have URLs like

https://www.postgresql.org/docs/10/release-9-6-5.html

but those are going to be 404 beginning tomorrow.  It sounds from your
response like this patch isn't meant to restore those cross-version URLs.
Which is fine by me, really, but we might get pushback about it.  If so,
there might be material for follow-on work to redirect those into this
new set of URLs.

> WRT loading the release notes themselves, basically it is using the data
> found in the current doc imports. Often in the release notes, there are
> URLs either to older release notes (like in the top line) or various
> features/sections of the documentation (e.g. a GUC, a SQL command, etc.)

Right.  The latter type will link to the main docs for the associated
release, correct?

            regards, tom lane


Re: Release Notes Archive Patch

From
Magnus Hagander
Date:
On Wed, Feb 13, 2019 at 5:18 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 2/13/19 10:13 AM, Tom Lane wrote:
>> The patch doesn't convey a lot to my mind, so I have to ask: exactly
>> what URLs would this present?

> I'm a bit confuse by the question, so I'll answer it from two angles.

Neither of those are quite what I meant.  As I understood Andres'
complaint, it was that right now we have URLs like

https://www.postgresql.org/docs/10/release-9-6-5.html

but those are going to be 404 beginning tomorrow.  It sounds from your
response like this patch isn't meant to restore those cross-version URLs.
Which is fine by me, really, but we might get pushback about it.  If so,
there might be material for follow-on work to redirect those into this
new set of URLs.

We can make those become redirects at the webserver level, I'm pretty sure. Seems like it should be easy enough to regexp-match it for a simple mapping.

I'm less certain if we can have that tested and deployed by tomorrow. (Which goes for the whole patch from Jonathan as well)

//Magnus

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/13/19 11:18 AM, Tom Lane wrote:
> "Jonathan S. Katz" <jkatz@postgresql.org> writes:
>> On 2/13/19 10:13 AM, Tom Lane wrote:
>>> The patch doesn't convey a lot to my mind, so I have to ask: exactly
>>> what URLs would this present?
>
>> I'm a bit confuse by the question, so I'll answer it from two angles.
>
> Neither of those are quite what I meant.  As I understood Andres'
> complaint, it was that right now we have URLs like
>
> https://www.postgresql.org/docs/10/release-9-6-5.html
>
> but those are going to be 404 beginning tomorrow.  It sounds from your
> response like this patch isn't meant to restore those cross-version URLs.
> Which is fine by me, really, but we might get pushback about it.  If so,
> there might be material for follow-on work to redirect those into this
> new set of URLs

Thanks for clarifying!


So, in anticipation of push back, I also want to provide some stats. I
looked at overall traffic since Jul 1, 2018 to yesterday. Traffic to any
of the release notes in general was a very small chunk of it (< 0.5%).

When just looking at the subset of release notes, the top 5 receivers were:

1. 11.0
2. 10.5
3. 10.4
4. "current/static/release.html"
5. 9.6 release index

This constituted a little over 35% of the overall release note traffic.

My inferences are that people tend to go to the release notes for the
"latest" release.

Now, in terms of things like "/docs/9.5/9-4-19.html" and the like, the
only one that had significant traffic was
/docs/9.6/static/release-9-5.html, and that was #25 on the list (the
notes coming before it accounting for 60%+ of the traffic). In fact, I
did not see another of the "mismatch" URLs until #37 on the list, which
was /docs/10/static/release-9-5.html

From a user experience perspective, does it stink that we're potentially
creating a bunch of 404s? Yes. Is it going to upset some people? Yes,
and I'm sympathetic to that. However, looking at the overall numbers, I
think we'll be ok, even if we don't have the redirects per Magnus'
suggestion downthread (which I will also reply to).


>
>> WRT loading the release notes themselves, basically it is using the data
>> found in the current doc imports. Often in the release notes, there are
>> URLs either to older release notes (like in the top line) or various
>> features/sections of the documentation (e.g. a GUC, a SQL command, etc.)
>
> Right.  The latter type will link to the main docs for the associated
> release, correct?

Correct.

Thanks,

Jonathan


Attachment

Re: Release Notes Archive Patch

From
Tom Lane
Date:
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 2/13/19 11:18 AM, Tom Lane wrote:
>> https://www.postgresql.org/docs/10/release-9-6-5.html
>> but those are going to be 404 beginning tomorrow.  It sounds from your
>> response like this patch isn't meant to restore those cross-version URLs.
>> Which is fine by me, really, but we might get pushback about it.  If so,
>> ...

> Now, in terms of things like "/docs/9.5/9-4-19.html" and the like, the
> only one that had significant traffic was
> /docs/9.6/static/release-9-5.html, and that was #25 on the list (the
> notes coming before it accounting for 60%+ of the traffic).

I wonder how much of that just amounts to people clicking on the "next
page" link till they get to a page that's not relevant to what they're
interested in.  If so, there won't be any UX problem, because the
"next page" link won't take them there anymore.

Making these URLs go away only matters to the extent that people have
them bookmarked (seems unlikely) or get sent there by google searches
(rather more likely).

Anyway, I think we could push what you have here and then await
developments before deciding if there's a need for more work.

            regards, tom lane


Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/13/19 11:23 AM, Magnus Hagander wrote:
> On Wed, Feb 13, 2019 at 5:18 PM Tom Lane <tgl@sss.pgh.pa.us
> <mailto:tgl@sss.pgh.pa.us>> wrote:
>
>     "Jonathan S. Katz" <jkatz@postgresql.org
>     <mailto:jkatz@postgresql.org>> writes:
>     > On 2/13/19 10:13 AM, Tom Lane wrote:
>     >> The patch doesn't convey a lot to my mind, so I have to ask: exactly
>     >> what URLs would this present?
>
>     > I'm a bit confuse by the question, so I'll answer it from two angles.
>
>     Neither of those are quite what I meant.  As I understood Andres'
>     complaint, it was that right now we have URLs like
>
>     https://www.postgresql.org/docs/10/release-9-6-5.html
>
>     but those are going to be 404 beginning tomorrow.  It sounds from your
>     response like this patch isn't meant to restore those cross-version
>     URLs.
>     Which is fine by me, really, but we might get pushback about it.  If so,
>     there might be material for follow-on work to redirect those into this
>     new set of URLs.
>
>
> We can make those become redirects at the webserver level, I'm pretty
> sure. Seems like it should be easy enough to regexp-match it for a
> simple mapping.

+1; I would question whether or not its worth it, but we do have that
option.

>
> I'm less certain if we can have that tested and deployed by tomorrow.
> (Which goes for the whole patch from Jonathan as well)

I will profess confidence with the patch -- of course, I would
appreciate additional validation (and I'm there are different approaches
as well, given I put forward one back in the summer :)

If we want to push it, I'm ok to own it and be available for any fixes
(which is easier now that I'm in one place for awhile, and given it's
release day I try to stay around).

Thanks,

Jonathan


Attachment

Re: Release Notes Archive Patch

From
Andres Freund
Date:
Hi,

On 2019-02-13 13:01:46 -0500, Jonathan S. Katz wrote:
> So, in anticipation of push back, I also want to provide some stats. I
> looked at overall traffic since Jul 1, 2018 to yesterday. Traffic to any
> of the release notes in general was a very small chunk of it (< 0.5%).

Given the number of pages, that doesn't seem that small.  I personally
have hit broken /devel/release-xy.html links several times since the
hasty removal.


> From a user experience perspective, does it stink that we're potentially
> creating a bunch of 404s? Yes. Is it going to upset some people? Yes,
> and I'm sympathetic to that. However, looking at the overall numbers, I
> think we'll be ok, even if we don't have the redirects per Magnus'
> suggestion downthread (which I will also reply to).

We're going to break current/relase-xy.html links after every single
major release this way. This seems absurd.


I'm quite pissed at how haphazardly this was done.

Greetings,

Andres Freund


Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/13/19 1:44 PM, Andres Freund wrote:
> Hi,
>
> On 2019-02-13 13:01:46 -0500, Jonathan S. Katz wrote:
>> So, in anticipation of push back, I also want to provide some stats. I
>> looked at overall traffic since Jul 1, 2018 to yesterday. Traffic to any
>> of the release notes in general was a very small chunk of it (< 0.5%).
>
> Given the number of pages, that doesn't seem that small.  I personally
> have hit broken /devel/release-xy.html links several times since the
> hasty removal.

It's an incredibly small amount relative to other parts of the site. By
the above measure, it's statistically insignificant even with alpha of .01

To give it more context, it makes up 1% of all pages that are viewed in
the documentation. Looking from Jul 1, 2018, in the top 100 viewed doc
pages, #91 was the PG11 release, and nothing else. Basically, the major
release pushes the views of the release notes way up by a significant
portion.

Also, this did hurt my workflow as well when putting together the press
release. Given the release notes were not in devel, I ended up building
the docs from the REL_11_STABLE branch to put it together (rather than
parsing the SGML manually). It was inconvenient, but IMV the benefits
far outweigh the inconvenience.

>
>
>> From a user experience perspective, does it stink that we're potentially
>> creating a bunch of 404s? Yes. Is it going to upset some people? Yes,
>> and I'm sympathetic to that. However, looking at the overall numbers, I
>> think we'll be ok, even if we don't have the redirects per Magnus'
>> suggestion downthread (which I will also reply to).
>
> We're going to break current/relase-xy.html links after every single
> major release this way. This seems absurd.

We mentioned a way we can fix this -- having a map from /current/ (and
if need be /devel) to the proper version of the old release notes can be
done with some regular expressions, as Magnus said. It should not be too
difficult to do. I may even be able to get a POC done later today.

> I'm quite pissed at how haphazardly this was done.

We had discussion on this for months, and after some idling, we proposed
a solution and worked through it quickly. Sometimes you just need to rip
the band-aid off.

The nice thing about web is that you can test your assumptions and
iterate much more quickly than other ways to deliver software (e.g. a
database).

Thanks,

Jonathan


Attachment

Re: Release Notes Archive Patch

From
Andres Freund
Date:
Hi,

On February 13, 2019 8:03:03 PM GMT+01:00, "Jonathan S. Katz" <jkatz@postgresql.org> wrote:
>> I'm quite pissed at how haphazardly this was done.
>
>We had discussion on this for months, and after some idling, we
>proposed
>a solution and worked through it quickly. Sometimes you just need to
>rip
>the band-aid off.

Well, it was done under the heading of a modest proposal. But even leaving that aside, nothing, absolutely nothing,
preventedputting the web piece into place before breaking things. 

Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.


Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/13/19 2:10 PM, Andres Freund wrote:
> Hi,
>
> On February 13, 2019 8:03:03 PM GMT+01:00, "Jonathan S. Katz" <jkatz@postgresql.org> wrote:
>>> I'm quite pissed at how haphazardly this was done.
>>
>> We had discussion on this for months, and after some idling, we
>> proposed
>> a solution and worked through it quickly. Sometimes you just need to
>> rip
>> the band-aid off.
>
> But even leaving that aside, nothing, absolutely nothing, prevented putting the web piece into place before breaking
things.

I don't disagree with the above point. With that said, we are working to
get the web piece up prior to the majority of the links going 404.

I will go to work on a patch for current (which could be similar to
"devel") and see if I can have that ready. I will do it as a separate
patch from the one upthread.

Thanks,

Jonathan


Attachment

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/13/19 2:34 PM, Jonathan S. Katz wrote:
> On 2/13/19 2:10 PM, Andres Freund wrote:
>> Hi,
>>
>> On February 13, 2019 8:03:03 PM GMT+01:00, "Jonathan S. Katz" <jkatz@postgresql.org> wrote:
>>>> I'm quite pissed at how haphazardly this was done.
>>>
>>> We had discussion on this for months, and after some idling, we
>>> proposed
>>> a solution and worked through it quickly. Sometimes you just need to
>>> rip
>>> the band-aid off.
>>
>> But even leaving that aside, nothing, absolutely nothing, prevented putting the web piece into place before breaking
things.
>
> I don't disagree with the above point. With that said, we are working to
> get the web piece up prior to the majority of the links going 404.
>
> I will go to work on a patch for current (which could be similar to
> "devel") and see if I can have that ready. I will do it as a separate
> patch from the one upthread.

This turned out to be not so bad, please see attached.

Here's the way it works:

- If you are viewing release notes in documentation prior to 9.4, do nothing
- If you are viewing the release notes in documentation for 9.4 and
above and you are viewing the release notes in your version (e.g. 9.4.8
in 9.4), do nothing
- If you are viewing the release notes in docs for 9.4+ and you are
viewing a "mismatched" version (e.g. /docs/10/release-9-6-4.html),
permanent redirect (301) to the proper version (e.g.
/docs/9.6/release-6-4.html)

It accounts for version numbering changes and all the fun

Notes:

- Tested in multiple scenarios, also dealing with different versioning
number permutations as well (e.g. release-10.html, release-10-1.html,
release-9-6.html, release-9-6-1.html)
- Tested with current. I did not test with devel as I do not have
"devel" docs set up on my local. However, I am going to work on setting
that up
- Given docs on the site go back to 7.2, there is a stop that says if
you are viewing a release note older than 7.2, then view the releases
notes in 7.2

This can be pushed independently of the previous patch as it redirects
users to existing documentation.

Feedback welcome. Thanks,

Jonathan

Attachment

Re: Release Notes Archive Patch

From
Magnus Hagander
Date:
On Wed, Feb 13, 2019 at 2:13 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
Hi,

Attached is a proposed patch for archiving the release notes on the
website. The patch builds the release note archive without modifying the
schema and works with old and new releases.

It works as such:

- Find all release notes that have been loaded into pgweb (a process
that occurs each release when we load the documentation)
- Find the last modified release notes for each version. This accounts
for the old docs, e.g. there is a copy of the 9.2.8 notes in the 9.3.9
release, so we will render those
- There is a list of all release notes. If you click into it, you will
see the version specific release notes

Known adjustments to be made:

- The URLs on each release note page are broken. I have to figure out
how the best way to change the relative path in the URLs that are on the
page as it is going from the page root (/docs/release) instead of the
site root. This _should_ be easy
- Even with above fix, some URLs will be broken in them, i.e. references
to other release notes
- PostgreSQL v1 + v0: I need to relabel this for Postgres95
- Also, some of the earlier versions do not have the proper release
numbering (e.g. 1.03)

I'm hoping we can put this out with the upcoming release, but am ok with
doing this shortly thereafter.

Comments on patch and approach welcome.


A few comments on this one:

1. It loses the "go to next minor" and "go to previous minor" links in the previous release notes. Those should be easy enough to get back, but should be done. E.g. if I'm on 9.6.3, I want links to 9.6.2 and 9.6.4. And we should also use the change to make it more "explicit" and *also* have a link to 9.6.0 on all of them amonst the links themselves.

2b. In fact, we should use this chance to create a proper set of links, e.g. when you get to release notes for 9.6.0, it should right then and there provide a list of all the minor releases I think -- to avoid having to go back.

2. Links within the docs are just plain broken? E.g. the 9.6.0 release notes link to /docs/release/9.6.0/app-pg-dumpall.html which clearly doesn't exist. IIRC we discussed something around this, but it seems to have never made it into the patches? (It is listed in your known adjustments to be made above, but I could also not find a newer patch?)

3. That SQL query really needs some documentation, because it takes quite a bit of reading to figure out what it really means. And while it works on my dev install, it fails when i test-ran it on the current database with the error "ERROR:  invalid input syntax for type numeric: "prior". Most likely it comes from there being a file called release-prior.html in versions 9.4-11.0 now. 

It should also be reviewed for the fact that you are pulling back *all* the documentation including *all* the contents in order to render a list of versions...That gets annoyingly expensive when you actually have the full set of documentation.

4. The "jump to..." list should probably be split between supported and unsupported versions, to keep it a bit less.

5. In general, there's a huge amount of numbers on the index page. I'm not sure what's a good way to format it better, I normally leave that to you :) But the page is not very friendly at this point with the massive list of numbers you have to scroll through.

//Magnus

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
Hi,

On 2/19/19 5:42 AM, Magnus Hagander wrote:

> A few comments on this one:
>
> 1. It loses the "go to next minor" and "go to previous minor" links in
> the previous release notes. Those should be easy enough to get back, but
> should be done. E.g. if I'm on 9.6.3, I want links to 9.6.2 and 9.6.4.
> And we should also use the change to make it more "explicit" and *also*
> have a link to 9.6.0 on all of them amonst the links themsel
>
> 2b. In fact, we should use this chance to create a proper set of links,
> e.g. when you get to release notes for 9.6.0, it should right then and
> there provide a list of all the minor releases I think -- to avoid
> having to go back.

So there could be an issue with that as we would have to carry the
lag/lead somehow. Not insurmountable, but given this is minimally
invasive, it would take a bit more complicating of that SQL query that
was just simplified.

>
> 2. Links within the docs are just plain broken? E.g. the 9.6.0 release
> notes link to /docs/release/9.6.0/app-pg-dumpall.html which clearly
> doesn't exist. IIRC we discussed something around this, but it seems to
> have never made it into the patches? (It is listed in your known
> adjustments to be made above, but I could also not find a newer patch?)

I could not reproduce this. The links work fine for me.

>
> 3. That SQL query really needs some documentation, because it takes
> quite a bit of reading to figure out what it really means. And while it
> works on my dev install, it fails when i test-ran it on the current
> database with the error "ERROR:  invalid input syntax for type numeric:
> "prior". Most likely it comes from there being a file called
> release-prior.html in versions 9.4-11.0 now. 
>
> It should also be reviewed for the fact that you are pulling back *all*
> the documentation including *all* the contents in order to render a list
> of versions...That gets annoyingly expensive when you actually have the
> full set of documentation.

Per off-list discussion, Magnus added a simpler query that was provided.

>
> 4. The "jump to..." list should probably be split between supported and
> unsupported versions, to keep it a bit less.

Well, with the below, I don't think we need that.

>
> 5. In general, there's a huge amount of numbers on the index page. I'm
> not sure what's a good way to format it better, I normally leave that to
> you :) But the page is not very friendly at this point with the massive
> list of numbers you have to scroll through.

We could list out the major versions, then have the minor versions. I'm
not opposed to that. See attached.


Jonathan

Attachment

Re: Release Notes Archive Patch

From
Magnus Hagander
Date:


On Thu, Feb 28, 2019 at 12:30 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
Hi,

On 2/19/19 5:42 AM, Magnus Hagander wrote:

> A few comments on this one:
>
> 1. It loses the "go to next minor" and "go to previous minor" links in
> the previous release notes. Those should be easy enough to get back, but
> should be done. E.g. if I'm on 9.6.3, I want links to 9.6.2 and 9.6.4.
> And we should also use the change to make it more "explicit" and *also*
> have a link to 9.6.0 on all of them amonst the links themsel
>
> 2b. In fact, we should use this chance to create a proper set of links,
> e.g. when you get to release notes for 9.6.0, it should right then and
> there provide a list of all the minor releases I think -- to avoid
> having to go back.

So there could be an issue with that as we would have to carry the
lag/lead somehow. Not insurmountable, but given this is minimally
invasive, it would take a bit more complicating of that SQL query that
was just simplified.

Well, for "previous" you wouldn't need SQL :) But yes, you'd need a "do we have release notes for the next version" in there. But you only need those on the one-page views, so it's really not that bad.

I think it's worth it, it's definitely something *I* use a lot in the old interface.

(And as a minor note, for the query, I don't think adding major/minor versions in the WHERE clause is enough to make it fast. You need to push it down into the actual filename in the innermost where-clause, to make sure we get an index scan. Shouldn't be too hard)

> 2. Links within the docs are just plain broken? E.g. the 9.6.0 release
> notes link to /docs/release/9.6.0/app-pg-dumpall.html which clearly
> doesn't exist. IIRC we discussed something around this, but it seems to
> have never made it into the patches? (It is listed in your known
> adjustments to be made above, but I could also not find a newer patch?)

I could not reproduce this. The links work fine for me.

It could be because I was on the wrong version of the patch. They do work in the patch you sent now.

 
> 3. That SQL query really needs some documentation, because it takes
> quite a bit of reading to figure out what it really means. And while it
> works on my dev install, it fails when i test-ran it on the current
> database with the error "ERROR:  invalid input syntax for type numeric:
> "prior". Most likely it comes from there being a file called
> release-prior.html in versions 9.4-11.0 now. 
>
> It should also be reviewed for the fact that you are pulling back *all*
> the documentation including *all* the contents in order to render a list
> of versions...That gets annoyingly expensive when you actually have the
> full set of documentation.

Per off-list discussion, Magnus added a simpler query that was provided.

>
> 4. The "jump to..." list should probably be split between supported and
> unsupported versions, to keep it a bit less.

Well, with the below, I don't think we need that.

>
> 5. In general, there's a huge amount of numbers on the index page. I'm
> not sure what's a good way to format it better, I normally leave that to
> you :) But the page is not very friendly at this point with the massive
> list of numbers you have to scroll through.

We could list out the major versions, then have the minor versions. I'm
not opposed to that. See attached.
 
Hmm. I'm still not sure I like that big long list of numbers offhand.

In fact -- do we need a separate page? What if we just added them to /docs/manual/ and /docs/manual/archive/?

And then we highlight the release notes of the latest version directly on /docs/. 

And maybe add a link in the "online manuals" section out on the righthand side of /docs/?

Thoughts?

--

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/28/19 7:19 AM, Magnus Hagander wrote:
>
>
> On Thu, Feb 28, 2019 at 12:30 AM Jonathan S. Katz <jkatz@postgresql.org
> <mailto:jkatz@postgresql.org>> wrote:

>     So there could be an issue with that as we would have to carry the
>     lag/lead somehow. Not insurmountable, but given this is minimally
>     invasive, it would take a bit more complicating of that SQL query that
>     was just simplified.
>
>
> Well, for "previous" you wouldn't need SQL :) But yes, you'd need a "do
> we have release notes for the next version" in there. But you only need
> those on the one-page views, so it's really not that ba
>
> I think it's worth it, it's definitely something *I* use a lot in the
> old interface.

I'll look into adding it then.

>
> (And as a minor note, for the query, I don't think adding major/minor
> versions in the WHERE clause is enough to make it fast. You need to push
> it down into the actual filename in the innermost where-clause, to make
> sure we get an index scan. Shouldn't be too hard)

Yeah...it kind of makes constructing the query a bit uglier, which is
why I avoided it. I'm happy to do it, but I think that given a) it's not
a lot of rows and b) it's going to be behind varnish most of the time
anyway that I'm not too concerned.

As a compromise, I can come up with a version that does this, and we can
decide what to do from there.

>
>     > 2. Links within the docs are just plain broken? E.g. the 9.6.0 release
>     > notes link to /docs/release/9.6.0/app-pg-dumpall.html which clearly
>     > doesn't exist. IIRC we discussed something around this, but it
>     seems to
>     > have never made it into the patches? (It is listed in your known
>     > adjustments to be made above, but I could also not find a newer
>     patch?)
>
>     I could not reproduce this. The links work fine for me.
>
>
> It could be because I was on the wrong version of the patch. They do
> work in the patch you sent now.

Great.

>     > 5. In general, there's a huge amount of numbers on the index page. I'm
>     > not sure what's a good way to format it better, I normally leave
>     that to
>     > you :) But the page is not very friendly at this point with the
>     massive
>     > list of numbers you have to scroll through.
>
>     We could list out the major versions, then have the minor versions. I'm
>     not opposed to that. See attached.
>
>  
> Hmm. I'm still not sure I like that big long list of numbers offhand.

Why? It's in the specific version that you're looking for. It also makes
it easy to find the version you exactly want, instead of having to click
previous/next to find it. It's no different than:

https://www.postgresql.org/docs/9.4/release.html

In fact, it's a similar workflow:

1. Pick your major version
2. Pick your minor version
3. Enjoy a helping of release notes.

If anything, it's one less step because you don't have to go into the
specific documentation, click "Release Notes" and find them.

>
> In fact -- do we need a separate page? What if we just added them to
> /docs/manual/ and /docs/manual/archive/?

I strongly feel we need a separate page. It makes them so much easier to
find, and given all the hubbub let's make it easy for our users to find
the release notes, not buried somewhere.

> And then we highlight the release notes of the latest version directly
> on /docs/. 
>
> And maybe add a link in the "online manuals" section out on the
> righthand side of /docs/?

I was actually thinking in the release notes themselves, we add a right
nav that shows all the minor versions for the particular major version
of the release notes.

I'll post another revision soon.

Thanks,

Jonathan


Attachment

Re: Release Notes Archive Patch

From
Magnus Hagander
Date:
On Thu, Feb 28, 2019 at 4:15 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
On 2/28/19 7:19 AM, Magnus Hagander wrote:
>
>
> On Thu, Feb 28, 2019 at 12:30 AM Jonathan S. Katz <jkatz@postgresql.org
> <mailto:jkatz@postgresql.org>> wrote:

>     So there could be an issue with that as we would have to carry the
>     lag/lead somehow. Not insurmountable, but given this is minimally
>     invasive, it would take a bit more complicating of that SQL query that
>     was just simplified.
>
>
> Well, for "previous" you wouldn't need SQL :) But yes, you'd need a "do
> we have release notes for the next version" in there. But you only need
> those on the one-page views, so it's really not that ba
>
> I think it's worth it, it's definitely something *I* use a lot in the
> old interface.

I'll look into adding it then.

>
> (And as a minor note, for the query, I don't think adding major/minor
> versions in the WHERE clause is enough to make it fast. You need to push
> it down into the actual filename in the innermost where-clause, to make
> sure we get an index scan. Shouldn't be too hard)

Yeah...it kind of makes constructing the query a bit uglier, which is
why I avoided it. I'm happy to do it, but I think that given a) it's not
a lot of rows and b) it's going to be behind varnish most of the time
anyway that I'm not too concerned.

As a compromise, I can come up with a version that does this, and we can
decide what to do from there.

:) Sounds like a plan.

 
>     > 2. Links within the docs are just plain broken? E.g. the 9.6.0 release
>     > notes link to /docs/release/9.6.0/app-pg-dumpall.html which clearly
>     > doesn't exist. IIRC we discussed something around this, but it
>     seems to
>     > have never made it into the patches? (It is listed in your known
>     > adjustments to be made above, but I could also not find a newer
>     patch?)
>
>     I could not reproduce this. The links work fine for me.
>
>
> It could be because I was on the wrong version of the patch. They do
> work in the patch you sent now.

Great.

>     > 5. In general, there's a huge amount of numbers on the index page. I'm
>     > not sure what's a good way to format it better, I normally leave
>     that to
>     > you :) But the page is not very friendly at this point with the
>     massive
>     > list of numbers you have to scroll through.
>
>     We could list out the major versions, then have the minor versions. I'm
>     not opposed to that. See attached.
>
>  
> Hmm. I'm still not sure I like that big long list of numbers offhand.

Why? It's in the specific version that you're looking for. It also makes
it easy to find the version you exactly want, instead of having to click
previous/next to find it. It's no different than:

https://www.postgresql.org/docs/9.4/release.html

I don't like that one either :)

I prefer compating it to the rest of the site, in particular after the work Sarah and you put in there. And compared to that, I think it feels a bit off.

In the end, I'll be happy to defer to your judgement, just wanted to make sure you had heard my comments :)

 
In fact, it's a similar workflow:

1. Pick your major version
2. Pick your minor version
3. Enjoy a helping of release notes.

If anything, it's one less step because you don't have to go into the
specific documentation, click "Release Notes" and find them.

Right. I'm just saying you can make it even better :)


> In fact -- do we need a separate page? What if we just added them to
> /docs/manual/ and /docs/manual/archive/?

I strongly feel we need a separate page. It makes them so much easier to
find, and given all the hubbub let's make it easy for our users to find
the release notes, not buried somewhere.

Fair enough.


> And then we highlight the release notes of the latest version directly
> on /docs/. 
>
> And maybe add a link in the "online manuals" section out on the
> righthand side of /docs/?

I was actually thinking in the release notes themselves, we add a right
nav that shows all the minor versions for the particular major version
of the release notes.

Ah yeah, that could be pretty good.
 
--

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 2/28/19 2:12 PM, Magnus Hagander wrote:
> On Thu, Feb 28, 2019 at 4:15 PM Jonathan S. Katz <jkatz@postgresql.org
> <mailto:jkatz@postgresql.org>> wrote:
>
>     On 2/28/19 7:19 AM, Magnus Hagander wrote:

>     > Well, for "previous" you wouldn't need SQL :) But yes, you'd need
>     a "do
>     > we have release notes for the next version" in there. But you only
>     need
>     > those on the one-page views, so it's really not that ba
>     >
>     > I think it's worth it, it's definitely something *I* use a lot in the
>     > old interface.
>
>     I'll look into adding it then.

Added. Yay window functions :) It adds a bit more complexity to the
query (see below), but it lets people browse more easily, so yay.

>
>     >
>     > (And as a minor note, for the query, I don't think adding major/minor
>     > versions in the WHERE clause is enough to make it fast. You need
>     to push
>     > it down into the actual filename in the innermost where-clause, to
>     make
>     > sure we get an index scan. Shouldn't be too hard)
>
>     Yeah...it kind of makes constructing the query a bit uglier, which is
>     why I avoided it. I'm happy to do it, but I think that given a) it's not
>     a lot of rows and b) it's going to be behind varnish most of the time
>     anyway that I'm not too concerned.
>
>     As a compromise, I can come up with a version that does this, and we can
>     decide what to do from there.
>
>
> :) Sounds like a plan.

...so if we want to keep it to one query + support lead and lag, this
does not work out. Additionally, if we try to look it up over a major
version, we end up losing out on all of the legacy release notes from
9.3 & older. It also gets complicated when you deal with release notes
for versions of documentation we don't have loaded (e.g. 6.1.x).

However, we can at least limit the search to all release notes that were
loaded from 9.3 and newer, as 9.3 will contain all of the historical
release notes.

If we really want to use the index, we can add some magic to to check
which version we're looking up (e.g. 6.1.1) and then map it to the last
known version that had all docs (i.e. 9.3).

IMHO it's creating more trouble than its worth. The # of rows is small
(~2200, and with the 9.3 restriction ~1500), it does not grow much, and
these pages will be cached the vast majority of the time. Yes, it's wide
due to "content" but again, I'm not that concerned.


> I don't like that one either :)
>
> I prefer compating it to the rest of the site, in particular after the
> work Sarah and you put in there. And compared to that, I think it feels
> a bit off.
>
> In the end, I'll be happy to defer to your judgement, just wanted to
> make sure you had heard my comments :)

So I hopefully met somewhere in the middle on this: I brought it back
down to two pages, and had an accordion on the first page:

1. Click the major version, a list of the minor versions in that major
pop out
2. Select the minor version, you're taken to the release notes. There
are some controls to navigate to diff minor versions within that version.

I have some ideas for other UX that make it nicer to display (something
more "card" based), but in the interest of time I'd prefer we get this up.

>
>     > And then we highlight the release notes of the latest version directly
>     > on /docs/. 
>     >
>     > And maybe add a link in the "online manuals" section out on the
>     > righthand side of /docs/?
>
>     I was actually thinking in the release notes themselves, we add a right
>     nav that shows all the minor versions for the particular major version
>     of the release notes.
>
>
> Ah yeah, that could be pretty good.

This is done. Latest version attached.

I also added a new Django filter given there was some code I kept
copying-and-pasting around the minor version display formatting.

If there is nothing that's grossly flawed, I'd like to get this pushed.
The two things I want us to be happy with are a) the query and b) the
URLs, as everything else can conceivably change (well, the query can
change, but I want us to be happy with how we are generating it).

Thanks,

Jonathan

Attachment

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 3/2/19 12:22 PM, Jonathan S. Katz wrote:
This is done. Latest version attached.

Per in-person patch review, attached is the latest version.

Noted changes:

1. Fixed bug with previous/next in template
2. Only load the "content" when it is needed, i.e. when looking at the
specific release notes for a (major,minor) version pair
3. Use some arrows to make the collapsible list look more..collapsible
4. Updated some comments

Thanks,

Jonathan

Attachment

Re: Release Notes Archive Patch

From
Magnus Hagander
Date:
On Fri, Mar 8, 2019 at 11:21 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
On 3/2/19 12:22 PM, Jonathan S. Katz wrote:
This is done. Latest version attached.

Per in-person patch review, attached is the latest version.

Noted changes:

1. Fixed bug with previous/next in template
2. Only load the "content" when it is needed, i.e. when looking at the
specific release notes for a (major,minor) version pair
3. Use some arrows to make the collapsible list look more..collapsible
4. Updated some comments

LGTM! 

--

Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 3/9/19 11:10 AM, Magnus Hagander wrote:
> On Fri, Mar 8, 2019 at 11:21 AM Jonathan S. Katz <jkatz@postgresql.org
> <mailto:jkatz@postgresql.org>> wrote:
>
>     On 3/2/19 12:22 PM, Jonathan S. Katz wrote:
>     This is done. Latest version attached.
>
>     Per in-person patch review, attached is the latest version.
>
>     Noted changes:
>
>     1. Fixed bug with previous/next in template
>     2. Only load the "content" when it is needed, i.e. when looking at the
>     specific release notes for a (major,minor) version pair
>     3. Use some arrows to make the collapsible list look more..collapsible
>     4. Updated some comments
>
>
> LGTM!

Thanks for the review. This has been pushed.

Jonathan


Attachment

Re: Release Notes Archive Patch

From
Tom Lane
Date:
"Jonathan S. Katz" <jkatz@postgresql.org> writes:
> On 3/9/19 11:10 AM, Magnus Hagander wrote:
>> LGTM! 

> Thanks for the review. This has been pushed.

Looks good to me too.  I've updated the current-branch release note
sources to point to this archive.

            regards, tom lane


Re: Release Notes Archive Patch

From
"Jonathan S. Katz"
Date:
On 3/9/19 6:43 PM, Tom Lane wrote:
> "Jonathan S. Katz" <jkatz@postgresql.org> writes:
>> On 3/9/19 11:10 AM, Magnus Hagander wrote:
>>> LGTM!
>
>> Thanks for the review. This has been pushed.
>
> Looks good to me too.  I've updated the current-branch release note
> sources to point to this archive.

Andres noted a bug to me where if you went to a URL like:

https://www.postgresql.org/docs/devel/release-9-6.html

it would 404. I pushed a fix that redirects the user to the correct page.

Thanks,

Jonathan


Attachment