Thread: documentation is now XML
The documentation sources are now DocBook XML, not SGML. (The files are still named *.sgml. That's something to think about separately.) Besides the recent change to require full end tags, which is compatible between SGML and XML, being XML now requires empty-element tags to use the XML syntax like <xref linkend="foo"/>. When backpatching documentation changes, the extra slash needs to be removed, however. (The documentation build in backbranches will error if that is not done.) For the required tools, there are no new requirements, but you can drop docbook-sgml and opensp. The instructions at <https://www.postgresql.org/docs/devel/static/docguide-toolsets.html> will be up-to-date once built. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Thu, Nov 23, 2017 at 6:01 PM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote: > The documentation sources are now DocBook XML, not SGML. (The files are > still named *.sgml. That's something to think about separately.) Congratulations to you and Alexander ! That is what I waited for a long time. Now we could think about diagrams :) > > Besides the recent change to require full end tags, which is compatible > between SGML and XML, being XML now requires empty-element tags to use > the XML syntax like <xref linkend="foo"/>. When backpatching > documentation changes, the extra slash needs to be removed, however. > (The documentation build in backbranches will error if that is not done.) > > For the required tools, there are no new requirements, but you can drop > docbook-sgml and opensp. The instructions at > <https://www.postgresql.org/docs/devel/static/docguide-toolsets.html> > will be up-to-date once built. > > -- > Peter Eisentraut http://www.2ndQuadrant.com/ > PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> The documentation sources are now DocBook XML, not SGML. (The files are
> still named *.sgml. That's something to think about separately.)
I think we should have a discussion about whether it'd be smart
to convert the back branches' documentation to XML as well.
The main reason that I want to consider this is that back-patching
documentation fixes is going to be a huge problem if we don't.
Using the same doc-building toolchain across all branches seems like a win
as well. You could argue that switching build requirements in a minor
release is unfriendly to packagers; but those who build their own docs
have already had to adapt to the xsltproc/fop toolchain for v10, so
standardizing on that for 9.3-9.6 as well doesn't seem like it complicates
their lives. (Possibly we should canvass opinions on pgsql-packagers to
be sure of that.)
Also, we're way overdue for getting out from under the creaky TeX-based
toolchain for producing PDFs. Every time we make releases, I worry
whether we're going to get blindsided by its bugs with hotlinks that get
split across pages, since page breaks tend to vary in position depending
on exactly whose version of the toolchain and style sheets you build with.
I've also been living in fear of the day we hit some hardwired TeX limit
that we can't increase or work around. We've had to hack around such
limits repeatedly in the past (eg commit 944b41fc0). Now, it's probably
unlikely that growth of the release notes would be enough to put us over
the top in any back branch --- but if it did happen, we'd find out about
it at a point in the release cycle where there's very little margin for
error.
As against all that, there's our traditional conservatism about making
not-strictly-necessary changes in the back branches. Which is a strong
factor, for sure, but I think maybe here's a case for overriding it.
regards, tom lane
On Fri, Nov 24, 2017 at 12:21 AM, Oleg Bartunov <obartunov@gmail.com> wrote: > On Thu, Nov 23, 2017 at 6:01 PM, Peter Eisentraut > <peter.eisentraut@2ndquadrant.com> wrote: >> The documentation sources are now DocBook XML, not SGML. (The files are >> still named *.sgml. That's something to think about separately.) > > Congratulations to you and Alexander ! +1. -- Michael
On Fri, Nov 24, 2017 at 5:39 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes: >> The documentation sources are now DocBook XML, not SGML. (The files are >> still named *.sgml. That's something to think about separately.) > > I think we should have a discussion about whether it'd be smart > to convert the back branches' documentation to XML as well. > > The main reason that I want to consider this is that back-patching > documentation fixes is going to be a huge problem if we don't. Things like c29c578 and 1ff01b3 only found their way on HEAD. There is a bit of work needed here for back-branches. At the end I would vote for having consistent documentation on all active branches. > Using the same doc-building toolchain across all branches seems like a win > as well. You could argue that switching build requirements in a minor > release is unfriendly to packagers; but those who build their own docs > have already had to adapt to the xsltproc/fop toolchain for v10, so > standardizing on that for 9.3-9.6 as well doesn't seem like it complicates > their lives. (Possibly we should canvass opinions on pgsql-packagers to > be sure of that.) My own packaging is going to need some tweaks as well, but there is nothing difficult. A discussion is definitely deserved on -packagers, all don't have the same toolchain set. -- Michael
On 11/23/17 15:39, Tom Lane wrote: > I think we should have a discussion about whether it'd be smart > to convert the back branches' documentation to XML as well. My short answer to that is, I don't have time for that. I don't know if anyone else wants to investigate it. But it took us years to get to this point, and backpatching and back-testing all of that is just a lot of work that was not planned. Also, we will want to keep moving forward. As the title of the thread on -docs shows, some people want to move to DocBook 5. If every move like that will require backpatching everything, nobody is going to want to sign up for it anymore. > The main reason that I want to consider this is that back-patching > documentation fixes is going to be a huge problem if we don't. I understand that. I would like to think about a way, maybe a git hook or wrapper or something, to make that simpler. I haven't found any promising approach so far, however. > Using the same doc-building toolchain across all branches seems like a win > as well. You could argue that switching build requirements in a minor > release is unfriendly to packagers; but those who build their own docs > have already had to adapt to the xsltproc/fop toolchain for v10, so > standardizing on that for 9.3-9.6 as well doesn't seem like it complicates > their lives. (Possibly we should canvass opinions on pgsql-packagers to > be sure of that.) xsltproc has been required since 9.0 for the man pages, so that wouldn't be a problem. > Also, we're way overdue for getting out from under the creaky TeX-based > toolchain for producing PDFs. Makefile rules to build via FOP have been available since 9.4, so there is a backup plan there. However, there has also been a fair amount of tweaking to make things look good before switching in 10, so that would all have to be collected and analyzed. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 11/23/17 15:39, Tom Lane wrote:
>> I think we should have a discussion about whether it'd be smart
>> to convert the back branches' documentation to XML as well.
> My short answer to that is, I don't have time for that. I don't know if
> anyone else wants to investigate it. But it took us years to get to
> this point, and backpatching and back-testing all of that is just a lot
> of work that was not planned.
I thought that might be your answer :-(. I can't argue with it ---
if it's not a simple thing to back-patch, then it's unclear whether
the net annoyance over the next five years would be enough to justify
doing the work.
>> The main reason that I want to consider this is that back-patching
>> documentation fixes is going to be a huge problem if we don't.
> I understand that. I would like to think about a way, maybe a git hook
> or wrapper or something, to make that simpler. I haven't found any
> promising approach so far, however.
One thing we'd definitely better do is enable some buildfarm coverage.
AFAIK, the only buildfarm animal that's building the docs is guaibasaurus,
and it only seems to be doing that on HEAD. Since this has considerably
increased the risks of back-patching creating busted docs in the back
branches, we'd better spin up some back-branch testing as well.
regards, tom lane
On 11/28/2017 12:06 PM, Tom Lane wrote: > > One thing we'd definitely better do is enable some buildfarm coverage. > AFAIK, the only buildfarm animal that's building the docs is guaibasaurus, > and it only seems to be doing that on HEAD. Since this has considerably > increased the risks of back-patching creating busted docs in the back > branches, we'd better spin up some back-branch testing as well. > > crake builds the docs on all branches pretty regularly. I have changed its config so it builds them pretty much every build. Of course, it's only building the HTML docs. Do we need more than that? Continuously making PDFs seems a bit excessive. cheers andrew -- Andrew Dunstan https://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 11/28/2017 12:06 PM, Tom Lane wrote:
>> One thing we'd definitely better do is enable some buildfarm coverage.
>> AFAIK, the only buildfarm animal that's building the docs is guaibasaurus,
>> and it only seems to be doing that on HEAD. Since this has considerably
>> increased the risks of back-patching creating busted docs in the back
>> branches, we'd better spin up some back-branch testing as well.
> crake builds the docs on all branches pretty regularly. I have changed
> its config so it builds them pretty much every build.
Oh, excellent, thanks.
> Of course, it's only building the HTML docs. Do we need more than that?
> Continuously making PDFs seems a bit excessive.
I think sanity-checking the SGML/XML is the only part we need to do
regularly, so building HTML seems like plenty. The only point of
doing test PDF builds is to check for the link-crosses-a-page issue,
and I do not think it's worth worrying about that except very near
to a release. A problem we fix today might be gone tomorrow anyway.
regards, tom lane
Hello,
28.11.2017 20:06, Tom Lane wrote:
> Peter Eisentraut writes:
>> On 11/23/17 15:39, Tom Lane wrote:
>>> I think we should have a discussion about whether it'd be smart
>>> to convert the back branches' documentation to XML as well.
>> My short answer to that is, I don't have time for that. I don't know if
>> anyone else wants to investigate it. But it took us years to get to
>> this point, and backpatching and back-testing all of that is just a lot
>> of work that was not planned.
> I thought that might be your answer :-(. I can't argue with it ---
> if it's not a simple thing to back-patch, then it's unclear whether
> the net annoyance over the next five years would be enough to justify
> doing the work.
I can prepare such patches (scripts to generate them). In fact we
(Postgres Pro) perform such conversion (SGML->XML) on-fly when building
docs starting from 9.6. So it's not problem to convert *.sgml and
replace Makefile and *.xsl.
But I would prefer to perform the conversion when we finish the move on
11devel (renaming sgml to xml, maybe optimizing xsl's...).
Best regards,
------
Alexander Lakhin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: > Also, we're way overdue for getting out from under the creaky TeX-based > toolchain for producing PDFs. Every time we make releases, I worry > whether we're going to get blindsided by its bugs with hotlinks that get > split across pages, since page breaks tend to vary in position depending > on exactly whose version of the toolchain and style sheets you build with. > I've also been living in fear of the day we hit some hardwired TeX limit > that we can't increase or work around. We've had to hack around such > limits repeatedly in the past (eg commit 944b41fc0). Now, it's probably > unlikely that growth of the release notes would be enough to put us over > the top in any back branch --- but if it did happen, we'd find out about > it at a point in the release cycle where there's very little margin for > error. I am coming in late here, but I am not aware of any open source professional typesetting software that has output quality as good as TeX. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 01/23/2018 04:59 PM, Bruce Momjian wrote: > On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: >> Also, we're way overdue for getting out from under the creaky TeX-based >> toolchain for producing PDFs. Every time we make releases, I worry >> whether we're going to get blindsided by its bugs with hotlinks that get >> split across pages, since page breaks tend to vary in position depending >> >> I am coming in late here, but I am not aware of any open source >> professional typesetting software that has output quality as good as >> TeX. It has been ages since I have dealt with this but as I recall but moving to XML we can use XSLT to push to PDF. The resulting product is as good (if not better in some ways) to old fashion SGML->TEX->PDF. Here are a couple of articles: https://www.codeproject.com/Articles/595650/Creating-PDF-documents-from-XML https://www.ibm.com/developerworks/library/x-xmlxsltpdf/index.html JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org ***** Unless otherwise stated, opinions are my own. *****
On Tue, Jan 23, 2018 at 05:25:30PM -0800, Joshua Drake wrote: > On 01/23/2018 04:59 PM, Bruce Momjian wrote: > >On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: > >>Also, we're way overdue for getting out from under the creaky TeX-based > >>toolchain for producing PDFs. Every time we make releases, I worry > >>whether we're going to get blindsided by its bugs with hotlinks that get > >>split across pages, since page breaks tend to vary in position depending > >> > >>I am coming in late here, but I am not aware of any open source > >>professional typesetting software that has output quality as good as > >>TeX. > > It has been ages since I have dealt with this but as I recall but moving to > XML we can use XSLT to push to PDF. The resulting product is as good (if not > better in some ways) to old fashion SGML->TEX->PDF. > Here are a couple of articles: > > https://www.codeproject.com/Articles/595650/Creating-PDF-documents-from-XML > https://www.ibm.com/developerworks/library/x-xmlxsltpdf/index.html Uh, the first article says it uses FO.NET, and on their webpage they say: https://fonet.codeplex.com/ XSL-FO to PDF renderer written in managed C# code for the .NET framework FO.NET has been written for the Microsoft .NET Framework and is suitable for use from any .NET compatible language such as C#, VB.NET or C#. That sounds like a boatload of dependencies. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Bruce Momjian <bruce@momjian.us> writes:
> On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote:
>> Also, we're way overdue for getting out from under the creaky TeX-based
>> toolchain for producing PDFs.
> I am coming in late here, but I am not aware of any open source
> professional typesetting software that has output quality as good as
> TeX.
I like TeX as much as the next guy --- I wrote my thesis with it,
a long time ago --- but there's no denying that (a) it's got hard
limits we're approaching, (b) the downstream conversion to PDF is
buggy, and (c) nobody is working on fixing it.
regards, tom lane
On Tue, Jan 23, 2018 at 10:22:53PM -0500, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Thu, Nov 23, 2017 at 03:39:24PM -0500, Tom Lane wrote: > >> Also, we're way overdue for getting out from under the creaky TeX-based > >> toolchain for producing PDFs. > > > I am coming in late here, but I am not aware of any open source > > professional typesetting software that has output quality as good as > > TeX. > > I like TeX as much as the next guy --- I wrote my thesis with it, > a long time ago --- but there's no denying that (a) it's got hard > limits we're approaching, (b) the downstream conversion to PDF is > buggy, and (c) nobody is working on fixing it. All agreed, but what alternatives are being developed? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On Tuesday, January 23, 2018, Bruce Momjian <bruce@momjian.us> wrote:
On Tue, Jan 23, 2018 at 10:22:53PM -0500, Tom Lane wrote:
(a) it's got hard
> limits we're approaching,
All agreed, but what alternatives are being developed?
I seem to recall a proposal a while back to gain margin on some of the limits by pruning the release notes section down to at least this century and archiving putting the older ones elsewhere.
David J.
"David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Tuesday, January 23, 2018, Bruce Momjian <bruce@momjian.us> wrote:
>> All agreed, but what alternatives are being developed?
> I seem to recall a proposal a while back to gain margin on some of the
> limits by pruning the release notes section down to at least this century
> and archiving putting the older ones elsewhere.
Yeah; I did and still do think that's a good idea. But so far as the
toolchain is concerned, that's just a band-aid.
Anyway, we're on XML now, and it seems to be working fairly well.
I don't feel a need to revisit that. It's probably true that the
TeX-based toolchain was potentially capable of producing finer
typesetting results than the XML chain ... but, honestly, who's
printing the PG manual on dead trees anymore? I find the PDF output
to be mostly a legacy thing in the first place.
regards, tom lane
On 1/23/18 22:24, Bruce Momjian wrote: >> I like TeX as much as the next guy --- I wrote my thesis with it, >> a long time ago --- but there's no denying that (a) it's got hard >> limits we're approaching, (b) the downstream conversion to PDF is >> buggy, and (c) nobody is working on fixing it. > > All agreed, but what alternatives are being developed? We are using FOP now, which appears to give reasonable results. There are also commercial/proprietary/expensive XSL-FO processors that give even better results. Some of those are used in professional publishing. That's where things are now. The next wave in this area will be doing the whole thing with HTML and CSS. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Jan 24, 2018 at 08:21:21AM -0500, Peter Eisentraut wrote:
> On 1/23/18 22:24, Bruce Momjian wrote:
> >> I like TeX as much as the next guy --- I wrote my thesis with it,
> >> a long time ago --- but there's no denying that (a) it's got hard
> >> limits we're approaching, (b) the downstream conversion to PDF is
> >> buggy, and (c) nobody is working on fixing it.
> >
> > All agreed, but what alternatives are being developed?
>
> We are using FOP now, which appears to give reasonable results. There
> are also commercial/proprietary/expensive XSL-FO processors that give
> even better results. Some of those are used in professional publishing.
>
> That's where things are now. The next wave in this area will be doing
> the whole thing with HTML and CSS.
Ah, so we are using fop now:
https://xmlgraphics.apache.org/fop/
and it seems I have fop installed on my server already. So we are not
using TeX anymore for PG11+ docs?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On Tue, Jan 23, 2018 at 11:16:23PM -0500, Tom Lane wrote: > "David G. Johnston" <david.g.johnston@gmail.com> writes: > > On Tuesday, January 23, 2018, Bruce Momjian <bruce@momjian.us> wrote: > >> All agreed, but what alternatives are being developed? > > > I seem to recall a proposal a while back to gain margin on some of > > the limits by pruning the release notes section down to at least > > this century and archiving putting the older ones elsewhere. > > Yeah; I did and still do think that's a good idea. But so far as > the toolchain is concerned, that's just a band-aid. > > Anyway, we're on XML now, and it seems to be working fairly well. I > don't feel a need to revisit that. It's probably true that the > TeX-based toolchain was potentially capable of producing finer > typesetting results than the XML chain ... but, honestly, who's > printing the PG manual on dead trees anymore? I find the PDF output > to be mostly a legacy thing in the first place. Speaking of legacy things, do we want to see about deprecating LaTeX output for tables in psql? I haven't seen such a thing in production, but maybe I'm missing something important. Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David Fetter wrote: > Speaking of legacy things, do we want to see about deprecating LaTeX > output for tables in psql? I haven't seen such a thing in production, > but maybe I'm missing something important. I've used it to prepare slides with the Beamer package. It's not a thing I do all that often since I give less talks these days than I used to; but I do use it. I wouldn't want to see it go. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> David Fetter wrote:
>> Speaking of legacy things, do we want to see about deprecating LaTeX
>> output for tables in psql? I haven't seen such a thing in production,
>> but maybe I'm missing something important.
> I've used it to prepare slides with the Beamer package. It's not a
> thing I do all that often since I give less talks these days than I used
> to; but I do use it. I wouldn't want to see it go.
Yeah, I don't see the argument for removing that. It's not been any
particular maintenance problem AFAIR.
regards, tom lane
On Wed, Jan 24, 2018 at 10:06:49AM -0500, Tom Lane wrote: > Alvaro Herrera <alvherre@alvh.no-ip.org> writes: > > David Fetter wrote: > >> Speaking of legacy things, do we want to see about deprecating LaTeX > >> output for tables in psql? I haven't seen such a thing in production, > >> but maybe I'm missing something important. > > > I've used it to prepare slides with the Beamer package. It's not a > > thing I do all that often since I give less talks these days than I used > > to; but I do use it. I wouldn't want to see it go. > > Yeah, I don't see the argument for removing that. It's not been any > particular maintenance problem AFAIR. I see. It doesn't appear to have any TeX dependencies anyhow. Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
David, Alvaro, * Alvaro Herrera (alvherre@alvh.no-ip.org) wrote: > David Fetter wrote: > > > Speaking of legacy things, do we want to see about deprecating LaTeX > > output for tables in psql? I haven't seen such a thing in production, > > but maybe I'm missing something important. > > I've used it to prepare slides with the Beamer package. It's not a > thing I do all that often since I give less talks these days than I used > to; but I do use it. I wouldn't want to see it go. Same here.. Let's not remove things that people are using. Thanks! Stephen
Attachment
On Wed, Jan 24, 2018 at 10:49:38AM -0500, Stephen Frost wrote: > David, Alvaro, > > * Alvaro Herrera (alvherre@alvh.no-ip.org) wrote: > > David Fetter wrote: > > > > > Speaking of legacy things, do we want to see about deprecating > > > LaTeX output for tables in psql? I haven't seen such a thing in > > > production, but maybe I'm missing something important. > > > > I've used it to prepare slides with the Beamer package. It's not > > a thing I do all that often since I give less talks these days > > than I used to; but I do use it. I wouldn't want to see it go. > > Same here.. Let's not remove things that people are using. People are using it, so that's the end of the story. Sorry about the noise. Best, David. -- David Fetter <david(at)fetter(dot)org> http://fetter.org/ Phone: +1 415 235 3778 Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate
On Wed, Jan 24, 2018 at 04:53:57PM +0100, David Fetter wrote:
> On Wed, Jan 24, 2018 at 10:49:38AM -0500, Stephen Frost wrote:
> > David, Alvaro,
> >
> > * Alvaro Herrera (alvherre@alvh.no-ip.org) wrote:
> > > David Fetter wrote:
> > >
> > > > Speaking of legacy things, do we want to see about deprecating
> > > > LaTeX output for tables in psql? I haven't seen such a thing in
> > > > production, but maybe I'm missing something important.
> > >
> > > I've used it to prepare slides with the Beamer package. It's not
> > > a thing I do all that often since I give less talks these days
> > > than I used to; but I do use it. I wouldn't want to see it go.
> >
> > Same here.. Let's not remove things that people are using.
>
> People are using it, so that's the end of the story. Sorry about the
> noise.
Oh, yes, people are using it, particularly LaTeX longtable, which was
added in 2013:
commit b14f81bc9a65993129e93052634e358b310b8554
Author: Bruce Momjian <bruce@momjian.us>
Date: Thu Jan 17 11:39:38 2013 -0500
Add a latex-longtable output format to psql
latex longtable is more powerful than the 'tabular' output format
'latex' uses. Also add border=3 support to 'latex'.
I used to get bugged at conference about no longtable support, so I have
a personal stake in keeping it. :-)
Now if you want obscure, we support troff output too. :-)
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ As you are, so once was I. As I am, so you will be. +
+ Ancient Roman grave inscription +
On 1/24/18 09:45, Bruce Momjian wrote: > So we are not > using TeX anymore for PG11+ docs? as of PG10 -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi everyone,
Reading this thread, I got an impression that everyone would benefit from converting back branches to XML, but the main concern is lack of resources to complete this task. Are there any other issues that affect this decision? Looks like Aleksander Lakhin's offer to prepare patches was missed somehow as the discussion sidetracked to other issues:
I can prepare such patches (scripts to generate them). In fact we (Postgres Pro) perform such conversion (SGML->XML) on-fly when building docs starting from 9.6. So it's not problem to convert *.sgml and replace Makefile and *.xsl. But I would prefer to perform the conversion when we finish the move on 11devel (renaming sgml to xml, maybe optimizing xsl's...).
Do you think it's worth considering after May releases? I could also help with manual testing of the prepared patches if required. Supporting documentation and translation in several branches will be much easier if the sources are consistent.
-- Liudmila Mantrova Technical writer at Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On Mon, Apr 23, 2018 at 12:54:40PM +0300, Liudmila Mantrova wrote: > Hi everyone, > > Reading this thread, I got an impression that everyone would benefit from > converting back branches to XML, but the main concern is lack of resources to > complete this task. Are there any other issues that affect this decision? Looks > like Aleksander Lakhin's offer to prepare patches was missed somehow as the > discussion sidetracked to other issues: > > I can prepare such patches (scripts to generate them). In fact we (Postgres > Pro) perform such conversion (SGML->XML) on-fly when building docs starting > from 9.6. So it's not problem to convert *.sgml and replace Makefile and > *.xsl. But I would prefer to perform the conversion when we finish the move > on 11devel (renaming sgml to xml, maybe optimizing xsl's...). > > Do you think it's worth considering after May releases? I could also help with > manual testing of the prepared patches if required. Supporting documentation > and translation in several branches will be much easier if the sources are > consistent. Well, I will say that backpatching doc changes is more complex having SGML and XML in supported versions, particularly because of the different tag endings between them. Therefore, having all the docs be XML would be nice from my perspective. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
On 4/23/18 05:54, Liudmila Mantrova wrote: > Reading this thread, I got an impression that everyone would benefit > from converting back branches to XML, but the main concern is lack of > resources to complete this task. Are there any other issues that affect > this decision? Looks like Aleksander Lakhin's offer to prepare patches > was missed somehow as the discussion sidetracked to other issues That proposal seemed to indicate not only converting the source code to XML but also the build system to XSL. The latter is out of the question, I think. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Apr 27, 2018 at 11:00:36AM -0400, Peter Eisentraut wrote: > On 4/23/18 05:54, Liudmila Mantrova wrote: > > Reading this thread, I got an impression that everyone would benefit > > from converting back branches to XML, but the main concern is lack of > > resources to complete this task. Are there any other issues that affect > > this decision? Looks like Aleksander Lakhin's offer to prepare patches > > was missed somehow as the discussion sidetracked to other issues > > That proposal seemed to indicate not only converting the source code to > XML but also the build system to XSL. The latter is out of the > question, I think. Why is that? I thought people building current Postgres would already have the XML toolchain built to build older release documentation. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Bruce Momjian <bruce@momjian.us> writes:
> On Fri, Apr 27, 2018 at 11:00:36AM -0400, Peter Eisentraut wrote:
>> That proposal seemed to indicate not only converting the source code to
>> XML but also the build system to XSL. The latter is out of the
>> question, I think.
> Why is that? I thought people building current Postgres would already
> have the XML toolchain built to build older release documentation.
Yeah, in principle, anyone who's able to build the v10 branch's docs
should be ready for such a change. We had discussed polling the
pgsql-packagers list to see whether anyone would have a problem with
changing the docs build toolset for the pre-v10 branches, but I don't
think we actually asked them.
regards, tom lane
On 4/27/18 11:03, Bruce Momjian wrote: > On Fri, Apr 27, 2018 at 11:00:36AM -0400, Peter Eisentraut wrote: >> On 4/23/18 05:54, Liudmila Mantrova wrote: >>> Reading this thread, I got an impression that everyone would benefit >>> from converting back branches to XML, but the main concern is lack of >>> resources to complete this task. Are there any other issues that affect >>> this decision? Looks like Aleksander Lakhin's offer to prepare patches >>> was missed somehow as the discussion sidetracked to other issues >> >> That proposal seemed to indicate not only converting the source code to >> XML but also the build system to XSL. The latter is out of the >> question, I think. > > Why is that? Because there would be a thousand lines of tooling changes to be backported and thousands of pages of documentation to be checked manually that it doesn't create a mess (times branches times platforms). -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Apr 30, 2018 at 03:20:44PM -0400, Peter Eisentraut wrote: > On 4/27/18 11:03, Bruce Momjian wrote: > > On Fri, Apr 27, 2018 at 11:00:36AM -0400, Peter Eisentraut wrote: > >> On 4/23/18 05:54, Liudmila Mantrova wrote: > >>> Reading this thread, I got an impression that everyone would benefit > >>> from converting back branches to XML, but the main concern is lack of > >>> resources to complete this task. Are there any other issues that affect > >>> this decision? Looks like Aleksander Lakhin's offer to prepare patches > >>> was missed somehow as the discussion sidetracked to other issues > >> > >> That proposal seemed to indicate not only converting the source code to > >> XML but also the build system to XSL. The latter is out of the > >> question, I think. > > > > Why is that? > Because there would be a thousand lines of tooling changes to be > backported and thousands of pages of documentation to be checked > manually that it doesn't create a mess (times branches times platforms). Oh, OK. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
Hi Peter, Thank you for taking your time to comment on this long-discussed topic. I was not around when all the great work was done to implement XSLT transforms in branch 10 and convert branch 11 to XML, but judging by this thread (https://www.postgresql.org/message-id/flat/4ffd72d6-8ab6-37c6-d7d5-dfed8967c0fc%40gmail.com), much of the testing was automated and could be re-applied to older branches. Taking into account that both XSLT transforms and testing scripts can be reused for the most part, do you think that the benefits of having consistent source (easy back-porting of doc patches and one-time translation to all languages) in all the supported branches could potentially outweigh the inevitable conversion overhead? Are there any specific features that make branch 10 considerably different from its predecessors that we are missing? On 04/30/2018 10:20 PM, Peter Eisentraut wrote: > On 4/27/18 11:03, Bruce Momjian wrote: >> On Fri, Apr 27, 2018 at 11:00:36AM -0400, Peter Eisentraut wrote: >>> On 4/23/18 05:54, Liudmila Mantrova wrote: >>>> Reading this thread, I got an impression that everyone would benefit >>>> from converting back branches to XML, but the main concern is lack of >>>> resources to complete this task. Are there any other issues that affect >>>> this decision? Looks like Aleksander Lakhin's offer to prepare patches >>>> was missed somehow as the discussion sidetracked to other issues >>> That proposal seemed to indicate not only converting the source code to >>> XML but also the build system to XSL. The latter is out of the >>> question, I think. >> Why is that? > Because there would be a thousand lines of tooling changes to be > backported and thousands of pages of documentation to be checked > manually that it doesn't create a mess (times branches times platforms). > -- Liudmila Mantrova Technical writer at Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On 04/27/2018 06:36 PM, Tom Lane wrote:
To complete the picture of possible issues with older branches in XML, we posted a question in packager lists some time ago and didn't receive any objections. Just to keep record of all related questions in one place, here's the link:
Bruce Momjian <bruce@momjian.us> writes:On Fri, Apr 27, 2018 at 11:00:36AM -0400, Peter Eisentraut wrote:That proposal seemed to indicate not only converting the source code to XML but also the build system to XSL. The latter is out of the question, I think.Why is that? I thought people building current Postgres would already have the XML toolchain built to build older release documentation.Yeah, in principle, anyone who's able to build the v10 branch's docs should be ready for such a change. We had discussed polling the pgsql-packagers list to see whether anyone would have a problem with changing the docs build toolset for the pre-v10 branches, but I don't think we actually asked them. regards, tom lane
To complete the picture of possible issues with older branches in XML, we posted a question in packager lists some time ago and didn't receive any objections. Just to keep record of all related questions in one place, here's the link:
We totally understand the reluctance to volunteer personal time for manual testing of legacy branches - sad as it is that we may miss some benefits of the previous efforts. E.g. all styles and transforms have been stabilized for version 11, so they could be reused for older branches. As for testing the content conversion, our scripts can handle it in a fully automated way by comparing the original and the final outputs via the .txt format, so all possible differences will be caught.
It's unfortunate that we'll have to deal with different formats in the supported branches for several more years, but we at Postgres Professional are ready to accept any your decision on this matter for now.
-- Liudmila Mantrova Technical writer at Postgres Professional: http://www.postgrespro.com The Russian Postgres Company
On Wed, Jun 20, 2018 at 10:48:47AM +0300, Liudmila Mantrova wrote: > To complete the picture of possible issues with older branches in XML, we > posted a question in packager lists some time ago and didn't receive any > objections. Just to keep record of all related questions in one place, here's > the link: > > https://www.postgresql.org/message-id/flat/ > 06efc906-16f8-0cde-5bee-e3d5abfc00ba%40postgrespro.ru# > 06efc906-16f8-0cde-5bee-e3d5abfc00ba@postgrespro.ru > > We totally understand the reluctance to volunteer personal time for manual > testing of legacy branches - sad as it is that we may miss some benefits of the > previous efforts. E.g. all styles and transforms have been stabilized for > version 11, so they could be reused for older branches. As for testing the > content conversion, our scripts can handle it in a fully automated way by > comparing the original and the final outputs via the .txt format, so all > possible differences will be caught. > > It's unfortunate that we'll have to deal with different formats in the > supported branches for several more years, but we at Postgres Professional are > ready to accept any your decision on this matter for now. I had not thought of translations, but wouldn't all translations also need to switch from SGML to XML for back branches? That would be a lot of work. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
20.06.2018 15:03, Bruce Momjian wrote:
On Wed, Jun 20, 2018 at 10:48:47AM +0300, Liudmila Mantrova wrote:It's unfortunate that we'll have to deal with different formats in the supported branches for several more years, but we at Postgres Professional are ready to accept any your decision on this matter for now.I had not thought of translations, but wouldn't all translations also need to switch from SGML to XML for back branches? That would be a lot of work.
Hello Bruce,
I think there are two approaches here. First, if translation is performed directly in sgml (say, we have adminpack_fr.sgml), we'd have to convert the existing translation files to XML anyway - otherwise we'll have to start translating version 11 from scratch. So some tools/scripts are needed to perform the conversion. I don't think it would be difficult to use these tools to convert existing translation for version 10 (and 9.6, 9.5...) to XML and to continue working with it. And after the conversion is done we could maintain all the translations in a consistent way, using a single tool.
Second, if translation is performed indirectly, via the PO (as we do, e.g. see [1]), then the conversion is a matter of automatic producing .po from xml. We see no difficulties with it either.
So I see no extra work for translators as a consequence of backporting XML format except for reusing some tools, which are needed anyway.
[1] https://github.com/alexanderlaw/pg-doc-ru/blob/master/postgresql-doc/adminpack.po
Best regards,
------
Alexander Lakhin
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
On Wed, Jun 20, 2018 at 04:46:24PM +0300, Alexander Lakhin wrote: > 20.06.2018 15:03, Bruce Momjian wrote: > > On Wed, Jun 20, 2018 at 10:48:47AM +0300, Liudmila Mantrova wrote: > > It's unfortunate that we'll have to deal with different formats in the > supported branches for several more years, but we at Postgres Professional are > ready to accept any your decision on this matter for now. > > I had not thought of translations, but wouldn't all translations also > need to switch from SGML to XML for back branches? That would be a lot > of work. > > Hello Bruce, > > I think there are two approaches here. > > First, if translation is performed directly in sgml (say, we have > adminpack_fr.sgml), we'd have to convert the existing translation files to XML > anyway - otherwise we'll have to start translating version 11 from scratch. So > some tools/scripts are needed to perform the conversion. I don't think it would > be difficult to use these tools to convert existing translation for version 10 > (and 9.6, 9.5...) to XML and to continue working with it. And after the > conversion is done we could maintain all the translations in a consistent way, > using a single tool. > > Second, if translation is performed indirectly, via the PO (as we do, e.g. see > [1]), then the conversion is a matter of automatic producing .po from xml. We > see no difficulties with it either. > > So I see no extra work for translators as a consequence of backporting XML > format except for reusing some tools, which are needed anyway. Yes, if you can do the translation automated, you are right. However, in talking to Peter E, he seemed to say there was a lot of manual work, and obviously doing that only once for a translation is preferable. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + As you are, so once was I. As I am, so you will be. + + Ancient Roman grave inscription +
>> It's unfortunate that we'll have to deal with different formats in the >> supported branches for several more years, but we at Postgres Professional are >> ready to accept any your decision on this matter for now. > > I had not thought of translations, but wouldn't all translations also > need to switch from SGML to XML for back branches? That would be a lot > of work. Regarding Japanese translations, we only do translations for the latest stable branch (for now 10.x). I don't know about other language translations though. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp