Thread: Can extension build own SGML document?
Hello, The pgxs makefile (pgxs.ml) says: # DOCS -- random files to install under $PREFIX/doc/$MODULEDIR It is a bunch of files to be copied to document directory, however, not looks like a variable to specify SGML source as PostgreSQL core doing. Do we have way to build SGML source of extension using makefile? Thanks, -- NEC Business Creation Division / PG-Strom Project KaiGai Kohei <kaigai@ak.jp.nec.com>
On 9/14/15 5:35 AM, Kouhei Kaigai wrote: > Hello, > > The pgxs makefile (pgxs.ml) says: > > # DOCS -- random files to install under $PREFIX/doc/$MODULEDIR > > It is a bunch of files to be copied to document directory, however, > not looks like a variable to specify SGML source as PostgreSQL core > doing. > > Do we have way to build SGML source of extension using makefile? No. We could, but a project that's large enough to want this kind of thing will likely have its own ideas and requirements, so it unclear how difficult it would be to support that.
Peter Eisentraut wrote: > On 9/14/15 5:35 AM, Kouhei Kaigai wrote: > > Hello, > > > > The pgxs makefile (pgxs.ml) says: > > > > # DOCS -- random files to install under $PREFIX/doc/$MODULEDIR > > > > It is a bunch of files to be copied to document directory, however, > > not looks like a variable to specify SGML source as PostgreSQL core > > doing. > > > > Do we have way to build SGML source of extension using makefile? > > No. > > We could, but a project that's large enough to want this kind of thing > will likely have its own ideas and requirements, so it unclear how > difficult it would be to support that. FWIW bdr has such a thing. Mostly, they replaced postgres.sgml with bdr.sgml and stripped down and edited the Makefile to match. The resulting doc is a standalone "book" (in docbook parlance). You can see the source code here http://git.postgresql.org/gitweb/?p=2ndquadrant_bdr.git;a=tree;f=doc;hb=HEAD Note there's no support for anything other than HTML --- no PDF, no EPUB, etc. The resulting HTML is published as http://bdr-project.org/docs/stable/ I think the only way upstream Postgres could offer this is as a way to build a separate "book", i.e. not a chapter/section within the main book. I think it would require huge complications in doc/src/sgml's Makefile. Not sure it's worthwhile. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Sep 15, 2015 at 6:01 AM, Alvaro Herrera wrote: > I think the only way upstream Postgres could offer this is as a way to > build a separate "book", i.e. not a chapter/section within the main > book. I think it would require huge complications in doc/src/sgml's > Makefile. Not sure it's worthwhile. I am not sure either, and as each project/developer have always different requirements I am convinced that this is going to be finished with enforced rules in Makefile rules for sure, so it is really unclear what would be the potential benefit to have a centralized facility. Take for example man pages, those should not be installed in share/doc/extension/ which is the default path, but in $(DESTDIR)$(mandir)/man1... -- Michael
On 9/14/15 6:06 PM, Michael Paquier wrote: > On Tue, Sep 15, 2015 at 6:01 AM, Alvaro Herrera wrote: >> I think the only way upstream Postgres could offer this is as a way to >> build a separate "book", i.e. not a chapter/section within the main >> book. I think it would require huge complications in doc/src/sgml's >> Makefile. Not sure it's worthwhile. I'm not sure SGML is the way to go anymore anyways. Asciidoc offers a lot of what our SGML does in a much easier to support toolchain. It's also natively supported by github, which makes it nice for others to view the output (see [1] as an exmaple). If asciidoc isn't powerful enough for what you need you can switch to asciidoctor which is even more powerful[2]. > I am not sure either, and as each project/developer have always > different requirements I am convinced that this is going to be > finished with enforced rules in Makefile rules for sure, so it is > really unclear what would be the potential benefit to have a > centralized facility. Take for example man pages, those should not be > installed in share/doc/extension/ which is the default path, but in > $(DESTDIR)$(mandir)/man1... I do wish there was better infrastructure for creating extensions, but I agree that the main project is not the way to handle that. There needs to be more than just a Makefile you can include too. In particular, the test framework is pretty ugly to deal with (unless you really like wading through regress.c diff output...) Bumping extension versions, creating distfiles and uploading to PGXN are also ripe for automation. pgxn-utils makes an attempt at this by creating an extension template for you to build from, but that's ultimately problematic because there's no way to pull in improvements to the overall infrastructure (such as how to create manpages). I think something you can pull in via a git subtree might work better, at least for a makefile framework. I've been meaning to create that but haven't found the time yet. [1] https://github.com/decibel/trunklet-format/blob/master/doc/trunklet-format.asc [2] http://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/ -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
Jim Nasby <Jim.Nasby@BlueTreble.com> writes:
> I'm not sure SGML is the way to go anymore anyways. Asciidoc offers a
> lot of what our SGML does in a much easier to support toolchain. It's
> also natively supported by github, which makes it nice for others to
> view the output (see [1] as an exmaple). If asciidoc isn't powerful
> enough for what you need you can switch to asciidoctor which is even
> more powerful[2].
AFAICT from a quick look at its documentation, asciidoc can produce
either html or docbook output; so as soon as you want something other
than html output (in particular, PDF), you're back to relying on the
exact same creaky docbook toolchain we use now. Only with one extra
dependency in front of it.
Personally I never look at anything but the HTML rendering, but I doubt
that dropping support for all other output formats would fly :-(
I do agree that the SGML toolchain is getting pretty long in the tooth
and we need to be looking for something else.
regards, tom lane
On 9/15/15 8:43 AM, Tom Lane wrote: > Jim Nasby <Jim.Nasby@BlueTreble.com> writes: >> I'm not sure SGML is the way to go anymore anyways. Asciidoc offers a >> lot of what our SGML does in a much easier to support toolchain. It's >> also natively supported by github, which makes it nice for others to >> view the output (see [1] as an exmaple). If asciidoc isn't powerful >> enough for what you need you can switch to asciidoctor which is even >> more powerful[2]. > > AFAICT from a quick look at its documentation, asciidoc can produce > either html or docbook output; so as soon as you want something other > than html output (in particular, PDF), you're back to relying on the > exact same creaky docbook toolchain we use now. Only with one extra > dependency in front of it. > > Personally I never look at anything but the HTML rendering, but I doubt > that dropping support for all other output formats would fly :-( > > I do agree that the SGML toolchain is getting pretty long in the tooth > and we need to be looking for something else. I wasn't thinking of trying to replace the Postgres toolchain, but... a2x (http://www.methods.co.nz/asciidoc/a2x.1.html) states that it can generate "PDF, EPUB, DVI, PS, LaTeX, XHTML (single page or chunked), man page, HTML Help or plain text formats using asciidoc(1) and other applications (see REQUISITES section). SOURCE_FILE can also be a DocBook file with an .xml extension." What I expect would be a lot more effort is actually converting all the SGML to asciidoc. A quick google search doesn't turn up anything promising. If the only bad part of our current toolchain is PDF then perhaps we should just rethink how that's being generated. Since a2x can take docbook input maybe that's the way to go. But my understanding was that we've modified docbook and that's where part of the pain is coming from? In that case a full-out migration to asciidoc might be better. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
Jim Nasby <Jim.Nasby@bluetreble.com> writes: > On 9/15/15 8:43 AM, Tom Lane wrote: >> AFAICT from a quick look at its documentation, asciidoc can produce >> either html or docbook output; so as soon as you want something other >> than html output (in particular, PDF), you're back to relying on the >> exact same creaky docbook toolchain we use now. Only with one extra >> dependency in front of it. > a2x (http://www.methods.co.nz/asciidoc/a2x.1.html) states that it can > generate "PDF, EPUB, DVI, PS, LaTeX, XHTML (single page or chunked), man > page, HTML Help or plain text formats using asciidoc(1) and other > applications (see REQUISITES section). SOURCE_FILE can also be a DocBook > file with an .xml extension." AFAICS, for all cases other than HTML output, the "other applications" are basically the docbook toolchain. > What I expect would be a lot more effort is actually converting all the > SGML to asciidoc. A quick google search doesn't turn up anything promising. Yeah, the cost of conversion means we're not likely to want to experiment to see what's better :-(. regards, tom lane
On Tue, Sep 15, 2015 at 9:43 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > AFAICT from a quick look at its documentation, asciidoc can produce > either html or docbook output; so as soon as you want something other > than html output (in particular, PDF), you're back to relying on the > exact same creaky docbook toolchain we use now. Only with one extra > dependency in front of it. > > Personally I never look at anything but the HTML rendering, but I doubt > that dropping support for all other output formats would fly :-( Just out of curiosity, really? I mean, I can't see that building a PDF of the documentation really has much value, and I don't know even what else we can build. Who in 2015 would use a PDF instead of HTML? (If there is somebody, that is fine. But I am curious who it is and why, because it seems to me like it would just be a nuisance.) -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On Tue, Sep 15, 2015 at 8:45 AM, Robert Haas <robertmhaas@gmail.com> wrote: > I mean, I can't see that building a PDF of the documentation really > has much value, and I don't know even what else we can build. Who in > 2015 would use a PDF instead of HTML? > > (If there is somebody, that is fine. But I am curious who it is and > why, because it seems to me like it would just be a nuisance.) Paper publication, say that for example: http://www.network-theory.co.uk/postgresql9/vol1a/ I imagine that this applies as well to other languages as it is easier to maintain a translation of the documentation by hacking the sgml code. -- Michael
Michael Paquier <michael.paquier@gmail.com> writes: > On Tue, Sep 15, 2015 at 8:45 AM, Robert Haas <robertmhaas@gmail.com> wrote: >> I mean, I can't see that building a PDF of the documentation really >> has much value, and I don't know even what else we can build. Who in >> 2015 would use a PDF instead of HTML? >> >> (If there is somebody, that is fine. But I am curious who it is and >> why, because it seems to me like it would just be a nuisance.) > Paper publication, say that for example: > http://www.network-theory.co.uk/postgresql9/vol1a/ Peter's done work somewhat recently on the output of EPUB format, so I assume he thinks there's still interest in that. regards, tom lane
Tom Lane wrote: > Michael Paquier <michael.paquier@gmail.com> writes: > > On Tue, Sep 15, 2015 at 8:45 AM, Robert Haas <robertmhaas@gmail.com> wrote: > >> I mean, I can't see that building a PDF of the documentation really > >> has much value, and I don't know even what else we can build. Who in > >> 2015 would use a PDF instead of HTML? > >> > >> (If there is somebody, that is fine. But I am curious who it is and > >> why, because it seems to me like it would just be a nuisance.) > > > Paper publication, say that for example: > > http://www.network-theory.co.uk/postgresql9/vol1a/ > > Peter's done work somewhat recently on the output of EPUB format, > so I assume he thinks there's still interest in that. Yeah, it's handy for e-readers. I don't think that's too nice for reference material, but it is for longer prose chapters we have. You can feed PDFs to e-readers but they don't work nearly as well, and if you want to use HTML there I think the only way is to convert the HTML to EPUB first, with worse results than creating the EPUB directly. As I recall, we also support Info format. That could be useful to Emacs fans, but I haven't heard of anyone actually using it. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 9/15/15 11:45 AM, Robert Haas wrote: > On Tue, Sep 15, 2015 at 9:43 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> AFAICT from a quick look at its documentation, asciidoc can produce >> either html or docbook output; so as soon as you want something other >> than html output (in particular, PDF), you're back to relying on the >> exact same creaky docbook toolchain we use now. Only with one extra >> dependency in front of it. >> >> Personally I never look at anything but the HTML rendering, but I doubt >> that dropping support for all other output formats would fly :-( > > Just out of curiosity, really? Man pages are in wide use, I think. > I mean, I can't see that building a PDF of the documentation really > has much value, and I don't know even what else we can build. Who in > 2015 would use a PDF instead of HTML? PDF is actually kind of neat for searches across the whole document.
On Wed, Sep 16, 2015 at 12:16 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > On 9/15/15 11:45 AM, Robert Haas wrote: >> On Tue, Sep 15, 2015 at 9:43 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> AFAICT from a quick look at its documentation, asciidoc can produce >>> either html or docbook output; so as soon as you want something other >>> than html output (in particular, PDF), you're back to relying on the >>> exact same creaky docbook toolchain we use now. Only with one extra >>> dependency in front of it. >>> >>> Personally I never look at anything but the HTML rendering, but I doubt >>> that dropping support for all other output formats would fly :-( >> >> Just out of curiosity, really? > > Man pages are in wide use, I think. Hmm, yeah. >> I mean, I can't see that building a PDF of the documentation really >> has much value, and I don't know even what else we can build. Who in >> 2015 would use a PDF instead of HTML? > > PDF is actually kind of neat for searches across the whole document. OK. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
On 9/15/15 10:13 AM, Tom Lane wrote: > Jim Nasby <Jim.Nasby@bluetreble.com> writes: >> On 9/15/15 8:43 AM, Tom Lane wrote: >>> AFAICT from a quick look at its documentation, asciidoc can produce >>> either html or docbook output; so as soon as you want something other >>> than html output (in particular, PDF), you're back to relying on the >>> exact same creaky docbook toolchain we use now. Only with one extra >>> dependency in front of it. > >> a2x (http://www.methods.co.nz/asciidoc/a2x.1.html) states that it can >> generate "PDF, EPUB, DVI, PS, LaTeX, XHTML (single page or chunked), man >> page, HTML Help or plain text formats using asciidoc(1) and other >> applications (see REQUISITES section). SOURCE_FILE can also be a DocBook >> file with an .xml extension." > > AFAICS, for all cases other than HTML output, the "other applications" > are basically the docbook toolchain. I just started looking at , which seems to be the newer way to handle asciidoc. Aside from being a lot faster than a2x/asciidoc, it can produce docbook natively. However... >> What I expect would be a lot more effort is actually converting all the >> SGML to asciidoc. A quick google search doesn't turn up anything promising. > > Yeah, the cost of conversion means we're not likely to want to experiment > to see what's better :-(. If the only concern is handling docbook format (which is what our SGML docs produce? Then https://github.com/asciidoctor/asciidoctor-fopub might be an option. It's intended for use with asciidoctor, but the README does state: "Using the asciidoctor-fopub project, you can convert any DocBook file into a nicely formatted PDF with nothing more than a Java runtime (JVM) and development kit (JDK). All the open source software required to perform the conversion is automatically fetched from the internet the first time you run it." So maybe it would allow removing some of more problematic parts of the toolchain? Also, if our SGML does produce docbook as an intermediate it might be possible to convert that to asciidoc via https://github.com/oreillymedia/docbook2asciidoc. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in Analytics, Data Architecture and PostgreSQL Data in Trouble? Get it in Treble! http://BlueTreble.com
On 14 October 2015 at 13:04, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
postgres@cbbrowne ~/p/d/s/sgml> grep DOCTYPE postgres.sgml master!?
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [
--
On 9/15/15 10:13 AM, Tom Lane wrote:Jim Nasby <Jim.Nasby@bluetreble.com> writes:On 9/15/15 8:43 AM, Tom Lane wrote:AFAICT from a quick look at its documentation, asciidoc can produce
either html or docbook output; so as soon as you want something other
than html output (in particular, PDF), you're back to relying on the
exact same creaky docbook toolchain we use now. Only with one extra
dependency in front of it.a2x (http://www.methods.co.nz/asciidoc/a2x.1.html) states that it can
generate "PDF, EPUB, DVI, PS, LaTeX, XHTML (single page or chunked), man
page, HTML Help or plain text formats using asciidoc(1) and other
applications (see REQUISITES section). SOURCE_FILE can also be a DocBook
file with an .xml extension."
AFAICS, for all cases other than HTML output, the "other applications"
are basically the docbook toolchain.
I just started looking at , which seems to be the newer way to handle asciidoc. Aside from being a lot faster than a2x/asciidoc, it can produce docbook natively. However...What I expect would be a lot more effort is actually converting all the
SGML to asciidoc. A quick google search doesn't turn up anything promising.
Yeah, the cost of conversion means we're not likely to want to experiment
to see what's better :-(.
If the only concern is handling docbook format (which is what our SGML docs produce? Then https://github.com/asciidoctor/asciidoctor-fopub might be an option. It's intended for use with asciidoctor, but the README does state:
"Using the asciidoctor-fopub project, you can convert any DocBook file into a nicely formatted PDF with nothing more than a Java runtime (JVM) and development kit (JDK). All the open source software required to perform the conversion is automatically fetched from the internet the first time you run it."
So maybe it would allow removing some of more problematic parts of the toolchain?
Also, if our SGML does produce docbook as an intermediate it might be possible to convert that to asciidoc via https://github.com/oreillymedia/docbook2asciidoc.
There's a misconception there...
Our SGML *is* DocBook. Natively, no translation needed.
DocBook is a document type, and our documentation is already written using that document type (DOCTYPE). Easily seen thus:
postgres@cbbrowne ~/p/d/s/sgml> grep DOCTYPE postgres.sgml master!?
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook V4.2//EN" [
No "intermediate"; we're directly already using DocBook version 4.2.
We'd presumably lose something by a DocBook ---> asciidoc ---> DocBook transition.
The trouble that we have is that what we have isn't a "DocBook file", but rather a fairly large set of files representing a DocBook document.
I'm not sure what improvement we'd get out of using asciidoctor-fopub.
There would be some merit to some remapping to transform "creaky old DocBook 4.2" (what we're using) to a newer version, perhaps biased towards XML, and have our toolset merge the bits into a big XML (in DocBook 5, presumably) file for processing using more modern DocBook tools.
I could probably build some DSSSL as helper (my HTML-to-DocBook DSSSL was highly incomplete, and nonetheless surprisingly widely referenced for years...), but we'd best be clear on what we think we're getting as improvement. Switching to a less expressive format is unlikely to be a win, however creaky the current DocBook/DSSSL tools are.
When confronted by a difficult problem, solve it by reducing it to the
question, "How would the Lone Ranger handle this?"
question, "How would the Lone Ranger handle this?"
Christopher Browne wrote: > There would be some merit to some remapping to transform "creaky old > DocBook 4.2" (what we're using) to a newer version, perhaps biased towards > XML, and have our toolset merge the bits into a big XML (in DocBook 5, > presumably) file for processing using more modern DocBook tools. As I recall, last time we discussed this, we found that the process for XML docbook took 10 times longer to produce the output files. The XML toolchain was just too young at the time. It would be nice to know whether it has aged well, i.e. has the runtime to build our docs improved? -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services