Thread: [HACKERS] port of INSTALL file generation to XSLT
A further step toward getting rid of the DSSSL tool chain requirement, here is a patch to change the generation of the text INSTALL file to use XLST and Pandoc. The change to Pandoc is not essential to this change, but it creates much better looking output and simplifies the locale/encoding handling over using Lynx. We'll need to get Pandoc installed on borka and check that that version works as well as the one I have been using. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
On Sat, Dec 31, 2016 at 6:57 AM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
A further step toward getting rid of the DSSSL tool chain requirement,
here is a patch to change the generation of the text INSTALL file to use
XLST and Pandoc.
The change to Pandoc is not essential to this change, but it creates
much better looking output and simplifies the locale/encoding handling
over using Lynx.
Yeah, that seems a lot more clean than using Lynx.
We'll need to get Pandoc installed on borka and check that that version
works as well as the one I have been using.
Borka being a standard debian jessie install has Pandoc 1.12.4.2~dfsg-1+b14. Should be no problem installing that.
I ran a "make INSTALL" on a jessie box, and it comes out with some formatting still in the file, see attachment. That seems incorrect.
Attachment
On 12/31/16 07:33, Magnus Hagander wrote: > Borka being a standard debian jessie install has Pandoc > 1.12.4.2~dfsg-1+b14. Should be no problem installing that. > > I ran a "make INSTALL" on a jessie box, and it comes out with some > formatting still in the file, see attachment. That seems incorrect. It appears we need pandoc 1.13 to get the good output. This won't be available until Debian stretch. So for PG10, I propose the attached revised patch which keeps using lynx but uses xsltproc instead of jade. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
On Mon, Feb 27, 2017 at 4:55 PM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
On 12/31/16 07:33, Magnus Hagander wrote:
> Borka being a standard debian jessie install has Pandoc
> 1.12.4.2~dfsg-1+b14. Should be no problem installing that.
>
> I ran a "make INSTALL" on a jessie box, and it comes out with some
> formatting still in the file, see attachment. That seems incorrect.
It appears we need pandoc 1.13 to get the good output. This won't be
available until Debian stretch.
So for PG10, I propose the attached revised patch which keeps using lynx
but uses xsltproc instead of jade.
It is available for people not using debian though? Might it be worthwhile to make it dependent on the version of pandoc -- so use that method if people have the newer pandoc and fall back to lynx if they don't?
On 2/28/17 08:57, Magnus Hagander wrote: > It appears we need pandoc 1.13 to get the good output. This won't be > available until Debian stretch. > > So for PG10, I propose the attached revised patch which keeps using lynx > but uses xsltproc instead of jade. > > > It is available for people not using debian though? Might it be > worthwhile to make it dependent on the version of pandoc -- so use that > method if people have the newer pandoc and fall back to lynx if they don't? Well, this really only runs once every couple of months on borka and here or there for those building their own snapshot tarballs. I don't think we need to cater to a wide audience here. In fact, variety could be bad here: We don't want to find out that a tarball was rolled with the wrong variant. The pandoc change can be revisited independently. The main point right now is to get away from the DSSSL toolchain. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Mar 1, 2017 at 3:58 AM, Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
-- On 2/28/17 08:57, Magnus Hagander wrote:
> It appears we need pandoc 1.13 to get the good output. This won't be
> available until Debian stretch.
>
> So for PG10, I propose the attached revised patch which keeps using lynx
> but uses xsltproc instead of jade.
>
>
> It is available for people not using debian though? Might it be
> worthwhile to make it dependent on the version of pandoc -- so use that
> method if people have the newer pandoc and fall back to lynx if they don't?
Well, this really only runs once every couple of months on borka and
here or there for those building their own snapshot tarballs. I don't
think we need to cater to a wide audience here. In fact, variety could
be bad here: We don't want to find out that a tarball was rolled with
the wrong variant.
Good point. Let 's just go for it then.
On 3/1/17 20:23, Magnus Hagander wrote: > On Wed, Mar 1, 2017 at 3:58 AM, Peter Eisentraut > <peter.eisentraut@2ndquadrant.com > <mailto:peter.eisentraut@2ndquadrant.com>> wrote: > > On 2/28/17 08:57, Magnus Hagander wrote: > > It appears we need pandoc 1.13 to get the good output. This won't be > > available until Debian stretch. > > > > So for PG10, I propose the attached revised patch which keeps using lynx > > but uses xsltproc instead of jade. > > > > > > It is available for people not using debian though? Might it be > > worthwhile to make it dependent on the version of pandoc -- so use that > > method if people have the newer pandoc and fall back to lynx if they don't? > > Well, this really only runs once every couple of months on borka and > here or there for those building their own snapshot tarballs. I don't > think we need to cater to a wide audience here. In fact, variety could > be bad here: We don't want to find out that a tarball was rolled with > the wrong variant. > > > Good point. Let 's just go for it then. Committed that way. Thanks. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 2/27/17 10:55, Peter Eisentraut wrote: > It appears we need pandoc 1.13 to get the good output. This won't be > available until Debian stretch. I understand that borka is updated to stretch now. So we could give this another try. Updated patch attached. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachment
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation: tested, failed Hi! I tested this. Patch applied cleanly and INSTALL file was produced. Formatting looks differently from before, but I thinkthat it looks better. We lost central alignment of some headings, but many code/command snippets are now better/correctlyindented. So I think this is overall better. Documentation about documentation generation should be updated though. Please add pandoc to requirements here [1]. [1] https://www.postgresql.org/docs/devel/docguide-toolsets.html Mitar The new status of this patch is: Waiting on Author
On 09/01/2019 10:05, Mi Tar wrote: > I tested this. Patch applied cleanly and INSTALL file was produced. Formatting looks differently from before, but I thinkthat it looks better. We lost central alignment of some headings, but many code/command snippets are now better/correctlyindented. So I think this is overall better. > > Documentation about documentation generation should be updated though. Please add pandoc to requirements here [1]. > > [1] https://www.postgresql.org/docs/devel/docguide-toolsets.html Committed with a fix for that, thanks. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 09/01/2019 10:05, Mi Tar wrote:
>> I tested this. Patch applied cleanly and INSTALL file was produced. Formatting looks differently from before, but I
thinkthat it looks better. We lost central alignment of some headings, but many code/command snippets are now
better/correctlyindented. So I think this is overall better.
> Committed with a fix for that, thanks.
So this has got a couple of problems:
1. No pandoc on borka, where we build tarballs:
pgsql@borka:~$ which pandoc
pgsql@borka:~$
Probably we can get the sysadmin team to fix that, but ...
2. If there's no pandoc, this coding silently produces a zero-size
INSTALL file. I do not find that acceptable.
pgsql@borka:~$ mk-release 96b8b8b6f9d8de4af01a77797273ad88c7a8e32e
...
Commit 96b8b8b6f9d8de4af01a77797273ad88c7a8e32e packaged as postgresql-12devel.tar.bz2 postgresql-12devel.tar.bz2.md5
postgresql-12devel.tar.bz2.sha256postgresql-12devel.tar.gz postgresql-12devel.tar.gz.md5
postgresql-12devel.tar.gz.sha256
pgsql@borka:~$ tar tvfj output/postgresql-12devel.tar.bz2 | grep INSTALL
-rw-r--r-- pgsql/pgsql 0 2019-01-11 15:09 postgresql-12devel/INSTALL
regards, tom lane
I wrote:
> 1. No pandoc on borka, where we build tarballs:
> pgsql@borka:~$ which pandoc
> pgsql@borka:~$
That part's sorted, anyway.
pgsql@borka:~$ pandoc --version
pandoc 1.17.2
Compiled with texmath 0.8.6.7, highlighting-kate 0.6.3.
Syntax highlighting is supported for the following languages:
...
> 2. If there's no pandoc, this coding silently produces a zero-size
> INSTALL file. I do not find that acceptable.
Seems like it might be sufficient for the rule to be
$(PANDOC) $< -t plain > $@.tmp
$(ICONV) -f utf8 -t us-ascii//TRANSLIT < $@.tmp > $@
rm -f $@.tmp
Failure would leave a .tmp file behind, but I doubt we care enough
about that to work harder than this.
regards, tom lane
Hi! On Fri, Jan 11, 2019 at 1:05 PM Tom Lane <tgl@sss.pgh.pa.us> wrote: > Failure would leave a .tmp file behind, but I doubt we care enough > about that to work harder than this. Maybe just make sure that "make clean" removes it? Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m
On 11/01/2019 22:05, Tom Lane wrote: >> 2. If there's no pandoc, this coding silently produces a zero-size >> INSTALL file. I do not find that acceptable. > > Seems like it might be sufficient for the rule to be > > $(PANDOC) $< -t plain > $@.tmp > $(ICONV) -f utf8 -t us-ascii//TRANSLIT < $@.tmp > $@ > rm -f $@.tmp Fixed. (We use the same pattern for the dtrace invocation.) I suppose the same issue would have existed with the previous rule using lynx, but no one has ever run into it. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services