Re: [HACKERS] Other formats in pset like markdown, rst, mediawiki - Mailing list pgsql-hackers

<resent, wrong from, sorry for those who may get multiple copies>

Hello Jan,

Please give a number to submitted patches. I think that this was v3.

The patch does NOT fix various issues I pointed out in my previous review:
 - tabs introduced in "doc/src/sgml/ref/psql-ref.sgml" - too long help line in "src/bin/psql/help.c" - spurious space
aftera comma in "src/fe_utils/print.c"   and possibly elsewhere.
 

On Sun, 23 Apr 2017, Jan Michálek wrote:

> > > Markdown include characters/sequences which are interpreted as markers:
> > > _Italic_, **Bold**, *** => horizontal rules, > block quote... `inline
> > > code`... If they are interpreted within a table cell then probably they
> > > should be escaped somehow.
> 
> I have treated "_*|<>"

Probably not enough, see below. Note the escaping chars should also be escaped.

> > I`m able to sanitize characters, but complex sequences will be problem. I
> > will look on this, but I don`t know, if I`m able to do this.

I do not know whether only those are necessary. Have you checked? Guessing is
probably not the right approach.


Concerning MARKDOWN, and according to the following source about github markdown
implementation:
    https://enterprise.github.com/downloads/en/markdown-cheatsheet.pdf

The following characters may need to be backslash escaped, although it does not
cover HTML stuff.
    \   backslash    `   backtick    *   asterisk    _   underscore    {}  curly braces    []  square brackets    ()
parentheses   #   hash mark    +   plus sign    -   minus sign (hyphen)    .   dot    !   exclamation
 

Another source https://genius.com/3057216 suggests (* # / ( ) [ ] < >),
which should protect HTML.

However, the escaping seems to be the backslash character, NOT using html
encoding < as done in your version.

Where did you find the precise escaping rules for markdown? I do not think that
it should be invented...


I have looked at RST, according to this reference:
    http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#grid-
tables

The good news is that you do not need to handle a special | case because you
would only produce clean tables.

I've tested UTF-8 with plane 1 (你好!) and plane 2 (𠀡) and the alignment seems
to worked well, incredible!

> > My main interest on this was in rst. I`m using markdown only in github
> > issues and my knowldge about md is poor.

Then maybe only do RST?!

It looks much simpler anyway, and if you do MARKDOWN the support needs to be
clean.

About the code:

I'm still at odds with the code which needs to test for markdown to call for
different functions in multiple places. If you keep md and in order to avoid
that, I would suggest to extend the pg_wcs* functions with a list of caracters
which may have different sizes with additionnal args, say:
  pg_wcssize(// same args, plus:             char * escaped_chars, // will require escaping             int escape_len,
//how many chars added when escaping             int nllen // len of newline if substituted             );
 

So that pg_wcssize(..., "\r", 1, 1) would behave as before (\n and \t are rather
special cases), and the various constants could be held in the format
description so the whole thing would be parametric.

Same approach with format.

That would allow to simplify the code significantly and to share it between
MARKDOWN and others. Also, the multiple "else if" list would be simplified by
using strchr or the escaped_chars string.

-- 
Fabien.