Thread: REINDEX VERBOSE iso-8859-1 option

On Sun, Nov 17, 2019 at 04:41:59AM +0100, Pavel Stehule wrote:
> Documentation patch is good idea.

The documentation is rather clear about the need to of parenthesis
when using the VERBOSE option, and that it is not a mandatory option:
REINDEX [ ( VERBOSE ) ] { INDEX | TABLE | SCHEMA | DATABASE | SYSTEM } name
--
Michael

On Mon, Nov 18, 2019 at 10:27:24AM +0100, Josef Šimánek wrote:
> This is clear once you understand what does it mean. I was aware of VERBOSE
> option of EXPLAIN and tried to use it without needed parentheses (the same
> way EXPLAIN can understand it). In the parameter list of REINDEX, it is
> still called VERBOSE (not "( VERBOSE )") and there's no info
> that parentheses are needed.

What would you do in the documentation once there is support for more
than one option then?

The grammar is done this way to remain extensible.  If you look at any
command in the documentation, there are two things to be aware of:
1) Clauses within square brackets are optional.
2) Multiple clauses within braces and separated by '|' mean that at
least one is mandatory.

Parenthesis do not matter here.  They are part of the query syntax.

> PS: AFAIK VERBOSE is option for EXPLAIN, but parameter for REINDEX. Is that
> the reason for different syntax?

Mainly historical reasons.  REINDEX VERBOSE has been added in 9.5.
EXPLAIN VERBOSE is around since at least 7.1.  Using options within
parenthesis is preferred lately because it is much easier to make the
grammar more extensible for future purposes and it eases the option
parsing.
--
Michael

On Mon, 18 Nov 2019 at 11:46, Michael Paquier <michael@paquier.xyz> wrote:
> On Mon, Nov 18, 2019 at 10:27:24AM +0100, Josef Šimánek wrote:
> > This is clear once you understand what does it mean. I was aware of VERBOSE
> > option of EXPLAIN and tried to use it without needed parentheses (the same
> > way EXPLAIN can understand it). In the parameter list of REINDEX, it is
> > still called VERBOSE (not "( VERBOSE )") and there's no info
> > that parentheses are needed.
[snip]
> Mainly historical reasons.  REINDEX VERBOSE has been added in 9.5.
> EXPLAIN VERBOSE is around since at least 7.1.  Using options within
> parenthesis is preferred lately because it is much easier to make the
> grammar more extensible for future purposes and it eases the option
> parsing.

All well and good (although personally I think it's arguable whether
it's productive to have two different syntaxes for something that in
the user's perspective does the same thing). But if the parentheses
are part of the parameter, I think putting the parentheses in the
parameter list might be a good compromise.

Geoff

On 2019-11-18 12:24:40 +0000, Geoff Winkless wrote:
> On Mon, 18 Nov 2019 at 11:46, Michael Paquier <michael@paquier.xyz> wrote:
> > On Mon, Nov 18, 2019 at 10:27:24AM +0100, Josef Šimánek wrote:
> > > This is clear once you understand what does it mean. I was aware of VERBOSE
> > > option of EXPLAIN and tried to use it without needed parentheses (the same
> > > way EXPLAIN can understand it). In the parameter list of REINDEX, it is
> > > still called VERBOSE (not "( VERBOSE )") and there's no info
> > > that parentheses are needed.
[...]
> But if the parentheses are part of the parameter, I think putting the
> parentheses in the parameter list might be a good compromise.

The parentheses aren't part of the parameter. They are part of the
syntax of the command.

It might useful to distinguish better between symbols which must be
typed as shown ("REINDEX", "(", ")", etc. in this case) and those which
describe the grammar ("[", "]", "name" in this case).

Maybe print the square brackets in bold and italics like "name"? But
that is just the opposite of the usual convention.

How about this?

* Terminals (stuff that has to be typed as shown) in bold.

* Non-Terminals (stuff which has to be replaced) in italic.

* Meta-characters ([, ], |, ...) in regular type.

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

On Mon, 18 Nov 2019 at 22:24, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
>
> On 2019-11-18 12:24:40 +0000, Geoff Winkless wrote:
> > On Mon, 18 Nov 2019 at 11:46, Michael Paquier <michael@paquier.xyz> wrote:
> > > On Mon, Nov 18, 2019 at 10:27:24AM +0100, Josef Šimánek wrote:
> > > > This is clear once you understand what does it mean. I was aware of VERBOSE
> > > > option of EXPLAIN and tried to use it without needed parentheses (the same
> > > > way EXPLAIN can understand it). In the parameter list of REINDEX, it is
> > > > still called VERBOSE (not "( VERBOSE )") and there's no info
> > > > that parentheses are needed.
> [...]
> > But if the parentheses are part of the parameter, I think putting the
> > parentheses in the parameter list might be a good compromise.
>
> The parentheses aren't part of the parameter. They are part of the
> syntax of the command.

Then at the very least draw attention to the fact that the parentheses
are required in the description of the parameter in the notes.

It's bad enough that you have the inconsistency that REINDEX VERBOSE
requires parentheses while the more recent REINDEX CONCURRENTLY does
not (presumably to match the syntax of CREATE INDEX CONCURRENTLY),
without insisting that the user parse the difference between { and (
in the manual (not the easiest difference to scan) before they can use
the command.

> How about this?
>
> * Terminals (stuff that has to be typed as shown) in bold.
>
> * Non-Terminals (stuff which has to be replaced) in italic.
>
> * Meta-characters ([, ], |, ...) in regular type.

Even if you do that you're still requiring the user to parse syntax
according to esoteric rules. I'm not sure that changing the rules
helps that much.

Geoff

On Tue, Nov 19, 2019 at 11:37:04AM +0000, Geoff Winkless wrote:
> It's bad enough that you have the inconsistency that REINDEX VERBOSE
> requires parentheses while the more recent REINDEX CONCURRENTLY does
> not (presumably to match the syntax of CREATE INDEX CONCURRENTLY),
> without insisting that the user parse the difference between { and (
> in the manual (not the easiest difference to scan) before they can use
> the command.

The first implementations of REINDEX CONCURRENTLY used the existing
grammar.  There was also a discussion when the feature was close to
commit about exactly that and I preferred the parenthesis option.
Who won t the end?  Consistency with the existing grammar for
CREATE/DROP INDEX here.

>> How about this?
>>
>> * Terminals (stuff that has to be typed as shown) in bold.
>>
>> * Non-Terminals (stuff which has to be replaced) in italic.
>>
>> * Meta-characters ([, ], |, ...) in regular type.
>
> Even if you do that you're still requiring the user to parse syntax
> according to esoteric rules. I'm not sure that changing the rules
> helps that much.

This does not concern only the page for REINDEX.  Perhaps this could
be improved, but I am not sure how and particularly if changing it is
worth it as many people are used to the existing way of presenting the
commands synopsis as well.
--
Michael

On 2019-11-19 11:37:04 +0000, Geoff Winkless wrote:
> Even if you do that you're still requiring the user to parse syntax
> according to esoteric rules.

Oh, please. Those "esoteric rules" have been in wide-spread use for
decades. If you look at any manual which tries to explain the syntax of
a programming language, markup language or something similar in a
(semi-)formal way, it probably uses something very similar. (More formal
texts often use BNF (or a variant), which are more expressive, but
harder to read. Personally I like syntax diagrams (sometimes called
railroad diagrams, but they seem to have mostly fallen out of fashion)

And of course, like almost any manual, the PostgreSQL manual explains
the notation in the preface:
https://www.postgresql.org/docs/current/notation.html
(Paper books are at an advantage here that readers usually actually
start at the beginning)

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

On Wed, 20 Nov 2019 at 22:48, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
>
> On 2019-11-19 11:37:04 +0000, Geoff Winkless wrote:
> > Even if you do that you're still requiring the user to parse syntax
> > according to esoteric rules.
>
> Oh, please. Those "esoteric rules" have been in wide-spread use for
> decades.

It wasn't meant to be insulting, I meant "esoteric" in the strict
sense: that you need to have specific knowledge to parse them.

My point was that modifying the rules (by making certain things bold
or italic) wouldn't really solve the problem - if you don't know what
the rules are, you're unlikely to be any better off if someone adds to
them.

Geoff

On 2019-11-21 09:43:26 +0000, Geoff Winkless wrote:
> On Wed, 20 Nov 2019 at 22:48, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
> >
> > On 2019-11-19 11:37:04 +0000, Geoff Winkless wrote:
> > > Even if you do that you're still requiring the user to parse syntax
> > > according to esoteric rules.
> >
> > Oh, please. Those "esoteric rules" have been in wide-spread use for
> > decades.
>
> It wasn't meant to be insulting, I meant "esoteric" in the strict
> sense: that you need to have specific knowledge to parse them.

I didn't understand it as insulting (why would I?), but don't think this
convention is "requiring ... knowledge that is restricted to a small
group" (Merriam-Webster). This specific convention for conveying grammar
rules is in my experience by far the most common (before BNF and
diagrams). Anybody who has read any documentation about any formal
language (e.g., a programming language, a query language, a markup or
configuration language) has very likely encountered it before.

Yes, you need specific knowledge to understand the PostgreSQL
documentation. For starters, you need to know English (or one the
handful languages in which it has been translated). You need to know
what a relational database is and why and how you would use one. You
need some generic knowledge about computing (what is a "client/server
model"? How do I start a command line tool?), etc. The convention for
describing the grammar is probably the least concern, and besides, it is
explained in the manual (unlike some concepts which are assumed to be
known).


> My point was that modifying the rules (by making certain things bold
> or italic) wouldn't really solve the problem - if you don't know what
> the rules are, you're unlikely to be any better off if someone adds to
> them.

Man is a pattern-matching animal. Even without an explicit explanation,
humans are quite good at deriving meaning from repeated patterns. So if
the parts you have to type verbatim are always printed in bold and parts
which have a meta-meaning are always printed in italic and optional
parts are always enclosed in (italic) square brackets, people are very
likely to understand that
    <i>[</i> <b>( VERBOSE )</b> <i>]</i>
means that "( VERBOSE )" must be typed as is, but is optional. Even if
they can't tell you the rules. Simply because they have seen it a few
dozen times before. There is a reason why almost any technical
documentation uses some typographical convention and why those
conventions are almost always very similar.

The PostgreSQL manual unfortunately uses the same typographic convention
for meta-characters ([], {}, |, ...) and terminals, which isn't as clear
as it could be.

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

On Thu, 21 Nov 2019 at 15:32, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
> On 2019-11-21 09:43:26 +0000, Geoff Winkless wrote:
> > It wasn't meant to be insulting, I meant "esoteric" in the strict
> > sense: that you need to have specific knowledge to parse them.
>
> I didn't understand it as insulting (why would I?),

I've absolutely no idea, but I couldn't imagine why on Earth you would
apparently take such exception to it otherwise. Maybe writing
sarcastic bombast in response to something that hasn't annoyed you is
just your thing. Oh well, you do you.

> but don't think this
> convention is "requiring ... knowledge that is restricted to a small
> group" (Merriam-Webster).

That's entirely the case here. I'd say the number of people able to
understand something like BNF is vanishingly small in terms of the
7bn(?) world-population.

> This specific convention for conveying grammar
> rules is in my experience by far the most common (before BNF and
> diagrams). Anybody who has read any documentation about any formal
> language (e.g., a programming language, a query language, a markup or
> configuration language) has very likely encountered it before.

OK, but parentheses not being part of the rules, ie that they should
be read literally, is something that is knowledge specific to
postgresql, because "this specific convention" you blithely reference
is only a convention, and there are several instances where
programmers have their own version of this convention where
parentheses are part of the grammar, not the syntax.

> > My point was that modifying the rules (by making certain things bold
> > or italic) wouldn't really solve the problem - if you don't know what
> > the rules are, you're unlikely to be any better off if someone adds to
> > them.
>
> people are very
> likely to understand that
>     <i>[</i> <b>( VERBOSE )</b> <i>]</i>
> means that "( VERBOSE )" must be typed as is, but is optional. Even if
> they can't tell you the rules

Depending on the font, I'm not at all confident that I could tell the
difference between <b><i>( VERBOSE )</i></b> and (<b><i> VERBOSE
</i></b>) unless they're actually next to each other.

Geoff

On 2019-11-21 16:48:14 +0000, Geoff Winkless wrote:
> On Thu, 21 Nov 2019 at 15:32, Peter J. Holzer <hjp-pgsql@hjp.at> wrote:
> > On 2019-11-21 09:43:26 +0000, Geoff Winkless wrote:
> > > It wasn't meant to be insulting, I meant "esoteric" in the strict
> > > sense: that you need to have specific knowledge to parse them.
> >
> > I didn't understand it as insulting (why would I?),
>
> I've absolutely no idea, but I couldn't imagine why on Earth you would
> apparently take such exception to it otherwise. Maybe writing
> sarcastic bombast in response to something that hasn't annoyed you is

I don't think we agree on the meaning of "sarcastic" and "bombast".


> > but don't think this convention is "requiring ... knowledge that is
> > restricted to a small group" (Merriam-Webster).
>
> That's entirely the case here. I'd say the number of people able to
> understand something like BNF is vanishingly small in terms of the
> 7bn(?) world-population.

The number of people reading the PostgreSQL manual is also vanishingly
small in terms of the world-population. These two aren't independent.

        hp

--
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"