Re: Proposal to add --single-row to psql - Mailing list pgsql-hackers

On Sat, May 11, 2013 at 12:27 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> By the time you've got an expression tree, the problem is mostly solved,
> at least so far as parser extension is concerned.

Right.

> More years ago than I care to admit, I worked on systems that had
> run-time-extensible parsers at Hewlett-Packard, so technology for this
> does exist.  But my (vague) memory of those systems is that the parser's
> language capabilities were more limited than bison's, perhaps only
> LL(1).  Parsing spec-compatible SQL that way might be a challenge.

If I understand bison correctly, it basically looks at the current
parser state and the next token and decides to either shift that token
onto the stack or reduce the stack using some rule.  If there's no
matching rule, we error out.  If someone wants to inject new rules
into the grammar, those state tables are all going to change.  But if
we could contrive things so that the state tables are built
dynamically and can be change as rules are added and removed, then it
seems to me that we could let a loadable module supply (or delete)
whatever grammar rules it likes.  Whenever it does this, we recompile
the grammar on next use (and complain if we get ambiguities).  This
does not sound all that easy to code, but at least in theory it seems
doable.

We'd also need a way to add keywords.  To be quite frank, I think our
whole approach to keywords is massive screwed up right now.  Aside
from the parser bloat, adding even unreserved keywords breaks stuff,
e.g. "SELECT 1 this_is_not_yet_a_keyword".  EnterpriseDB's internal
bug-tracking system must have at least half a dozen bug reports open
right now that are attributable to keywords either existing at all or
being more reserved than they are in Oracle.  That aside, the amount
of trouble we've caused for PostgreSQL users over the years, either by
adding new keywords or disallowing them in contexts where they used to
work, is not small.  I don't have a specific proposal for what we
should do to make this problem less painful, but I think if we ever
revise our parser infrastructure we ought to use that as an
opportunity to try to come up with something better than what exists
today, because the status quo is awfully painful.

> Another point is that extensions that are actually interesting require
> a lot more than new syntax.  Robert mentioned the rewriter, but you'd
> typically need planner and executor additions as well.  It's possible to
> see how whole new plan node types might be added by a plugin so far as
> the executor is concerned, but I haven't a clue how we'd get the planner
> to emit them ...

I don't have any ideas either.  Still, solving some of the problems
would be better than solving none of them.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company