Re: Updatable views/with check option parsing - Mailing list pgsql-hackers

I wrote:
> We can use the same technique that we used for UNION JOIN, but instead
> join, say, WITH and TIME into one token and make the time datatype
> productions look for "TIME WITHTIME ZONE" and so on.  (I propose this
> rather than putting the ugliness into WITH CHECK OPTION, because this
> way we should only need one merged token and thus only one case to
> check in the filter function; AFAICS we'd need three cases if we
> merge tokens on that end of it.)

On investigation that turns out to have been a bad idea: if we do it
that way, it becomes necessary to promote WITH to a fully reserved word.
The counterexample is

    CREATE VIEW v AS SELECT * FROM foo WITH ...

Is WITH an alias for foo (with no AS), or is it the start of a WITH
CHECK OPTION?  No way to tell without lookahead.

While I don't think that making WITH a fully reserved word would cause
any great damage, I'm unwilling to do it just to save a couple of lines
of code.  Accordingly, I propose the attached patch.  This reinstates
the filter yylex function formerly used for UNION JOIN (in a slightly
cleaner fashion than it was previously done) and parses WITH CHECK
OPTION without any bison complaints, and with no new reserved words.

If no objections, I'll go ahead and apply this, and Peter can get on
with making the stub productions do something useful.

            regards, tom lane

Index: src/backend/parser/Makefile
===================================================================
RCS file: /cvsroot/pgsql/src/backend/parser/Makefile,v
retrieving revision 1.43
diff -c -r1.43 Makefile
*** src/backend/parser/Makefile    7 Mar 2006 01:00:17 -0000    1.43
--- src/backend/parser/Makefile    27 May 2006 02:39:51 -0000
***************
*** 57,63 ****


  # Force these dependencies to be known even without dependency info built:
! gram.o keywords.o: $(srcdir)/parse.h


  # gram.c, parse.h, and scan.c are in the distribution tarball, so they
--- 57,63 ----


  # Force these dependencies to be known even without dependency info built:
! gram.o keywords.o parser.o: $(srcdir)/parse.h


  # gram.c, parse.h, and scan.c are in the distribution tarball, so they
Index: src/backend/parser/gram.y
===================================================================
RCS file: /cvsroot/pgsql/src/backend/parser/gram.y,v
retrieving revision 2.544
diff -c -r2.544 gram.y
*** src/backend/parser/gram.y    30 Apr 2006 18:30:39 -0000    2.544
--- src/backend/parser/gram.y    27 May 2006 02:39:52 -0000
***************
*** 70,75 ****
--- 70,81 ----
              (Current) = (Rhs)[0]; \
      } while (0)

+ /*
+  * The %name-prefix option below will make bison call base_yylex, but we
+  * really want it to call filtered_base_yylex (see parser.c).
+  */
+ #define base_yylex filtered_base_yylex
+
  extern List *parsetree;            /* final parse result is delivered here */

  static bool QueryIsRule = FALSE;
***************
*** 339,344 ****
--- 345,351 ----
  %type <list>    constraints_set_list
  %type <boolean> constraints_set_mode
  %type <str>        OptTableSpace OptConsTableSpace OptTableSpaceOwner
+ %type <list>    opt_check_option


  /*
***************
*** 356,362 ****
      BACKWARD BEFORE BEGIN_P BETWEEN BIGINT BINARY BIT
      BOOLEAN_P BOTH BY

!     CACHE CALLED CASCADE CASE CAST CHAIN CHAR_P
      CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
      CLUSTER COALESCE COLLATE COLUMN COMMENT COMMIT
      COMMITTED CONNECTION CONSTRAINT CONSTRAINTS CONVERSION_P CONVERT COPY CREATE CREATEDB
--- 363,369 ----
      BACKWARD BEFORE BEGIN_P BETWEEN BIGINT BINARY BIT
      BOOLEAN_P BOTH BY

!     CACHE CALLED CASCADE CASCADED CASE CAST CHAIN CHAR_P
      CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
      CLUSTER COALESCE COLLATE COLUMN COMMENT COMMIT
      COMMITTED CONNECTION CONSTRAINT CONSTRAINTS CONVERSION_P CONVERT COPY CREATE CREATEDB
***************
*** 431,436 ****
--- 438,449 ----

      ZONE

+ /* The grammar thinks these are keywords, but they are not in the keywords.c
+  * list and so can never be entered directly.  The filter in parser.c
+  * creates these tokens when required.
+  */
+ %token            WITH_CASCADED WITH_LOCAL WITH_CHECK
+
  /* Special token types, not actually keywords - see the "lex" file */
  %token <str>    IDENT FCONST SCONST BCONST XCONST Op
  %token <ival>    ICONST PARAM
***************
*** 4618,4629 ****
  /*****************************************************************************
   *
   *    QUERY:
!  *        CREATE [ OR REPLACE ] [ TEMP ] VIEW <viewname> '('target-list ')' AS <query>
   *
   *****************************************************************************/

  ViewStmt: CREATE OptTemp VIEW qualified_name opt_column_list
!                 AS SelectStmt
                  {
                      ViewStmt *n = makeNode(ViewStmt);
                      n->replace = false;
--- 4631,4643 ----
  /*****************************************************************************
   *
   *    QUERY:
!  *        CREATE [ OR REPLACE ] [ TEMP ] VIEW <viewname> '('target-list ')'
!  *            AS <query> [ WITH [ CASCADED | LOCAL ] CHECK OPTION ]
   *
   *****************************************************************************/

  ViewStmt: CREATE OptTemp VIEW qualified_name opt_column_list
!                 AS SelectStmt opt_check_option
                  {
                      ViewStmt *n = makeNode(ViewStmt);
                      n->replace = false;
***************
*** 4634,4640 ****
                      $$ = (Node *) n;
                  }
          | CREATE OR REPLACE OptTemp VIEW qualified_name opt_column_list
!                 AS SelectStmt
                  {
                      ViewStmt *n = makeNode(ViewStmt);
                      n->replace = true;
--- 4648,4654 ----
                      $$ = (Node *) n;
                  }
          | CREATE OR REPLACE OptTemp VIEW qualified_name opt_column_list
!                 AS SelectStmt opt_check_option
                  {
                      ViewStmt *n = makeNode(ViewStmt);
                      n->replace = true;
***************
*** 4646,4651 ****
--- 4660,4691 ----
                  }
          ;

+ /*
+  * We use merged tokens here to avoid creating shift/reduce conflicts against
+  * a whole lot of other uses of WITH.
+  */
+ opt_check_option:
+         WITH_CHECK OPTION
+                 {
+                     ereport(ERROR,
+                             (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+                              errmsg("WITH CHECK OPTION is not implemented")));
+                 }
+         | WITH_CASCADED CHECK OPTION
+                 {
+                     ereport(ERROR,
+                             (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+                              errmsg("WITH CHECK OPTION is not implemented")));
+                 }
+         | WITH_LOCAL CHECK OPTION
+                 {
+                     ereport(ERROR,
+                             (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+                              errmsg("WITH CHECK OPTION is not implemented")));
+                 }
+         | /* EMPTY */                            { $$ = NIL; }
+         ;
+
  /*****************************************************************************
   *
   *        QUERY:
***************
*** 8319,8324 ****
--- 8359,8365 ----
              | CACHE
              | CALLED
              | CASCADE
+             | CASCADED
              | CHAIN
              | CHARACTERISTICS
              | CHECKPOINT
***************
*** 9139,9142 ****
--- 9180,9189 ----
      }
  }

+ /*
+  * Must undefine base_yylex before including scan.c, since we want it
+  * to create the function base_yylex not filtered_base_yylex.
+  */
+ #undef base_yylex
+
  #include "scan.c"
Index: src/backend/parser/parser.c
===================================================================
RCS file: /cvsroot/pgsql/src/backend/parser/parser.c,v
retrieving revision 1.65
diff -c -r1.65 parser.c
*** src/backend/parser/parser.c    7 Mar 2006 01:00:17 -0000    1.65
--- src/backend/parser/parser.c    27 May 2006 02:39:52 -0000
***************
*** 22,32 ****
--- 22,36 ----
  #include "postgres.h"

  #include "parser/gramparse.h"
+ #include "parser/parse.h"
  #include "parser/parser.h"


  List       *parsetree;            /* result of parsing is left here */

+ static int    lookahead_token;    /* one-token lookahead */
+ static bool have_lookahead;        /* lookahead_token set? */
+

  /*
   * raw_parser
***************
*** 40,45 ****
--- 44,50 ----
      int            yyresult;

      parsetree = NIL;            /* in case grammar forgets to set it */
+     have_lookahead = false;

      scanner_init(str);
      parser_init();
***************
*** 53,55 ****
--- 58,120 ----

      return parsetree;
  }
+
+
+ /*
+  * Intermediate filter between parser and base lexer (base_yylex in scan.l).
+  *
+  * The filter is needed because in some cases the standard SQL grammar
+  * requires more than one token lookahead.  We reduce these cases to one-token
+  * lookahead by combining tokens here, in order to keep the grammar LALR(1).
+  *
+  * Using a filter is simpler than trying to recognize multiword tokens
+  * directly in scan.l, because we'd have to allow for comments between the
+  * words.  Furthermore it's not clear how to do it without re-introducing
+  * scanner backtrack, which would cost more performance than this filter
+  * layer does.
+  */
+ int
+ filtered_base_yylex(void)
+ {
+     int            cur_token;
+
+     /* Get next token --- we might already have it */
+     if (have_lookahead)
+     {
+         cur_token = lookahead_token;
+         have_lookahead = false;
+     }
+     else
+         cur_token = base_yylex();
+
+     /* Do we need to look ahead for a possible multiword token? */
+     switch (cur_token)
+     {
+         case WITH:
+             /*
+              * WITH CASCADED, LOCAL, or CHECK must be reduced to one token
+              */
+             lookahead_token = base_yylex();
+             switch (lookahead_token)
+             {
+                 case CASCADED:
+                     cur_token = WITH_CASCADED;
+                     break;
+                 case LOCAL:
+                     cur_token = WITH_LOCAL;
+                     break;
+                 case CHECK:
+                     cur_token = WITH_CHECK;
+                     break;
+                 default:
+                     have_lookahead = true;
+                     break;
+             }
+             break;
+
+         default:
+             break;
+     }
+
+     return cur_token;
+ }
Index: src/include/parser/gramparse.h
===================================================================
RCS file: /cvsroot/pgsql/src/include/parser/gramparse.h,v
retrieving revision 1.36
diff -c -r1.36 gramparse.h
*** src/include/parser/gramparse.h    21 May 2006 20:10:42 -0000    1.36
--- src/include/parser/gramparse.h    27 May 2006 02:39:52 -0000
***************
*** 40,45 ****
--- 40,48 ----
  extern bool standard_conforming_strings;


+ /* from parser.c */
+ extern int    filtered_base_yylex(void);
+
  /* from scan.l */
  extern void scanner_init(const char *str);
  extern void scanner_finish(void);