Thread: problems with parser

problems with parser

From
Massimo Dal Zotto
Date:
Hi,

I have some problems with the parser.

1)    Of the following queries, submitted with libpgtcl, the first two areparsed correctly while the third gives a
parseerror:
 
1.    select 12.    select 1; select 2;3.    select 1; select 2ERROR:  parser: parse error at or near ""
It seems that when a query consiste of two or more statements thelast one must be terminated by ';'. In my opinion this
isnotcorrect because it is not consistent with case 1) and because itbreaks many existing programs compatible with
previousversionsof pgsql where the syntax of point 2) was considered valid.
 
The same problem applies also to the body of sql functions, while itdoesn't apply to query submitted by psql because
theyare splittedin separate statements submitted one by one.
 

2)    The following query does't work:
create operator *= (         leftarg=_varchar,          rightarg=varchar,          procedure=array_varchareq);ERROR:
parser:parse error at or near "varchar"
 
The query should work because it is consistent with the documentedsyntax of the create operator:
Command: create operatorDescription: create a user-defined operatorSyntax:           CREATE OPERATOR operator_name (
      [LEFTARG = type1][,RIGHTARG = type2]           ,PROCEDURE = func_name,           [,COMMUTATOR = com_op][,NEGATOR
=neg_op]           [,RESTRICT = res_proc][,JOIN = join_proc][,HASHES]           [,SORT1 = left_sort_op][,SORT2 =
right_sort_op]);
and varchar is a valid type name (it is in pg_type).After a litte experimenting it turned out that varchar is also
areservedword and therefore not acceptable as a type name. To havethe above statement work you must quote the word
"varchar".
This is somewhat inconsistent with the syntax of create operatorand may confuse the user.

3)    The above query introduces another problem. How can the user knowwhat is wrong in the input. In the example
"parseerror at or near"is not a very explicative message. If I had read "reserved keyword"instead I would not have
spenttime trying to figure out what'swrong in my query.
 
The parser should be made more verbose and helpful about errors.

4)    And another related question: if the casual user can be confusedby obscure parser messages how can the postgres
hackerdebug theparser grammar? I tried with gdb but it is completely useless giventhe way the parser work.Is there any
toolor trick to debug the grammar?
 


-- 
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto               email: dz@cs.unitn.it               |
|  Via Marconi, 141                phone: ++39-0461534251              |
|  38057 Pergine Valsugana (TN)      www: http://www.cs.unitn.it/~dz/  |
|  Italy                             pgp: finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+


Re: [HACKERS] problems with parser

From
José Soares
Date:
Massimo Dal Zotto ha scritto:

> Hi,
>
> I have some problems with the parser.
>
> 1)      Of the following queries, submitted with libpgtcl, the first two are
>         parsed correctly while the third gives a parse error:
>
>         1.      select 1
>         2.      select 1; select 2;
>         3.      select 1; select 2
>         ERROR:  parser: parse error at or near ""
>
>         It seems that when a query consiste of two or more statements the
>         last one must be terminated by ';'. In my opinion this is not
>         correct because it is not consistent with case 1) and because it
>         breaks many existing programs compatible with previous versions
>         of pgsql where the syntax of point 2) was considered valid.
>
>         The same problem applies also to the body of sql functions, while it
>         doesn't apply to query submitted by psql because they are splitted
>         in separate statements submitted one by one.
>
> 2)      The following query does't work:
>
>         create operator *= (
>                 leftarg=_varchar,
>                 rightarg=varchar,
>                 procedure=array_varchareq);
>         ERROR:  parser: parse error at or near "varchar"
>
>         The query should work because it is consistent with the documented
>         syntax of the create operator:
>
>         Command: create operator
>         Description: create a user-defined operator
>         Syntax:
>                 CREATE OPERATOR operator_name (
>                 [LEFTARG = type1][,RIGHTARG = type2]
>                 ,PROCEDURE = func_name,
>                 [,COMMUTATOR = com_op][,NEGATOR = neg_op]
>                 [,RESTRICT = res_proc][,JOIN = join_proc][,HASHES]
>                 [,SORT1 = left_sort_op][,SORT2 = right_sort_op]);
>
>         and varchar is a valid type name (it is in pg_type).
>         After a litte experimenting it turned out that varchar is also a
>         reserved word and therefore not acceptable as a type name. To have
>         the above statement work you must quote the word "varchar".

Yes but some times the parser understands the keyword varchar without "" as in:

create function "varchar"(float) returns varchar as                       ^^^^^^                      ^^^^^^
'begin       return $1;
end;
' language 'plpgsql';
CREATE

drop table a;
DROP
create table a (a float8);
CREATE
insert into a values (1.2);
INSERT 128074 1

or  in the CAST()...

select cast(a as varchar) from a;
varchar
-------   1.2
(1 row)

select varchar(a) from a;
ERROR:  parser: parse error at or near "a"
select "varchar"(a) from a;
varchar
-------   1.2
(1 row)



>
>         This is somewhat inconsistent with the syntax of create operator
>         and may confuse the user.
>
> 3)      The above query introduces another problem. How can the user know
>         what is wrong in the input. In the example "parse error at or near"
>         is not a very explicative message. If I had read "reserved keyword"
>         instead I would not have spent time trying to figure out what's
>         wrong in my query.
>
>         The parser should be made more verbose and helpful about errors.
>
> 4)      And another related question: if the casual user can be confused
>         by obscure parser messages how can the postgres hacker debug the
>         parser grammar? I tried with gdb but it is completely useless given
>         the way the parser work.
>         Is there any tool or trick to debug the grammar?
>
> --
> Massimo Dal Zotto
>
> +----------------------------------------------------------------------+
> |  Massimo Dal Zotto               email: dz@cs.unitn.it               |
> |  Via Marconi, 141                phone: ++39-0461534251              |
> |  38057 Pergine Valsugana (TN)      www: http://www.cs.unitn.it/~dz/  |
> |  Italy                             pgp: finger dz@tango.cs.unitn.it  |
> +----------------------------------------------------------------------

PostgreSQL 6.5.0 on i586-pc-linux-gnu, compiled by gcc 2.7.2.3
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Jose'




Re: [HACKERS] problems with parser

From
Bruce Momjian
Date:
[Charset ISO-8859-1 unsupported, filtering to ASCII...]
> Hi,
> 
> I have some problems with the parser.
> 
> 1)    Of the following queries, submitted with libpgtcl, the first two are
>     parsed correctly while the third gives a parse error:
> 
>     1.    select 1
>     2.    select 1; select 2;
>     3.    select 1; select 2
>     ERROR:  parser: parse error at or near ""
> 
>     It seems that when a query consiste of two or more statements the
>     last one must be terminated by ';'. In my opinion this is not
>     correct because it is not consistent with case 1) and because it
>     breaks many existing programs compatible with previous versions
>     of pgsql where the syntax of point 2) was considered valid.
> 
>     The same problem applies also to the body of sql functions, while it
>     doesn't apply to query submitted by psql because they are splitted
>     in separate statements submitted one by one.


Added to TODO list.

> 
> 2)    The following query does't work:
> 
>     create operator *= (
>           leftarg=_varchar, 
>           rightarg=varchar, 
>           procedure=array_varchareq);
>     ERROR:  parser: parse error at or near "varchar"
> 
>     The query should work because it is consistent with the documented
>     syntax of the create operator:
> 
>     Command: create operator
>     Description: create a user-defined operator
>     Syntax:
>             CREATE OPERATOR operator_name (
>             [LEFTARG = type1][,RIGHTARG = type2]
>             ,PROCEDURE = func_name,
>             [,COMMUTATOR = com_op][,NEGATOR = neg_op]
>             [,RESTRICT = res_proc][,JOIN = join_proc][,HASHES]
>             [,SORT1 = left_sort_op][,SORT2 = right_sort_op]);
> 
>     and varchar is a valid type name (it is in pg_type).
>     After a litte experimenting it turned out that varchar is also a
>     reserved word and therefore not acceptable as a type name. To have
>     the above statement work you must quote the word "varchar".
> 
>     This is somewhat inconsistent with the syntax of create operator
>     and may confuse the user.


Also added to TODO list.

> 
> 3)    The above query introduces another problem. How can the user know
>     what is wrong in the input. In the example "parse error at or near"
>     is not a very explicative message. If I had read "reserved keyword"
>     instead I would not have spent time trying to figure out what's
>     wrong in my query.
> 
>     The parser should be made more verbose and helpful about errors.
> 
> 4)    And another related question: if the casual user can be confused
>     by obscure parser messages how can the postgres hacker debug the
>     parser grammar? I tried with gdb but it is completely useless given
>     the way the parser work.
>     Is there any tool or trick to debug the grammar?

I have not looked at this particular problem, but usually the errror
generated by the parser are poor.  Unfortunately, I don't know of any
way to insert our own error messaged based on the type of parser
failure.  This is locked up in yacc/bison.

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026
 


Re: [HACKERS] problems with parser

From
Thomas Lockhart
Date:
> > I have some problems with the parser.
> > 1)    Of the following queries, submitted with libpgtcl,

Massimo, what version of Postgres are you running? Is this a new
problem in the v6.5 beta (which includes a few changes from Stefan
which might have adversely affected the behavior)?
                     - Tom

-- 
Thomas Lockhart                lockhart@alumni.caltech.edu
South Pasadena, California


Re: [HACKERS] problems with parser

From
Massimo Dal Zotto
Date:
> 
> > > I have some problems with the parser.
> > > 1)    Of the following queries, submitted with libpgtcl,
> 
> Massimo, what version of Postgres are you running? Is this a new
> problem in the v6.5 beta (which includes a few changes from Stefan
> which might have adversely affected the behavior)?

It was a snapshot of 10-15 days ago. I have seen the problem also in
previous snapshots. The problem is in the grammar in the definition
of multiple queries but unfortunately I don't know yacc enough to fix
the bug.

-- 
Massimo Dal Zotto

+----------------------------------------------------------------------+
|  Massimo Dal Zotto               email: dz@cs.unitn.it               |
|  Via Marconi, 141                phone: ++39-0461534251              |
|  38057 Pergine Valsugana (TN)      www: http://www.cs.unitn.it/~dz/  |
|  Italy                             pgp: finger dz@tango.cs.unitn.it  |
+----------------------------------------------------------------------+


Re: [HACKERS] problems with parser

From
Bruce Momjian
Date:
[Charset ISO-8859-1 unsupported, filtering to ASCII...]
> > 
> > > > I have some problems with the parser.
> > > > 1)    Of the following queries, submitted with libpgtcl,
> > 
> > Massimo, what version of Postgres are you running? Is this a new
> > problem in the v6.5 beta (which includes a few changes from Stefan
> > which might have adversely affected the behavior)?
> 
> It was a snapshot of 10-15 days ago. I have seen the problem also in
> previous snapshots. The problem is in the grammar in the definition
> of multiple queries but unfortunately I don't know yacc enough to fix
> the bug.

The bug still exists.  Just start 'postgres' manually without the
postmaster, and type in a query:

#$ aspg gdb /u/pg/bin/postgresGNU gdb Copyright 1998 Free Software Foundation, Inc.GDB is free software, covered by the
GNUGeneral Public License, and youarewelcome to change it and/or distribute copies of it under certainconditions.Type
"showcopying" to see the conditions.There is absolutely no warranty for GDB.  Type "show warranty" fordetails.This GDB
wasconfigured as "i386-unknown-bsdi4.0"...run -(gdb) run -D /u/pg/data testStarting program: /u/pg/bin/postgres -D
/u/pg/datatestPOSTGRES backend interactive interface $Revision: 1.111 $ $Date: 1999/05/09 23:31:47 $> select 1; select
2ERROR: parser: parse error at or near ""ERROR:  parser: parse error at or near ""
 

> select 1;select 2;blank         1: ?column?    (typeid = 23, len = 4, typmod = -1, byval = t)        ----         1:
?column?= "1"      (typeid = 23, len = 4, typmod = -1, byval = t)        ----blank         1: ?column?    (typeid = 23,
len= 4, typmod = -1, byval = t)        ----         1: ?column? = "2"      (typeid = 23, len = 4, typmod = -1, byval =
t)       ----
 

--  Bruce Momjian                        |  http://www.op.net/~candle maillist@candle.pha.pa.us            |  (610)
853-3000+  If your life is a hard drive,     |  830 Blythe Avenue +  Christ can be your backup.        |  Drexel Hill,
Pennsylvania19026