Thread: function to filter out tokens (sql syntax lexical)?

Hallo,

according to
http://www.postgresql.org/docs/8.4/interactive/sql-syntax-lexical.htm
A token can be a key word, an identifier, a quoted identifier, a literal
(or constant), or a special character symbol.

I suppose these different tokens are used by the internal parser?
So my questions is,
is there a function one could use to filter out the specific tokes?

Example:

Given the statement: INSERT INTO MY_TABLE VALUES (3, 'hi there');

We can do something like:

select filter_tokens($$INSERT INTO MY_TABLE VALUES (3, 'hi there');$$);

With output:

token     |     type
Insert        keyword
into        keyword
my_table    identifier
values        keyword
..
'hi there'    literal

or in another way with similar functionality?

mvg,
Wim

Wim Bertels <wim.bertels@khleuven.be> writes:
> is there a function one could use to filter out the specific tokes?

In 8.4 and up, pg_get_keywords() might help.

            regards, tom lane

On Thu, 2011-04-14 at 16:53 -0400, Tom Lane wrote:
> Wim Bertels <wim.bertels@khleuven.be> writes:
> > is there a function one could use to filter out the specific tokes?
>
> In 8.4 and up, pg_get_keywords() might help.

indeed something i could use as reference

i'm actually looking for a way to compare two sql statement,
and to check if they are equivalent (not only semantically,
since this usually makes a difference in execution plan).

i suppose the postgresql internal must have some sort of functions
handling this, while parsing the statement.
i would be handy to have this function available as a db user.

tnx,
Wim

>
>             regards, tom lane
>