Re: Remove useless associativity/precedence from parsers - Mailing list pgsql-hackers

Akim Demaille <akim@lrde.epita.fr> writes:
> Honestly, I seriously doubt that you have contributors that don't
> have MacPorts or Brew installed, and both are pretty up to date on
> Bison.

Hm, well, I'm a counterexample ;-).  Right now you can develop PG
on a Mac just fine without any additional stuff, excepting maybe
OpenSSL if you want that.  If we have a strong reason to require
a newer Bison, I'd be willing to do so, but it needs to be a
strong reason.

>> * Speed of the generated parser could be better.

> Expect news this year about that.  I precisely came to look at
> PostgreSQL for this.

That's very cool news.

> Is there an easy way to bench pg and the various
> costs?  To be explicit: is there a way to see how long the parsing
> phase takes?  And some mighty inputs to bench against?

The easiest method is to fire up some client code that repeatedly
does whatever you want to test, and then look at perf or oprofile
or local equivalent to see where the time is going in the backend
process.

For the particular case of stressing the parser, probably the
best thing to look at is test cases that do a lot of low-overhead
DDL, such as creating views.  You could do worse than just repeatedly
sourcing our standard view files, like
    src/backend/catalog/system_views.sql
    src/backend/catalog/information_schema.sql
(In either case, I'd suggest adapting the file to create all
its objects in some transient schema that you can just drop.
Repointing information_schema.sql to some other schema is
trivial, just change a couple of commands at the top; and
you could tweak system_views.sql similarly.  Also consider
wrapping the whole thing in BEGIN; ... ROLLBACK; instead of
spending time on an explicit DROP.)

Somebody else might know of a better test case but I'd try
that first.

There would still be a fair amount of I/O and catalog lookup
overhead in a test run that way, but it would be an honest
approximation of useful real-world cases.  If you're willing to
put some blinders on and just micro-optimize the flex/bison
code, you could set up a custom function that just calls that
stuff.  I actually did that not too long ago; C code attached
for amusement's sake.

>> * Lack of run-time extensibility of the parser.  There are many PG
>> extensions that wish they could add things into the grammar, and can't.

> Are there documented examples of this?  What would that look like?

I'm just vaguely recalling occasional how-could-I-do-this complaints
on the pgsql-hackers mailing list.  Perhaps somebody else could say
something more concrete.

            regards, tom lane

/*

build this into a Postgres shared library, then

create function drive_parser(query text, count int) returns void
strict volatile language c as '.../drive_parser.so';

\timing

select drive_parser('some-query', 1000);

 */

#include "postgres.h"

#include "fmgr.h"
#include "miscadmin.h"
#include "tcop/tcopprot.h"
#include "utils/builtins.h"
#include "utils/memutils.h"

PG_MODULE_MAGIC;

/*
 * drive_parser(query text, count int) returns void
 */
PG_FUNCTION_INFO_V1(drive_parser);
Datum
drive_parser(PG_FUNCTION_ARGS)
{
    text       *txt = PG_GETARG_TEXT_PP(0);
    int32        count = PG_GETARG_INT32(1);
    char       *query_string = text_to_cstring(txt);
    MemoryContext mycontext;

    mycontext = AllocSetContextCreate(CurrentMemoryContext,
                                      "drive_parser work cxt",
                                      ALLOCSET_DEFAULT_SIZES);

    while (count-- > 0)
    {
        List       *parsetree_list;
        MemoryContext oldcontext;

        oldcontext = MemoryContextSwitchTo(mycontext);

        /* This times raw parsing only */
        parsetree_list = pg_parse_query(query_string);

        MemoryContextSwitchTo(oldcontext);

        MemoryContextReset(mycontext);

        CHECK_FOR_INTERRUPTS();
    }

    PG_RETURN_VOID();
}