Thread: Select parser at runtime
I've been experimenting with using a different parser (one which is
more Oracle compatible). While the parser is not yet ready for prime
time, I thought I would send out the code used to select the parser to
gauge its level of acceptability. If this patch isn't going to fly,
then my approach to an alternate parser isn't going to fly either.
After this patch is applied, you can switch to a new parser by doing
something like this:
create function my_parser(opaque, opaque, opaque) returns opaque
as '..../parser.so' language 'c';
set parser = my_parser;
After you do this, all subsequent input will be interpreted using the
specified parser. Note that you may want to leave yourself an escape
hatch of some sort to set the parser back to Postgres standard.
If this patch is accepted, then some further work needs to be done to
set the parser for SPI calls, so that it is possible for the user to
change the parser while still using ordinary PL/pgSQL.
I would appreciate any feedback.
Ian
Index: src/backend/tcop/postgres.c
===================================================================
RCS file: /home/projects/pgsql/cvsroot/pgsql/src/backend/tcop/postgres.c,v
retrieving revision 1.230
diff -u -r1.230 postgres.c
--- src/backend/tcop/postgres.c 2001/08/04 00:14:43 1.230
+++ src/backend/tcop/postgres.c 2001/08/11 00:53:08
@@ -58,9 +58,11 @@
#include "tcop/utility.h"
#include "storage/proc.h"
#include "utils/exc.h"
+#include "utils/fcache.h"
#include "utils/guc.h"
#include "utils/memutils.h"
#include "utils/ps_status.h"
+#include "utils/syscache.h"
#ifdef MULTIBYTE
#include "mb/pg_wchar.h"
#endif
@@ -118,6 +120,10 @@
*/
int XfuncMode = 0;
+char *parser_function_name = 0;
+static bool update_parser_function_fcache = true;
+static FunctionCachePtr parser_function_fcache = NULL;
+
/* ----------------------------------------------------------------
* decls for routines only used in this file
* ----------------------------------------------------------------
@@ -389,8 +395,23 @@
if (Show_parser_stats)
ResetUsage();
+
+ if (update_parser_function_fcache)
+ assign_parser(parser_function_name);
+
+ if (parser_function_fcache == NULL)
+ raw_parsetree_list = parser(query_string, typev, nargs);
+ else
+ {
+ Datum result;
+
+ parser_function_fcache->fcinfo.arg[0] = PointerGetDatum(query_string);
+ parser_function_fcache->fcinfo.arg[1] = PointerGetDatum(typev);
+ parser_function_fcache->fcinfo.arg[2] = Int32GetDatum(nargs);
- raw_parsetree_list = parser(query_string, typev, nargs);
+ result = FunctionCallInvoke(&parser_function_fcache->fcinfo);
+ raw_parsetree_list = (List *) DatumGetPointer(result);
+ }
if (Show_parser_stats)
{
@@ -399,6 +420,82 @@
}
return raw_parsetree_list;
+}
+
+/*
+ * Check that we can find a parser function. This is called when the
+ * user assigns a value to the `parser' variable.
+ */
+bool
+check_parser(const char *proposed)
+{
+ HeapTuple tup;
+ Oid argtypes[FUNC_MAX_ARGS];
+
+ if (proposed[0] == '\0' || strcmp(proposed, "postgres") == 0)
+ return true;
+
+ /* We can't check this unless we have started running. */
+ if (! IsNormalProcessingMode())
+ return true;
+
+ memset(argtypes, 0, sizeof argtypes);
+ tup = SearchSysCache(PROCNAME,
+ PointerGetDatum(proposed),
+ Int32GetDatum(3),
+ PointerGetDatum(argtypes),
+ 0);
+ if (! HeapTupleIsValid(tup))
+ return false;
+ ReleaseSysCache(tup);
+ return true;
+}
+
+/*
+ * Assign a new parser function.
+ */
+void
+assign_parser(const char *value)
+{
+ FunctionCachePtr fcache;
+ HeapTuple tup;
+ Oid argtypes[FUNC_MAX_ARGS];
+ Oid oid;
+
+ /* We can't update parser_function_fcache until we have started
+ * running.
+ */
+ if (! IsNormalProcessingMode())
+ {
+ update_parser_function_fcache = true;
+ return;
+ }
+
+ if (value[0] == '\0' || strcmp(value, "postgres") == 0)
+ fcache = NULL;
+ else
+ {
+ memset(argtypes, 0, sizeof argtypes);
+ tup = SearchSysCache(PROCNAME,
+ PointerGetDatum(value),
+ Int32GetDatum(3),
+ PointerGetDatum(argtypes),
+ 0);
+ if (! HeapTupleIsValid(tup))
+ elog(ERROR, "parser function %s does not exist", value);
+
+ oid = tup->t_data->t_oid;
+
+ ReleaseSysCache(tup);
+
+ fcache = init_fcache(oid, 3, TopMemoryContext);
+ }
+
+ if (parser_function_fcache != NULL)
+ pfree(parser_function_fcache);
+ parser_function_fcache = fcache;
+
+ update_parser_function_fcache = false;
}
/*
Index: src/backend/utils/misc/guc.c
===================================================================
RCS file: /home/projects/pgsql/cvsroot/pgsql/src/backend/utils/misc/guc.c,v
retrieving revision 1.45
diff -u -r1.45 guc.c
--- src/backend/utils/misc/guc.c 2001/07/05 15:19:40 1.45
+++ src/backend/utils/misc/guc.c 2001/08/11 00:53:11
@@ -376,6 +376,9 @@
{"krb_server_keyfile", PGC_POSTMASTER, &pg_krb_server_keyfile,
PG_KRB_SRVTAB, NULL, NULL},
+ {"parser", PGC_USERSET, &parser_function_name, "",
+ check_parser, assign_parser},
+
#ifdef ENABLE_SYSLOG
{"syslog_facility", PGC_POSTMASTER, &Syslog_facility,
"LOCAL0", check_facility, NULL},
Index: src/include/tcop/tcopprot.h
===================================================================
RCS file: /home/projects/pgsql/cvsroot/pgsql/src/include/tcop/tcopprot.h,v
retrieving revision 1.41
diff -u -r1.41 tcopprot.h
--- src/include/tcop/tcopprot.h 2001/06/08 21:16:48 1.41
+++ src/include/tcop/tcopprot.h 2001/08/11 00:53:12
@@ -31,6 +31,8 @@
extern bool HostnameLookup;
extern bool ShowPortNumber;
+extern char *parser_function_name;
+
#ifndef BOOTSTRAP_INCLUDE
extern List *pg_parse_and_rewrite(char *query_string,
@@ -39,6 +41,8 @@
extern void pg_exec_query_string(char *query_string,
CommandDest dest,
MemoryContext parse_context);
+extern bool check_parser(const char *proposed);
+extern void assign_parser(const char *value);
#endif /* BOOTSTRAP_INCLUDE */
Ian Lance Taylor <ian@airs.com> writes:
> I've been experimenting with using a different parser (one which is
> more Oracle compatible).
Hmm. What we have here is a mechanism to swap out the entire
backend/parser/ subdirectory --- and nothing else. Somehow this seems
like the wrong granularity. parser/ is an awful lot of code to replace
to make a few random tweaks that don't affect query semantics. Since
you aren't changing the querytree representation nor any of the rewrite/
plan/execute pipeline, it's hard to see how you can do more with this
than very marginal syntax hacks. But if that's all you want to do,
seems like replacing pieces of the parser/semantic analyzer is the right
mechanism, not the whole durn thing.
> Note that you may want to leave yourself an escape
> hatch of some sort to set the parser back to Postgres standard.
> If this patch is accepted, then some further work needs to be done to
> set the parser for SPI calls, so that it is possible for the user to
> change the parser while still using ordinary PL/pgSQL.
I think both of these issues would need to be addressed before, not
after, considering the patch for acceptance. In particular, how do we
cater for both plpgsql and a true "PL/SQL" language that uses the Oracle
parser?
> + {
> + Datum result;
> +
> + parser_function_fcache->fcinfo.arg[0] = PointerGetDatum(query_string);
> + parser_function_fcache->fcinfo.arg[1] = PointerGetDatum(typev);
> + parser_function_fcache->fcinfo.arg[2] = Int32GetDatum(nargs);
> - raw_parsetree_list = parser(query_string, typev, nargs);
> + result = FunctionCallInvoke(&parser_function_fcache->fcinfo);
> + raw_parsetree_list = (List *) DatumGetPointer(result);
> + }
Use FunctionCall3() to hide the cruft here.
> + fcache = init_fcache(oid, 3, TopMemoryContext);
This is a tad odd. Why don't you just do fmgr_info and store an FmgrInfo
structure? You have no use for the rest of an executor fcache
structure.
The update_parser_function_fcache business bothers me, too. I see the
problem: it doesn't work to do this lookup when postgresql.conf is read
in the postmaster. However, I really don't like the notion of disabling
check_parser and allowing a possibly-bogus value to be assigned on
faith. (If the function name *is* bogus, your code can never recover;
at the very least you ought to clear update_parser_function_fcache
before failing.) Given the extent to which the parser is necessarily
tied to the rest of the system, I'm not sure there's any value in
allowing it to be implemented as a dynamic-link function. I'd be more
than half inclined to go with a lower-tech solution wherein you expect
the alternate parser to be permanently linked in and known to the
check_parser and assign_parser subroutines. Then the state variable is
just a C function pointer, and assign_parser looks something like
#ifdef ORACLE_PARSER_SUPPORTED
if (strcasecmp(value, "Oracle") == 0)
parser_fn = oracle_parser;
else
#endif
regards, tom lane
Tom Lane <tgl@sss.pgh.pa.us> writes: > Ian Lance Taylor <ian@airs.com> writes: > > I've been experimenting with using a different parser (one which is > > more Oracle compatible). > > Hmm. What we have here is a mechanism to swap out the entire > backend/parser/ subdirectory --- and nothing else. Somehow this seems > like the wrong granularity. parser/ is an awful lot of code to replace > to make a few random tweaks that don't affect query semantics. Since > you aren't changing the querytree representation nor any of the rewrite/ > plan/execute pipeline, it's hard to see how you can do more with this > than very marginal syntax hacks. But if that's all you want to do, > seems like replacing pieces of the parser/semantic analyzer is the right > mechanism, not the whole durn thing. This patch doesn't actually replace the entire backend/parser subdirectory. It mainly only replaces scan.l and gram.y. This is because the code in postgres.c still passes the result of the replaced parser to pg_analyze_and_rewrite(). You are absolutely correct that at this point this can only do marginal syntax hacks. But since my goal is to support Oracle syntax, marginal syntax hacks are just what is needed. For example, it's hard to tweak the existing parser to support Oracle, because the set of reserved words is different, and because Oracle uses different names for datatypes. Also, CREATE FUNCTION in Oracle doesn't use a literal string, it actually switches to a different parser while parsing the function. It's not impossible to hack these into the Postgres parser; just hard, and a complex maintenance problem. It's true that increased Oracle compatibility will require code changes in other parts of the backend. > > Note that you may want to leave yourself an escape > > hatch of some sort to set the parser back to Postgres standard. > > If this patch is accepted, then some further work needs to be done to > > set the parser for SPI calls, so that it is possible for the user to > > change the parser while still using ordinary PL/pgSQL. > > I think both of these issues would need to be addressed before, not > after, considering the patch for acceptance. In particular, how do we > cater for both plpgsql and a true "PL/SQL" language that uses the Oracle > parser? I agree that these issues need to be addressed before the patch is accepted. I want to get a sense of whether I am on the right track at all. As noted above, PL/SQL needs to be more closely tied to the Oracle parser than PL/pgSQL is to the Postgres parser. I have been thinking in terms of a parser stack. Code which uses SPI could push the correct parser on top of the stack, and pop it off when done. Perhaps SPI would always use the postgres parser unless explicitly directed otherwise. The PL/SQL language interface would direct otherwise. Thanks for your other comments. I went with a dynamic link approach because it seemed minimally intrusive. Using a straight function pointer is certainly easier. Ian
Ian Lance Taylor <ian@airs.com> writes:
> This patch doesn't actually replace the entire backend/parser
> subdirectory. It mainly only replaces scan.l and gram.y. This is
> because the code in postgres.c still passes the result of the replaced
> parser to pg_analyze_and_rewrite().
Oh, of course, how silly of me. I was thinking that that call did the
analyze step too, but you're correct that it does not. Okay, replacing
lexer+syntaxer is a more reasonable chunk-size. (AFAIK there's no good
way to replace just part of a yacc/bison grammar on the fly, so you
couldn't go to a finer grain anyway, could you?)
regards, tom lane
Ian Lance Taylor writes: > I've been experimenting with using a different parser (one which is > more Oracle compatible). Why aren't you trying to add the missing pieces to the existing parser? -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Peter Eisentraut <peter_e@gmx.net> writes: > Ian Lance Taylor writes: > > > I've been experimenting with using a different parser (one which is > > more Oracle compatible). > > Why aren't you trying to add the missing pieces to the existing parser? Because my goal is a parser which can accept Oracle code directly, and Oracle does not use the same SQL syntax as Postgres. They are, of course, very similar, but it is not the case that all the differences are items missing from the Postgres parser. Some of them are items which Postgres does in a different, typically more standards- conforming, way. For example: the datatypes have different names; the set of reserved words is different; Oracle uses a weird syntax for outer joins. Ian
Ian Lance Taylor writes: > Because my goal is a parser which can accept Oracle code directly, and > Oracle does not use the same SQL syntax as Postgres. They are, of > course, very similar, but it is not the case that all the differences > are items missing from the Postgres parser. Some of them are items > which Postgres does in a different, typically more standards- > conforming, way. I'm not sure whether I like the notion of having to maintain multiple parsers in the future. We have always been quite fair in accepting extensions and aliases for compatibility, so I don't see a problem there. Then again, we're implemented an SQL server, not an Oracle server. If you want to convert your application there's this ora2pg thing. > For example: the datatypes have different names; the set of reserved > words is different; Unless you have implemented a different parsing algorithm or want to rip out features you're going to have a hard time changing the set of reserved words. > Oracle uses a weird syntax for outer joins. We had already rejected this idea. The earlier this disappears from the face of the earth the better. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Ian Lance Taylor <ian@airs.com> writes:
> For example: the datatypes have different names; the set of reserved
> words is different; Oracle uses a weird syntax for outer joins.
Is it really possible to fix these things strictly in the parser
(ie, without any semantic analysis)? For example, I don't quite see
how you're going to translate Oracle-style outer joins to SQL standard
style without figuring out which fields belong to which relations.
Keep in mind the cardinal rule for the parsing step: Thou Shalt Not
Do Any Database Access (because the parser must work even in
transaction-aborted state, else how do we recognize ROLLBACK command?)
regards, tom lane
> > Oracle uses a weird syntax for outer joins. > > We had already rejected this idea. The earlier this disappears from the > face of the earth the better. Sure, but until that happens, if we can easily give people portability, why not? My idea was to have a gram.y that rewrote _some_ of the commands, and output another query string that could then be passed into the existing gram.y. Kind of crazy, but it may limit the duplication of code in the two gram.y's. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Peter Eisentraut <peter_e@gmx.net> writes:
>> Oracle uses a weird syntax for outer joins.
> We had already rejected this idea. The earlier this disappears from the
> face of the earth the better.
Now now, what about our goal of Postgres world domination? Gonna be
tough to get there unless we can assimilate Oracle users ;-)
I do want to keep that brain-damaged syntax at arm's length, which
suggests a separate parser rather than merging it into our main
parser. I'm not convinced a translation can be done at the grammar
level with no semantic analysis --- but if Ian thinks he can do it,
let him try...
regards, tom lane
> Peter Eisentraut <peter_e@gmx.net> writes: > >> Oracle uses a weird syntax for outer joins. > > > We had already rejected this idea. The earlier this disappears from the > > face of the earth the better. > > Now now, what about our goal of Postgres world domination? Gonna be > tough to get there unless we can assimilate Oracle users ;-) "Fortress PostgreSQL", Lamar Owen said. We do have fortress feel, don't we? > I do want to keep that brain-damaged syntax at arm's length, which > suggests a separate parser rather than merging it into our main > parser. I'm not convinced a translation can be done at the grammar > level with no semantic analysis --- but if Ian thinks he can do it, > let him try... My guess is that he is going to need some changes in the /parser but those will be minor and triggered by some Oracle flag. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Tom Lane <tgl@sss.pgh.pa.us> writes: > Ian Lance Taylor <ian@airs.com> writes: > > For example: the datatypes have different names; the set of reserved > > words is different; Oracle uses a weird syntax for outer joins. > > Is it really possible to fix these things strictly in the parser > (ie, without any semantic analysis)? For example, I don't quite see > how you're going to translate Oracle-style outer joins to SQL standard > style without figuring out which fields belong to which relations. > Keep in mind the cardinal rule for the parsing step: Thou Shalt Not > Do Any Database Access (because the parser must work even in > transaction-aborted state, else how do we recognize ROLLBACK command?) I admit that I haven't sorted out the outer join thing yet. The others are easy enough. Ian
Peter Eisentraut <peter_e@gmx.net> writes: > I'm not sure whether I like the notion of having to maintain multiple > parsers in the future. We have always been quite fair in accepting > extensions and aliases for compatibility, so I don't see a problem there. > Then again, we're implemented an SQL server, not an Oracle server. If you > want to convert your application there's this ora2pg thing. An approach such as ora2pg solves a specific problem. It doesn't really solve the general problem of people familiar with Oracle who want to use Postgres. It doesn't at all solve my problem, which is to have an application which can easily speak to either Oracle or Postgres. > > For example: the datatypes have different names; the set of reserved > > words is different; > > Unless you have implemented a different parsing algorithm or want to rip > out features you're going to have a hard time changing the set of reserved > words. That's why I'm implementing a different parsing algorithm. Ian
Tom Lane writes: > Now now, what about our goal of Postgres world domination? Gonna be > tough to get there unless we can assimilate Oracle users ;-) In order to achieve world domination you don't want to offer compatibility, otherwise your users could move back and forth easily. What you want is static conversion tools so people can move to your product but not back to others. -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Peter Eisentraut <peter_e@gmx.net> writes: > Tom Lane writes: > > > Now now, what about our goal of Postgres world domination? Gonna be > > tough to get there unless we can assimilate Oracle users ;-) > > In order to achieve world domination you don't want to offer > compatibility, otherwise your users could move back and forth easily. > What you want is static conversion tools so people can move to your > product but not back to others. I disagree. To achieve world domination you should lower to barriers to adoption as much as possible, and then keep people with you due to the superiority of your product. If the barriers to adoption are high, people won't take the risk, and won't discover the superiority. Incompatible syntax is a barrier to adoption because people fear the time required to learn the new syntax, and they fear adopting Postgres and then discovering after three months of enhancements to their Postgres code that Postgres won't do the job and they have to switch back. Ian
> Tom Lane <tgl@sss.pgh.pa.us> writes: > > > Ian Lance Taylor <ian@airs.com> writes: > > > For example: the datatypes have different names; the set of reserved > > > words is different; Oracle uses a weird syntax for outer joins. > > > > Is it really possible to fix these things strictly in the parser > > (ie, without any semantic analysis)? For example, I don't quite see > > how you're going to translate Oracle-style outer joins to SQL standard > > style without figuring out which fields belong to which relations. > > Keep in mind the cardinal rule for the parsing step: Thou Shalt Not > > Do Any Database Access (because the parser must work even in > > transaction-aborted state, else how do we recognize ROLLBACK command?) > > I admit that I haven't sorted out the outer join thing yet. The > others are easy enough. > Another idea is to put the Oracle stuff in gram.y, but use #ifdef or something to mark the Oracle parts, and run gram.y through yacc/bison with the Oracle defines visiable, and another time to create a second parse state machine without Oracle. I think Jan did that for something like that once. -- Bruce Momjian | http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 853-3000+ If your life is a hard drive, | 830 Blythe Avenue + Christ can be your backup. | Drexel Hill, Pennsylvania19026
Hi guys, Not sure if Peter was joking, but Ian's approach sounds much more user-friendly. Getting Oracle users to convert to PostgreSQL then be "stuck-with-it" because they can't afford the migration elsewhere is not the right approach. PostgreSQL is a really good product, and the best way to emphasise it is "here's PostgreSQL, people use it coz it *works better*". And that's definitely achieveable. :) Regards and best wishes, Justin Clift Ian Lance Taylor wrote: > > Peter Eisentraut <peter_e@gmx.net> writes: > > > Tom Lane writes: > > > > > Now now, what about our goal of Postgres world domination? Gonna be > > > tough to get there unless we can assimilate Oracle users ;-) > > > > In order to achieve world domination you don't want to offer > > compatibility, otherwise your users could move back and forth easily. > > What you want is static conversion tools so people can move to your > > product but not back to others. > > I disagree. To achieve world domination you should lower to barriers > to adoption as much as possible, and then keep people with you due to > the superiority of your product. If the barriers to adoption are > high, people won't take the risk, and won't discover the superiority. > > Incompatible syntax is a barrier to adoption because people fear the > time required to learn the new syntax, and they fear adopting Postgres > and then discovering after three months of enhancements to their > Postgres code that Postgres won't do the job and they have to switch > back. > > Ian > > ---------------------------(end of broadcast)--------------------------- > TIP 3: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
On Mon, 13 Aug 2001, Justin Clift wrote: > Hi guys, > > Not sure if Peter was joking, but Ian's approach sounds much more > user-friendly. > > Getting Oracle users to convert to PostgreSQL then be "stuck-with-it" > because they can't afford the migration elsewhere is not the right > approach. If you think that people are going to flock to PostgreSQL from Oracle simply because it's a drop in replacement, I want some of whatever it is you're drinking! An Oracle compatibility mode wouldn't be a bad idea, but at what cost and at how much effort? What are you going to do with incompatible reserved words? Who do you expect to do it? How soon? I've seen alot of projects try to make themselves "user-friendly" only to suffer in the end from what they lost in the effort. Personally I'd prefer a PostgreSQL that was as SQL92 and beyond as it could possibly be rather than some of this and some of that. Vince. -- ========================================================================== Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net 56K Nationwide Dialup from $16.00/mo atPop4 Networking Online Campground Directory http://www.camping-usa.com Online Giftshop Superstore http://www.cloudninegifts.com ==========================================================================
On Sun, Aug 12, 2001 at 10:21:55PM -0400, Vince Vielhaber wrote: > > If you think that people are going to flock to PostgreSQL from Oracle > simply because it's a drop in replacement, I want some of whatever it > is you're drinking! > > An Oracle compatibility mode wouldn't be a bad idea, but at what cost > and at how much effort? What are you going to do with incompatible > reserved words? Who do you expect to do it? How soon? I've seen > alot of projects try to make themselves "user-friendly" only to suffer > in the end from what they lost in the effort. > > Personally I'd prefer a PostgreSQL that was as SQL92 and beyond as it > could possibly be rather than some of this and some of that. Compatability modes have a good side and a bad side: they make it easy for DBAs to move existing installations to your product at little cost, then slowly upgrade to using native features. The OpenACS project would have loved to have this possibility. Many of our current Oracle compatability features came about to ease such ports, right? However, they suffer from the problem of allowing continued operation in 'compatability' mode: there is no incentive for DB using project to use the native interfaces or capabilites, even if they are better. This is how OS/2's Windows compatabilty mode killed it in the long term: the 'defacto standard' problem. Would the OpenACS project even exist, if a full Oracle mode had existed at the time? So, I think you're right, in the long run it's best to conform to an open standard than try to chase a commercial product. Ross P.S. Of course, as an open project, anyone can write anything they want. If we end up with a 'better Oracle than Oracle', so be it.
Vince Vielhaber <vev@michvhf.com> writes: > An Oracle compatibility mode wouldn't be a bad idea, but at what cost > and at how much effort? That is why I focused on the relatively minor changes to Postgres required to hook in an alternate parser. I personally would not expect the mainline Postgres team to worry about Oracle support. But if an Oracle parser can be decoupled from the mainline of Postgres work, then most of the cost will be paid by the people who care about it. (Not all of the cost, because some communication will be required when the parse tree nodes are changed.) Along these lines, I don't think Bruce's suggestion of modifications to the Postgres gram.y is a good idea, because it causes the Oracle parser to add an ongoing cost to the Postgres parser. Ian
Hi Vince, The point I'll make is this : People who presently have installations on Oracle will be more inclined to test/trial PostgreSQL if they know the learning curve is much less than say, migrating to DB2 would be (or some other database without specific Oracle-transition compatibilities). Sure, they might move their installations to PostgreSQL-with-an-Oracle-like-parser and then never convert them to pure PostgreSQL. So? Does it matter? Probably not, they're still using PostgreSQL. I'm pretty sure over time newer projects and installations would become more PostgreSQL oriented as the DBA's gained more experience and understanding of PostgreSQL's strengths. i.e. "Chalk up a win." Vince Vielhaber wrote: > > On Mon, 13 Aug 2001, Justin Clift wrote: > > > Hi guys, > > > > Not sure if Peter was joking, but Ian's approach sounds much more > > user-friendly. > > > > Getting Oracle users to convert to PostgreSQL then be "stuck-with-it" > > because they can't afford the migration elsewhere is not the right > > approach. > > If you think that people are going to flock to PostgreSQL from Oracle > simply because it's a drop in replacement, I want some of whatever it > is you're drinking! If PostgreSQL was truly a drop-in-replacement then cost and good reputation (especially over the coming years) would mean a lot of places would use us instead of Oracle. Presently though, we're not a drop-in-replacement. > An Oracle compatibility mode wouldn't be a bad idea, but at what cost > and at how much effort? What are you going to do with incompatible > reserved words? Who do you expect to do it? How soon? I've seen > alot of projects try to make themselves "user-friendly" only to suffer > in the end from what they lost in the effort. The cost and effort is purely voluntary. :) i.e. $0-ish cost, and heaps of effort. > Personally I'd prefer a PostgreSQL that was as SQL92 and beyond as it > could possibly be rather than some of this and some of that. I don't see how having alternate parsers available, maintained and updated by those interested in them, is a bad thing. Certainly don't see how it detracts from the main effort. ??? > Vince. > -- > ========================================================================== > Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net > 56K Nationwide Dialup from $16.00/mo at Pop4 Networking > Online Campground Directory http://www.camping-usa.com > Online Giftshop Superstore http://www.cloudninegifts.com > ========================================================================== Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi
Ian Lance Taylor wrote: > Along these lines, I don't think Bruce's suggestion of modifications > to the Postgres gram.y is a good idea, because it causes the Oracle > parser to add an ongoing cost to the Postgres parser. Bruce, Tom and I discussed these issues during our time in San Diego last month. If we want to have both parsers available at runtime we need to replace the YY (case-insensitive) prefix in the generated files per parser and call the right one from tcop. Now for some flex/bison combo's at least the prefix switches(to have something different than YY) don't work reliable. There will be some global YY-objects left, causinglinkage problems. That's why PL/pgSQL's scanner/parser's C-code is run through sed(1). If Bruce's suggestion of having both parsers in one source with #ifdef, #else, #endif is better than separate sources depends mainly on how big the differences finally will be. Doesn't really bother me. Maybe we could start with a combined one and separate later if it turns out that they drift apart too much? Jan -- #======================================================================# # It's easier to get forgiveness for being wrong than for being right. # # Let's break this rule - forgive me. # #================================================== JanWieck@Yahoo.com # _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com
Ian Lance Taylor <ian@airs.com> writes:
> ... most of the cost will be paid by the people who care about
> it. (Not all of the cost, because some communication will be required
> when the parse tree nodes are changed.)
> Along these lines, I don't think Bruce's suggestion of modifications
> to the Postgres gram.y is a good idea, because it causes the Oracle
> parser to add an ongoing cost to the Postgres parser.
And managing grammar changes and parse-tree-node changes is not an
ongoing cost? I beg to differ. We do that a lot, and keeping multiple
grammar files in sync is not a pleasant prospect. (Look at ecpg ---
it's a major pain to keep it in sync with the main parser, even though
it only shares productions and not output code. Worse, I have zero
confidence that it actually *is* in sync.)
If the grammar changes are small and localized, I think Bruce's #ifdef
approach might well be the way to go. However, we're speculating in
a vacuum here, not having seen the details of the changes needed.
regards, tom lane
Jan Wieck <JanWieck@Yahoo.com> writes:
> files per parser and call the right one from tcop. Now for
> some flex/bison combo's at least the prefix switches (to have
> something different than YY) don't work reliable. There will
> be some global YY-objects left, causing linkage problems.
> That's why PL/pgSQL's scanner/parser's C-code is run through
> sed(1).
The only reason plpgsql's parser is still run through sed is that
I haven't gotten around to changing it ;-). The main system has
depended on -P for awhile, and we've seen no reports of trouble.
(This is not unrelated to the fact that we now ship pre-yacced and
pre-lexed .c files, no doubt. Only people who pull from CVS ever
rebuild the files at all, and we tell them they must use up-to-date
flex and bison. This policy seems to work a lot better than the old
way of trying to work with whatever broken tools a particular platform
might have...)
regards, tom lane
Jan Wieck <JanWieck@Yahoo.com> writes: > If we want to have both parsers available at runtime we need > to replace the YY (case-insensitive) prefix in the generated > files per parser and call the right one from tcop. Now for > some flex/bison combo's at least the prefix switches (to have > something different than YY) don't work reliable. There will > be some global YY-objects left, causing linkage problems. > That's why PL/pgSQL's scanner/parser's C-code is run through > sed(1). This is a solved problem. gdb, for example, links together four different Yacc-based parsers, without even using bison's -p option. Ian
Tom Lane <tgl@sss.pgh.pa.us> writes: > Ian Lance Taylor <ian@airs.com> writes: > > ... most of the cost will be paid by the people who care about > > it. (Not all of the cost, because some communication will be required > > when the parse tree nodes are changed.) > > > Along these lines, I don't think Bruce's suggestion of modifications > > to the Postgres gram.y is a good idea, because it causes the Oracle > > parser to add an ongoing cost to the Postgres parser. > > And managing grammar changes and parse-tree-node changes is not an > ongoing cost? I beg to differ. We do that a lot, and keeping multiple > grammar files in sync is not a pleasant prospect. (Look at ecpg --- > it's a major pain to keep it in sync with the main parser, even though > it only shares productions and not output code. Worse, I have zero > confidence that it actually *is* in sync.) Parse tree node changes are definitely an ongoing cost, as I mention in the first quoted paragraph. However, I would see this as a communication issue from the people maintaining the core parser to the people maintaining the Oracle parser. Perhaps it will be possible to better formalize the parse tree node interface. (This approach is from my gcc experience. The various gcc parsers (C, C++, etc.) generate tree nodes. The structure of the tree nodes does change from time to time, forcing all the other parsers to change. This is generally driven by the needs of some parser. Different groups of people maintain each parser.) I'm not sure what you mean by managing grammar changes, although perhaps I am reading too much into that. The Oracle grammar is set by Oracle, and will not change even if the Postgres grammar changes. > If the grammar changes are small and localized, I think Bruce's #ifdef > approach might well be the way to go. However, we're speculating in > a vacuum here, not having seen the details of the changes needed. I've been trying to avoid the details, because that is just going to raise another discussion which I honestly believe is orthogonal to this discussion. However, a quick sketch: proper handling of Oracle reserved words is very difficult in a Yacc-based parser, because the set of reserved words effectively changes on a contextual basis. The Postgres grammar uses various workarounds for this, but doesn't handle everything entirely correctly from the point of view of the Oracle language. Therefore, my prototype Oracle grammar is actually a recursive descent parser, not Yacc-based at all. Ian
Ian Lance Taylor writes: > I'm not sure what you mean by managing grammar changes, although > perhaps I am reading too much into that. The Oracle grammar is set by > Oracle, and will not change even if the Postgres grammar changes. Things like VACUUM and ANALYZE, which you will have to keep unless you want to implement an Oracle storage manager as well. ;-) -- Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter
Peter Eisentraut <peter_e@gmx.net> writes:
> Things like VACUUM and ANALYZE, which you will have to keep unless you
> want to implement an Oracle storage manager as well. ;-)
Evidently Ian is just interested in a parser that could be used by
Oracle-compatible applications, which'd not be invoking such operations
anyway. Maintenance scripts would have to use the regular PG parser.
That doesn't seem unreasonable.
Based on his further explanation, it seems that tracking grammar changes
wouldn't be an issue, but tracking parsetree changes definitely would
be. I'm also still concerned about whether this can be done within the
parse step (no database access).
regards, tom lane