Thread: idea, proposal: only preloadable libraries (conditional load)
Hello I am searching way to extensible parser (actually not bison, only transformations). I propose parserHook (transformation part). One Tom's objection is difference between heooked and unhooked parser. It serious problem. I have one idea - only preloadable libraries. These libs have to be specified in local_preload_libraries variable, and cannot be initialised by LOAD statement. Statement LOAD should by used only for reload library. Because pre loaded library is initialised before first SQL statement should be executed, then hook is installed on time and an behave is consistent. One technique is an calling library's function "canload" before initialising. When canload returns false, then dfmgr unloads lib. I thing so this behave complements current functions PG_init and PG_finit. What I can understand, PG_init, cannot throw exception or signalise any problems with initialisation. Ideas, objections? regards Pavel Stehule
On Wed, 2009-03-11 at 11:09 +0100, Pavel Stehule wrote: > I am searching way to extensible parser (actually not bison, only > transformations). I propose parserHook (transformation part). One > Tom's objection is difference between heooked and unhooked parser. It > serious problem. Do you mean hooking the whole parser? That sounds more useful and less hassle than trying to hook parts of it. That would be just one check to see if the hook exists per statement, rather than potentially thousands of times per statement. I'd go for an implementation that uses pg_language to store new languages, just with lanispl = false. We can then have a new parameter session_language (TEXT) with 'internal' as default. session_language cannot be reset while connected. That would allow us to have multiple session languages in use at one time and to add new ones (or modify existing ones) without changing core behaviour. In the longer term it will be very useful to have the ability to support multiple language variants, including older PostgreSQL syntax to allow legacy systems to work with Postgres at the same time as allowing new development to continue. > I have one idea - only preloadable libraries. These libs have to be > specified in local_preload_libraries variable, and cannot be > initialised by LOAD statement. Statement LOAD should by used only for > reload library. Because pre loaded library is initialised before first > SQL statement should be executed, then hook is installed on time and > an behave is consistent. One technique is an calling library's > function "canload" before initialising. When canload returns false, > then dfmgr unloads lib. I thing so this behave complements current > functions PG_init and PG_finit. What I can understand, PG_init, cannot > throw exception or signalise any problems with initialisation. I remember I had some differences between the way loading occurs at session start and as a result of a LOAD command. I think there's probably already a way of doing this - probably by checking for something that would only be there *after* having read reloadable libraries but before main session starts. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
On Wed, Mar 11, 2009 at 12:56 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > > In the longer term it will be very useful to have the ability to support > multiple language variants, including older PostgreSQL syntax to allow > legacy systems to work with Postgres at the same time as allowing new > development to continue. So I think having multiple parsers for different versions of Pg backwards compatibility is an awful idea. It would be a huge maintenance headache since every time we change a structure that the parser works someone would have to maintain all those compatibility parsers. And it's very rare that we make non-backwards compatible changes to the grammar with the exception of adding new reserved keywords. However that last thought led me to an interesting idea. We could fairly easily support SQL which used keywords which we later reserved. We could do this by marking each keyword in keywords.c with a version number that introduced it. Then have a guc which tells the lexer which version to target -- any keywords introduced after the desired version can just be passed up as regular urecognized identifiers. That would allow us to add new keywords more freely -- I think still not liberally since we would rather people not be forced to decide between new features and a working application. Hm, actually I see a fly in the ointment -- we often upgrade keywords from one kind of reservedness to another. That would mean we wouldn't be able to handle something like WITH which was previously some flavour of unreserved keyword and later became reserved. -- greg
On Wed, 2009-03-11 at 13:14 +0000, Greg Stark wrote: > On Wed, Mar 11, 2009 at 12:56 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > > > > In the longer term it will be very useful to have the ability to support > > multiple language variants, including older PostgreSQL syntax to allow > > legacy systems to work with Postgres at the same time as allowing new > > development to continue. > > > So I think having multiple parsers for different versions of Pg > backwards compatibility is an awful idea. There are a number of people interested in producing open source compatibility layers. I realise you may not be one of them, but that's no reason to stop the idea from taking root. > It would be a huge > maintenance headache since every time we change a structure that the > parser works someone would have to maintain all those compatibility > parsers. If it's a plugin that "someone" isn't any concern of ours. External projects can keep up with releases, or specific customer implementations may simply choose to standardise on one release and go with that. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
2009/3/11 Simon Riggs <simon@2ndquadrant.com>: > > On Wed, 2009-03-11 at 11:09 +0100, Pavel Stehule wrote: > >> I am searching way to extensible parser (actually not bison, only >> transformations). I propose parserHook (transformation part). One >> Tom's objection is difference between heooked and unhooked parser. It >> serious problem. > > Do you mean hooking the whole parser? That sounds more useful and less > hassle than trying to hook parts of it. That would be just one check to > see if the hook exists per statement, rather than potentially thousands > of times per statement. No, now I want to add hook only to current parser - concretely to transformations, although I can imagine any hook over whole parser. It could help with modules, that adds non sql statements like "show statistic", "show ... ", "explain " and others - all service statements, some extensions ... Nearest goal is support for some smart functions like decode, greatest, xmlelement, ... > > I'd go for an implementation that uses pg_language to store new > languages, just with lanispl = false. We can then have a new parameter > session_language (TEXT) with 'internal' as default. session_language > cannot be reset while connected. That would allow us to have multiple > session languages in use at one time and to add new ones (or modify > existing ones) without changing core behaviour. > > In the longer term it will be very useful to have the ability to support > multiple language variants, including older PostgreSQL syntax to allow > legacy systems to work with Postgres at the same time as allowing new > development to continue. > >> I have one idea - only preloadable libraries. These libs have to be >> specified in local_preload_libraries variable, and cannot be >> initialised by LOAD statement. Statement LOAD should by used only for >> reload library. Because pre loaded library is initialised before first >> SQL statement should be executed, then hook is installed on time and >> an behave is consistent. One technique is an calling library's >> function "canload" before initialising. When canload returns false, >> then dfmgr unloads lib. I thing so this behave complements current >> functions PG_init and PG_finit. What I can understand, PG_init, cannot >> throw exception or signalise any problems with initialisation. > > I remember I had some differences between the way loading occurs at > session start and as a result of a LOAD command. I think there's > probably already a way of doing this - probably by checking for > something that would only be there *after* having read reloadable > libraries but before main session starts. > I can test debug_query_string, but main problem is impossibility throw exception inside PG_init. regards Pavel Stehule > -- > Simon Riggs www.2ndQuadrant.com > PostgreSQL Training, Services and Support > >
On Wed, Mar 11, 2009 at 2:18 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > >> It would be a huge >> maintenance headache since every time we change a structure that the >> parser works someone would have to maintain all those compatibility >> parsers. > > If it's a plugin that "someone" isn't any concern of ours. External > projects can keep up with releases, or specific customer implementations > may simply choose to standardise on one release and go with that. That's not what I mean. I mean, for example, if someone adds a field to any of the structss in parsenodes.h to implement a new feature. The old parser would have to know how to initialize that field correctly to avoid triggering that new feature or trigger it in a manner compatible with the old version's implicit behaviour. The last few commits to that file include Tom's commit to handle ALTER TABLE SET WITHOUT OIDS, Alvaro's commit to handle reloptions with qualifiers, Stephen Frost's patch to support column-level privileges, Heikki's commit to handle vacuum_freeze_table_age, etc. Every one of these commits would have had to adjust every single old parser to generate the correct data for the changed nodes. The parser isn't a separable module interacting with the rest of the system through a static interface. It's closely in bed with the rest of the system implementing the syntax it's parsing. Every feature the parser can parse has to be communicated to the backend code implementing the feature so it has to have a corresponding knob in the interface between the parser and the rest of the system. -- greg
On Wed, 2009-03-11 at 14:45 +0000, Greg Stark wrote: > On Wed, Mar 11, 2009 at 2:18 PM, Simon Riggs <simon@2ndquadrant.com> wrote: > > > >> It would be a huge > >> maintenance headache since every time we change a structure that the > >> parser works someone would have to maintain all those compatibility > >> parsers. > > > > If it's a plugin that "someone" isn't any concern of ours. External > > projects can keep up with releases, or specific customer implementations > > may simply choose to standardise on one release and go with that. > > That's not what I mean. I mean, for example, if someone adds a field > to any of the structss in parsenodes.h to implement a new feature. The > old parser would have to know how to initialize that field correctly > to avoid triggering that new feature or trigger it in a manner > compatible with the old version's implicit behaviour. > > The last few commits to that file include Tom's commit to handle ALTER > TABLE SET WITHOUT OIDS, Alvaro's commit to handle reloptions with > qualifiers, Stephen Frost's patch to support column-level privileges, > Heikki's commit to handle vacuum_freeze_table_age, etc. > > Every one of these commits would have had to adjust every single old > parser to generate the correct data for the changed nodes. > > The parser isn't a separable module interacting with the rest of the > system through a static interface. It's closely in bed with the rest > of the system implementing the syntax it's parsing. Every feature the > parser can parse has to be communicated to the backend code > implementing the feature so it has to have a corresponding knob in the > interface between the parser and the rest of the system. It would be a matter for a plugin designer how they did that. If the new parser involved just some changes in specific areas, then presumably you would design it as a drop through parser: handle any special cases or drop through to normal Postgres parser. -- Simon Riggs www.2ndQuadrant.comPostgreSQL Training, Services and Support
Pavel Stehule <pavel.stehule@gmail.com> writes:
> 2009/3/11 Simon Riggs <simon@2ndquadrant.com>:
>> I remember I had some differences between the way loading occurs at
>> session start and as a result of a LOAD command. I think there's
>> probably already a way of doing this - probably by checking for
>> something that would only be there *after* having read reloadable
>> libraries but before main session starts.
> I can test debug_query_string, but main problem is impossibility throw
> exception inside PG_init.
If we can't support throwing an error there, I think we need to fix that.
There's no way a "precheck" function can completely guarantee that no
error will happen in the real "do it" function; at least not for
interesting values of "do it".
regards, tom lane