Thread: keyword list/ecpg
Hi, I recently (on my flight to Ottawa) changed ecpg to use the keyword list of the backend instead of its own. This means that there is one less file to sync manually. However, it also means that an additional keyword defined in the backend will break compilation as ecpg doesn't have this definition in its parser. Does anyone object to moving the symbol definition part to a file of its own that's inluded by both parser? Or does anyone have a better idea? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
Michael Meskes <meskes@postgresql.org> writes:
> I recently (on my flight to Ottawa) changed ecpg to use the keyword list
> of the backend instead of its own. This means that there is one less
> file to sync manually. However, it also means that an additional keyword
> defined in the backend will break compilation as ecpg doesn't have this
> definition in its parser.
> Does anyone object to moving the symbol definition part to a file of its
> own that's inluded by both parser? Or does anyone have a better idea?
AFAIK bison doesn't have an "include" capability, so I'm not sure how
you plan to make that work?
regards, tom lane
On Mon, May 26, 2008 at 04:15:18PM -0400, Tom Lane wrote: > > Does anyone object to moving the symbol definition part to a file of its > > own that's inluded by both parser? Or does anyone have a better idea? > > AFAIK bison doesn't have an "include" capability, so I'm not sure how > you plan to make that work? You found the weak part in my idea. :-) How about a small perl script that is run before bison and creates the real grammar file? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
On Tue, 2008-05-27 at 10:03 +0200, Michael Meskes wrote: > On Mon, May 26, 2008 at 04:15:18PM -0400, Tom Lane wrote: > > > Does anyone object to moving the symbol definition part to a file of its > > > own that's inluded by both parser? Or does anyone have a better idea? > > > > AFAIK bison doesn't have an "include" capability, so I'm not sure how > > you plan to make that work? > > You found the weak part in my idea. :-) > > How about a small perl script that is run before bison and creates the > real grammar file? man cpp ? ------------- Hannu
On Tue, May 27, 2008 at 11:23:28AM +0300, Hannu Krosing wrote: > > How about a small perl script that is run before bison and creates the > > real grammar file? > > man cpp ? As this would handle all includes and defines I'm unsure whether it is desirable. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
Michael Meskes <meskes@postgresql.org> writes:
> On Mon, May 26, 2008 at 04:15:18PM -0400, Tom Lane wrote:
>> AFAIK bison doesn't have an "include" capability, so I'm not sure how
>> you plan to make that work?
> You found the weak part in my idea. :-)
> How about a small perl script that is run before bison and creates the
> real grammar file?
If we were going to do that, I'd want it to go all the way and somehow
generate the common parts of the two .y files from a single source.
That'd be enough of a step forward that it would be worth whatever
ugliness is needed to make it happen ...
regards, tom lane
On Tue, May 27, 2008 at 09:44:02AM -0400, Tom Lane wrote:
> If we were going to do that, I'd want it to go all the way and somehow
> generate the common parts of the two .y files from a single source.
> That'd be enough of a step forward that it would be worth whatever
> ugliness is needed to make it happen ...
Any idea how to make this happen? We could of course do it with a lot of
ifdef like sequences:
rule {
IF ECPGCOMPILEecpg stuff
ELSEbackend stuff
ENDIF
}
Would you go into this direction?
Michael
--
Michael Meskes
Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
Michael Meskes <meskes@postgresql.org> writes:
> On Tue, May 27, 2008 at 09:44:02AM -0400, Tom Lane wrote:
>> If we were going to do that, I'd want it to go all the way and somehow
>> generate the common parts of the two .y files from a single source.
> Any idea how to make this happen? We could of course do it with a lot of
> ifdef like sequences:
> rule {
> IF ECPGCOMPILE
> ecpg stuff
> ELSE
> backend stuff
> ENDIF
> }
> Would you go into this direction?
Ugh :-(.
I have not spent much time looking at the ecpg grammar, so feel free to
laugh this off, but I had the impression that all the rules derived from
the backend grammar have boilerplate action sections (ie, just join the
strings together). So I was hoping that we could leave the backend's
.y file more or less as-is, and write a perl script that would go
through it and replace each { ... } action with a suitable cat_str call,
which it could build on-the-fly by counting the number of rule tokens.
Then combine that output with the ecpg-specific rules taken from a
separate source file. Obviously there would have to be a few small
kluges, but an #if-like mechanism might suffice for those. An #if
in every rule is way past my tolerance for ugliness though, especially
when one arm of it is essentially mechanically-generatable code.
regards, tom lane
On Wed, Jun 04, 2008 at 10:21:19AM -0400, Tom Lane wrote:
> Ugh :-(.
This is why I didn't want to go that route. :-)
> I have not spent much time looking at the ecpg grammar, so feel free to
> laugh this off, but I had the impression that all the rules derived from
> the backend grammar have boilerplate action sections (ie, just join the
This is true.
> strings together). So I was hoping that we could leave the backend's
> .y file more or less as-is, and write a perl script that would go
> through it and replace each { ... } action with a suitable cat_str call,
> which it could build on-the-fly by counting the number of rule tokens.
There is some small magic to know when to have blanks in between and
when not, but that should be doable.
> Then combine that output with the ecpg-specific rules taken from a
> separate source file. Obviously there would have to be a few small
This might work. Anyone with good perl knowledge interested?
Michael
--
Michael Meskes
Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org
Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
I wouldn't mind having a stab at this if you can expand on the 'magic'
required.
(I'm interested because I might be able to use the same logic to roll a third
version of the .y for Aubit4GL outside of the Postgresql tree)
On Wednesday 04 June 2008 16:11:49 Michael Meskes wrote:
> On Wed, Jun 04, 2008 at 10:21:19AM -0400, Tom Lane wrote:
> > Ugh :-(.
>
> This is why I didn't want to go that route. :-)
>
> > I have not spent much time looking at the ecpg grammar, so feel free to
> > laugh this off, but I had the impression that all the rules derived from
> > the backend grammar have boilerplate action sections (ie, just join the
>
> This is true.
>
> > strings together). So I was hoping that we could leave the backend's
> > .y file more or less as-is, and write a perl script that would go
> > through it and replace each { ... } action with a suitable cat_str call,
> > which it could build on-the-fly by counting the number of rule tokens.
>
> There is some small magic to know when to have blanks in between and
> when not, but that should be doable.
>
> > Then combine that output with the ecpg-specific rules taken from a
> > separate source file. Obviously there would have to be a few small
>
> This might work. Anyone with good perl knowledge interested?
>
> Michael
>
> --
> Michael Meskes
> Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
> ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org
> Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
--
Mike Aubury
http://www.aubit.com/
Aubit Computing Ltd is registered in England and Wales, Number: 3112827
Registered Address : Clayton House,59 Piccadilly,Manchester,M1 2AQ
Mike Aubury <mike.aubury@aubit.com> writes:
> On Wednesday 04 June 2008 16:11:49 Michael Meskes wrote:
>> There is some small magic to know when to have blanks in between and
>> when not, but that should be doable.
> I wouldn't mind having a stab at this if you can expand on the 'magic'
> required.
Wouldn't it work to just always insert a space between tokens, no matter
whether there was one originally?
regards, tom lane
It might depend on the tokens.. Are ">=", "++" etc single tokens ? On Wednesday 04 June 2008 17:06:44 Tom Lane wrote: > Mike Aubury <mike.aubury@aubit.com> writes: > > On Wednesday 04 June 2008 16:11:49 Michael Meskes wrote: > >> There is some small magic to know when to have blanks in between and > >> when not, but that should be doable. > > > > I wouldn't mind having a stab at this if you can expand on the 'magic' > > required. > > Wouldn't it work to just always insert a space between tokens, no matter > whether there was one originally? > > regards, tom lane -- Mike Aubury http://www.aubit.com/ Aubit Computing Ltd is registered in England and Wales, Number: 3112827 Registered Address : Clayton House,59 Piccadilly,Manchester,M1 2AQ
[Sorry, just noticed that I didn't answer this email. ] On Wed, Jun 04, 2008 at 05:06:41PM +0100, Mike Aubury wrote: > It might depend on the tokens.. > Are ">=", "++" etc single tokens ? > ... > > Wouldn't it work to just always insert a space between tokens, no matter > > whether there was one originally? There are a few cases where you must not enter a blank, but I'm not sure whethere these are all in ecpg specific rules anyway. One example that comes to my mind is the handling of ":<port>" in the connect statement. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
I took a quick look at this - would it be ok to add some small amounts of 'markup' to the gram.y ? eg : /* ECPGCOPYON */ .... /* ECPGCOPYOFF */ /* ECPGMODE=NOSPACE */ ... /* ECPGMODE=USESPACE */ etc ? On Friday 13 June 2008 10:47:55 Michael Meskes wrote: > [Sorry, just noticed that I didn't answer this email. ] > > On Wed, Jun 04, 2008 at 05:06:41PM +0100, Mike Aubury wrote: > > It might depend on the tokens.. > > Are ">=", "++" etc single tokens ? > > ... > > > > > Wouldn't it work to just always insert a space between tokens, no > > > matter whether there was one originally? > > There are a few cases where you must not enter a blank, but I'm not sure > whethere these are all in ecpg specific rules anyway. One example that > comes to my mind is the handling of ":<port>" in the connect statement. > > Michael > -- > Michael Meskes > Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) > ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org > Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL! -- Mike Aubury http://www.aubit.com/ Aubit Computing Ltd is registered in England and Wales, Number: 3112827 Registered Address : Clayton House,59 Piccadilly,Manchester,M1 2AQ
Mike Aubury <mike.aubury@aubit.com> writes:
> I took a quick look at this - would it be ok to add some small amounts
> of 'markup' to the gram.y ?
> eg :
> /* ECPGCOPYON */
> ....
> /* ECPGCOPYOFF */
> /* ECPGMODE=NOSPACE */
> ...
> /* ECPGMODE=USESPACE */
We're almost certainly going to need some kluges of that sort, so as
long as they're not all over the place I won't object.
But ... I've seen no evidence that those specific examples are needed.
Why wouldn't we copy all the backend rules? And based on Michael's last
comment it's unclear that we need to avoid adding spaces in the
mechanically generated actions, either (which squares with my gut
feeling about SQL syntax). You'll probably need to get into specific
cases before finding out what kluges you need.
regards, tom lane
> We're almost certainly going to need some kluges of that sort, so as > long as they're not all over the place I won't object. > > But ... I've seen no evidence that those specific examples are needed. > Why wouldn't we copy all the backend rules? And based on Michael's last > comment it's unclear that we need to avoid adding spaces in the > mechanically generated actions, either (which squares with my gut > feeling about SQL syntax). You'll probably need to get into specific > cases before finding out what kluges you need. I think this was more an 'in principle' - if thats route is ok, then I'll start hacking away properly... I was thinking about the copy on/copy off for more the header info (before the %%) - so we can have a really dumb script that just gets told what blocks to copy - and what to ignore.. There will also be some grammer in the original which we'll need to replace with some ecpg specifics - eg adding grammer for the variables etc. Might be easier to just turn 'off' the original rules and have some custom ecpg stuff appended to the generated code.. Theres also another thing that needs to be decided, which is if the generated ecpg grammer should be developer generated (ie. Michael Meskes runs a script and commits the output), or should be generated for each and every source based installation. I personally would stongly favour the script being a tool for ecpg tool developers and not used as part of a normal installation. -- Mike Aubury http://www.aubit.com/ Aubit Computing Ltd is registered in England and Wales, Number: 3112827 Registered Address : Clayton House,59 Piccadilly,Manchester,M1 2AQ
Mike Aubury wrote: > Theres also another thing that needs to be decided, which is if the generated > ecpg grammer should be developer generated (ie. Michael Meskes runs a script > and commits the output), or should be generated for each and every source > based installation. I personally would stongly favour the script being a tool > for ecpg tool developers and not used as part of a normal installation. What happens when a non-Michael developer changes the original gram.y? Is he expected to run the script before committing too? That sounds brittle to me. -- Alvaro Herrera http://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc.
Alvaro Herrera <alvherre@commandprompt.com> writes:
> Mike Aubury wrote:
>> Theres also another thing that needs to be decided, which is if the generated
>> ecpg grammer should be developer generated (ie. Michael Meskes runs a script
>> and commits the output), or should be generated for each and every source
>> based installation. I personally would stongly favour the script being a tool
>> for ecpg tool developers and not used as part of a normal installation.
> What happens when a non-Michael developer changes the original gram.y?
> Is he expected to run the script before committing too? That sounds
> brittle to me.
As long as the script is written in Perl and not exceedingly slow,
I see no reason it shouldn't be required to run as part of a build from
CVS. We already require Perl use elsewhere in the build if you're not
working from a tarball.
regards, tom lane
The same as happens at the moment - nothing... The grammer for the ecpg needs to be re-generated when the grammer in the main parser is changed - whether its a manual or (mostly) automatic task is largely irrelevant. The only downside is that if its not regenerated then the change to gram.y simply wont be reflected in the grammer for ecpg. I personally think its down to the ecpg developers (of which I believe Michael is the main developer) to decide when to do this and to check that its worked. Its just otherwise - there could be a serious case for 'unintended consequences'... Just my 2 pence worth... On Friday 13 June 2008 15:39:48 Alvaro Herrera wrote: > > I personally would stongly favour > > the script being a tool for ecpg tool developers and not used as part of > > a normal installation. > > What happens when a non-Michael developer changes the original gram.y? > Is he expected to run the script before committing too? That sounds > brittle to me. > -- Mike Aubury http://www.aubit.com/ Aubit Computing Ltd is registered in England and Wales, Number: 3112827 Registered Address : Clayton House,59 Piccadilly,Manchester,M1 2AQ
Mike Aubury <mike.aubury@aubit.com> writes:
> I was thinking about the copy on/copy off for more the header info (before
> the %%) - so we can have a really dumb script that just gets told what blocks
> to copy - and what to ignore..
I think you'll find that doesn't work very well; the script will need at
least some understanding of the bison %-declarations, and it can just
ignore everything else before %%. The reason is that (1) you're going
to need to alter the %type declarations and (2) you're going to need to
merge these declarations with ones from ecpg.
It might be that the merging is best handled via directive-like
comments, but I suspect they'll be in the ecpg-side file not in the
main grammar: you'll have something indicating "insert all the main
%type declarations here", one for "insert precedence declarations
here", etc.
But of course this is just speculation...
regards, tom lane
Mike Aubury <mike.aubury@aubit.com> writes:
>> What happens when a non-Michael developer changes the original gram.y?
> The same as happens at the moment - nothing...
Wrong answer. The entire point of this work is to get rid of manual
work in updating the ecpg grammar when the main changes. The above
seems about like saying that the build process shouldn't regenerate
gram.c from gram.y --- you're supposed to do that manually.
One of the things I want to have come out of this is that we find out
right away if a main-grammar change breaks ecpg. Right now, any
conflict is not discovered until Michael gets around to syncing the
files, which is often weeks or months later.
regards, tom lane
On Fri, Jun 13, 2008 at 02:57:54PM +0100, Mike Aubury wrote: > based installation. I personally would stongly favour the script being a tool > for ecpg tool developers and not used as part of a normal installation. Why? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!
On Fri, Jun 13, 2008 at 10:39:48AM -0400, Alvaro Herrera wrote: > What happens when a non-Michael developer changes the original gram.y? > Is he expected to run the script before committing too? That sounds > brittle to me. The situation used to be that this only caused ecpg to be out-of-sync. No big deal. But nowadays it might break compilation. If you add a new keyword to keywords.c but not to preproc.y you'll get an undefined symbol during build process. To make sure stuff like this cannot happen in the future I'd prefer to have the script run automatically, albeit having a defined, but maybe non-functional, default. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: meskes@jabber.org Go VfL Borussia! Go SF 49ers! Use Debian GNU/Linux! Use PostgreSQL!