Thread: Perl 5.12 complains about ecpg parser-hacking scripts
On Fedora 14 I see these warnings while building ecpg's preproc.y:
Use of assignment to $[ is deprecated at ./parse.pl line 21.
Use of assignment to $[ is deprecated at ./check_rules.pl line 18.
Any Perl experts want to make those go away?
regards, tom lane
On 01/22/2011 06:06 PM, Tom Lane wrote: > On Fedora 14 I see these warnings while building ecpg's preproc.y: > > Use of assignment to $[ is deprecated at ./parse.pl line 21. > Use of assignment to $[ is deprecated at ./check_rules.pl line 18. > > Any Perl experts want to make those go away? > > I think these really need to be rewritten from scratch. They look like they were written by someone who never heard of Perl 5 (it's only about 16 years old). We could probably silence the warnings by putting: use warnings; no warnings qw(deprecate); at the top of the file, but that would be rather like putting a bandaid on a massive chest wound. cheers andrew
Andrew Dunstan <andrew@dunslane.net> writes:
> On 01/22/2011 06:06 PM, Tom Lane wrote:
>> On Fedora 14 I see these warnings while building ecpg's preproc.y:
>>
>> Use of assignment to $[ is deprecated at ./parse.pl line 21.
>> Use of assignment to $[ is deprecated at ./check_rules.pl line 18.
>>
>> Any Perl experts want to make those go away?
> I think these really need to be rewritten from scratch.
I noticed they didn't "use strict" either ... have at it.
> We could probably silence the warnings by putting:
> use warnings; no warnings qw(deprecate);
Doesn't seem like a step forward.
regards, tom lane
On 01/22/2011 08:48 PM, Tom Lane wrote: > Andrew Dunstan<andrew@dunslane.net> writes: >> On 01/22/2011 06:06 PM, Tom Lane wrote: >>> On Fedora 14 I see these warnings while building ecpg's preproc.y: >>> >>> Use of assignment to $[ is deprecated at ./parse.pl line 21. >>> Use of assignment to $[ is deprecated at ./check_rules.pl line 18. >>> >>> Any Perl experts want to make those go away? >> I think these really need to be rewritten from scratch. > I noticed they didn't "use strict" either ... have at it. > If I do it's unlikely to be any time soon. I wasn't putting my hand up for the job, just pointing out that fixing these would barely be scratching the surface of what's needed. But there are quite a few perlheads around. ISTR Matt Trout was muttering about these scripts on IRC recently. cheers andrew
On 23 January 2011 13:14, Andrew Dunstan <andrew@dunslane.net> wrote: > But there are quite a few perlheads around. ISTR Matt Trout was muttering > about these scripts on IRC recently. A quick cleanup of the check_rules.pl... It's a starting point at least.
Attachment
On Sat, Jan 22, 2011 at 08:40:13PM -0500, Andrew Dunstan wrote: > I think these really need to be rewritten from scratch. They look > like they were written by someone who never heard of Perl 5 (it's > only about 16 years old). You might remember that we had this discussion before. The script was written in awk and then translated to perl by a2p. We knew that this code was less than optimal, but at least it works. As I already said when the script was introduced, I would love to have a real perl solution, but I'm not a perl programmer by any means. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at googlemail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
On 01/22/2011 09:28 PM, kris@shannon.id.au wrote: > On 23 January 2011 13:14, Andrew Dunstan<andrew@dunslane.net> wrote: >> But there are quite a few perlheads around. ISTR Matt Trout was muttering >> about these scripts on IRC recently. > > A quick cleanup of the check_rules.pl... > > It's a starting point at least. > > > > Oh! Perl is my favorite. Kris, if you're not going to, I'd love to work on this. -Andy
On 01/23/2011 06:11 AM, Michael Meskes wrote: > On Sat, Jan 22, 2011 at 08:40:13PM -0500, Andrew Dunstan wrote: >> I think these really need to be rewritten from scratch. They look >> like they were written by someone who never heard of Perl 5 (it's >> only about 16 years old). > You might remember that we had this discussion before. The script was written > in awk and then translated to perl by a2p. We knew that this code was less than > optimal, but at least it works. Oh, I'm sorry, I'd totally forgotten that. At least it explains why it looks like it's from a time warp. I bet the perl guys haven't touched a2p for many many years. cheers andrew
On 01/23/2011 08:29 AM, Andy Colson wrote: > On 01/22/2011 09:28 PM, kris@shannon.id.au wrote: >> On 23 January 2011 13:14, Andrew Dunstan<andrew@dunslane.net> wrote: >>> But there are quite a few perlheads around. ISTR Matt Trout was muttering >>> about these scripts on IRC recently. Ok, so I've figured out what its purpose is. Are there other problems with this script? Does it not pull out the rule names correct all the time or something? Whatproblem was Matt having with it? I think rewriting from scratch is overkill, unless this script is just failing. The code to pull the rule names out is abit complex, and it seems to work, so I'd rather not touch it. Are there other things you wished this script did? (reports, counts, etc) -Andy
On 01/23/2011 10:16 AM, Andy Colson wrote: > On 01/23/2011 08:29 AM, Andy Colson wrote: >> On 01/22/2011 09:28 PM, kris@shannon.id.au wrote: >>> On 23 January 2011 13:14, Andrew Dunstan<andrew@dunslane.net> wrote: >>>> But there are quite a few perlheads around. ISTR Matt Trout was >>>> muttering >>>> about these scripts on IRC recently. > > Ok, so I've figured out what its purpose is. > > Are there other problems with this script? Does it not pull out the > rule names correct all the time or something? What problem was Matt > having with it? > > I think rewriting from scratch is overkill, unless this script is just > failing. The code to pull the rule names out is a bit complex, and it > seems to work, so I'd rather not touch it. > > Are there other things you wished this script did? (reports, counts, > etc) > > It's doing the right thing. But it's really spaghetti code, generated by a2p. Matt was just (rightly) offended by the $[ setting, IIRC. The point is that it's close to being totally unmaintainable. But if you think you can remedy it without rewriting it, go for it. I think minimum requirements would be: * avoid setting $[ * use strict; * comments saying what it's actually doing We want to be in a situation where of it ever does break because of some external change, we're not left having to wade through the crap to find out how to fix it. cheers andrew
On Sun, Jan 23, 2011 at 09:16:33AM -0600, Andy Colson wrote: > Are there other problems with this script? Does it not pull out the rule names correct all the time or something? Whatproblem was Matt having with it? No, afaict it works correctly but throws some warnings. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at googlemail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
On 01/23/2011 10:06 AM, Andrew Dunstan wrote: > > > On 01/23/2011 10:16 AM, Andy Colson wrote: >> On 01/23/2011 08:29 AM, Andy Colson wrote: >>> On 01/22/2011 09:28 PM, kris@shannon.id.au wrote: >>>> On 23 January 2011 13:14, Andrew Dunstan<andrew@dunslane.net> wrote: >>>>> But there are quite a few perlheads around. ISTR Matt Trout was muttering >>>>> about these scripts on IRC recently. >> >> Ok, so I've figured out what its purpose is. >> >> Are there other problems with this script? Does it not pull out the rule names correct all the time or something? Whatproblem was Matt having with it? >> >> I think rewriting from scratch is overkill, unless this script is just failing. The code to pull the rule names out isa bit complex, and it seems to work, so I'd rather not touch it. >> >> Are there other things you wished this script did? (reports, counts, etc) >> >> > > > It's doing the right thing. But it's really spaghetti code, generated by a2p. Matt was just (rightly) offended by the $[setting, IIRC. > > The point is that it's close to being totally unmaintainable. > > But if you think you can remedy it without rewriting it, go for it. > > I think minimum requirements would be: > > * avoid setting $[ > * use strict; > * comments saying what it's actually doing > > > We want to be in a situation where of it ever does break because of some external change, we're not left having to wadethrough the crap to find out how to fix it. > > cheers > > andrew ** OOPS, forenote: I only noticed check_rules.pl. That's the only file I looked at. I'll go look at parse.pl right now. Below refers to check_rules.pl only. I can comment the code. $[ can safely be removed (from check_rules.pl) because we are not even using arrays. I disagree (check_rules.pl) is not spaghetti code. There are two loops, one thru gram.y to pick out the rule names, andone through ecpg.addons to make sure they are in use. Unmaintainable is another story. Parsing gram.y for rule names, then combining them to make up the same name in ecpg.addonsis a little complex. But then I think it would be a little complex in any language. A little change to the syntax/layoutof either file and it could go bad, but you'd have that same problem with any other language too. Is there anyway to make bison/yacc/gcc/etc spit out the rule names? I'm not sure rule name is the proper words. In gram.y we have: stmt : AlterDatabaseStmt | AlterDatabaseSetStmt We pull out and create: stmtAlterDatabaseStmt and stmtAlterDatabaseSetStmt -Andy
Andy Colson <andy@squeakycode.net> writes:
> Is there anyway to make bison/yacc/gcc/etc spit out the rule names?
bison -v produces a debug output file that includes nicely cleaned-up
versions of all the rules. But it includes a lot of other stuff too,
and I'm not at all sure that the file format is stable across bison
versions, so maybe depending on that isn't a hot idea.
> I'm not sure rule name is the proper words. In gram.y we have:
"Production" is the standard technical name, but on the other hand the
bison documentation seems to consistently use the word "rule", so
there's probably nothing wrong with doing so here too.
regards, tom lane
Kris Shannon <kris@shannon.id.au> writes:
> What is the minimal perl version that we are requiring these days?
5.8 according to configure.
regards, tom lane
On 24 January 2011 03:06, Andrew Dunstan <andrew@dunslane.net> wrote:
> It's doing the right thing. But it's really spaghetti code, generated by
> a2p. Matt was just (rightly) offended by the $[ setting, IIRC.
>
> The point is that it's close to being totally unmaintainable.
>
> But if you think you can remedy it without rewriting it, go for it.
>
> I think minimum requirements would be:
>
> * avoid setting $[
> * use strict;
> * comments saying what it's actually doing
>
>
> We want to be in a situation where of it ever does break because of some
> external change, we're not left having to wade through the crap to find out
> how to fix it.
>
> cheers
>
> andrew
>
Well here's an absolutely minimal patch to remove the $[ assignments
and fix up the array accesses.
I have tested that parse.pl produces identical output for both HEAD
and the 65 commits which touch ecpg.*
or gram.y since the introduction on check_rules.pl
git log 81a82a13^..HEAD -- src/backend/parser/gram.y
src/interfaces/ecpg/preproc/{ecpg.??*,*.pl}
I've also tested that the new check_rules.pl dies when I add an extra
unused rule to ecpg.addons.
I'll try and make a proper rewrite sometime this week.
What is the minimal perl version that we are requiring these days?
Attachment
On 23 January 2011 14:28, Kris Shannon <kris@shannon.id.au> wrote: > On 23 January 2011 13:14, Andrew Dunstan <andrew@dunslane.net> wrote: >> But there are quite a few perlheads around. ISTR Matt Trout was muttering >> about these scripts on IRC recently. > > A quick cleanup of the check_rules.pl... > > It's a starting point at least. > And I should have at least tried to run this before I submitted... :( New minimal tested submission momentarily.
On sön, 2011-01-23 at 12:23 -0500, Tom Lane wrote: > Andy Colson <andy@squeakycode.net> writes: > > Is there anyway to make bison/yacc/gcc/etc spit out the rule names? > > bison -v produces a debug output file that includes nicely cleaned-up > versions of all the rules. But it includes a lot of other stuff too, > and I'm not at all sure that the file format is stable across bison > versions, so maybe depending on that isn't a hot idea. Having maintained a gram.y -> keywords.sgml script based on that idea for N years, I can report that this is not stable enough to work here.
On 1/23/2011 5:11 AM, Michael Meskes wrote: > On Sat, Jan 22, 2011 at 08:40:13PM -0500, Andrew Dunstan wrote: >> I think these really need to be rewritten from scratch. They look >> like they were written by someone who never heard of Perl 5 (it's >> only about 16 years old). > > You might remember that we had this discussion before. The script was written > in awk and then translated to perl by a2p. We knew that this code was less than > optimal, but at least it works. > > As I already said when the script was introduced, I would love to have a real > perl solution, but I'm not a perl programmer by any means. > > Michael I thought Kris was going to work on this, but saw no progress, and I was bored the other day, so I started working on it. Here is a parse.pl, with some major refactoring. I named it with a 2 so I could run it beside the original and diff em: time perl parse.pl . < ../../../backend/parser/gram.y > preproc.y real 0m10.636s user 0m8.793s sys 0m0.050s time perl parse2.pl . < ../../../backend/parser/gram.y > preproc2.y real 0m0.289s user 0m0.214s sys 0m0.009s diff preproc.y preproc2.y I am sure there are new bugs. I have not run it on anything but 9.0.1. Are there other .y files you might feed it? (something other than backend/parser/gram.y?) I touched pretty much everything, and I broke a few things and had to go back and fix 'em, so it would not surprise me at all if there were still a few bugs. Anyway, feedback, suggestions, etc are welcomed. (oh, yeah, I probably need to go add some documentation) -Andy
Attachment
Andy Colson <andy@squeakycode.net> writes:
> Here is a parse.pl, with some major refactoring.
> I am sure there are new bugs. I have not run it on anything but 9.0.1.
> Are there other .y files you might feed it? (something other than
> backend/parser/gram.y?)
That's the only file it has to work for. You could try it against 8.4
and HEAD versions as well as 9.0, but I'm not sure how much additional
test coverage that will get you.
regards, tom lane
On Wed, Mar 02, 2011 at 01:33:35PM -0600, Andy Colson wrote: > I thought Kris was going to work on this, but saw no progress, and I > was bored the other day, so I started working on it. > > Here is a parse.pl, with some major refactoring. > > I named it with a 2 so I could run it beside the original and diff em: Thanks for all the work. > I am sure there are new bugs. I have not run it on anything but > 9.0.1. Are there other .y files you might feed it? (something other > than backend/parser/gram.y?) I ran it against several versions and it always gave the right output. So i decided to just commit it to the archive so we can see if it breaks anything. The old script is still in there so in case of a major problem that I cannot foresee we can simply change the Makefile back to using parse.pl. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at googlemail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
On 3/3/2011 6:49 AM, Michael Meskes wrote: > On Wed, Mar 02, 2011 at 01:33:35PM -0600, Andy Colson wrote: >> I thought Kris was going to work on this, but saw no progress, and I >> was bored the other day, so I started working on it. >> >> Here is a parse.pl, with some major refactoring. >> >> I named it with a 2 so I could run it beside the original and diff em: > > Thanks for all the work. > >> I am sure there are new bugs. I have not run it on anything but >> 9.0.1. Are there other .y files you might feed it? (something other >> than backend/parser/gram.y?) > > I ran it against several versions and it always gave the right output. So i > decided to just commit it to the archive so we can see if it breaks anything. > The old script is still in there so in case of a major problem that I cannot > foresee we can simply change the Makefile back to using parse.pl. > > Michael > And here is check_rules. Much less change, but I did run it through perl tidy, so I'll bet most of the lines changed. -Andy
Attachment
On 3 March 2011 06:33, Andy Colson <andy@squeakycode.net> wrote: > On 1/23/2011 5:11 AM, Michael Meskes wrote: >> As I already said when the script was introduced, I would love to have a >> real >> perl solution, but I'm not a perl programmer by any means. >> >> Michael > > I thought Kris was going to work on this, but saw no progress, and I was > bored the other day, so I started working on it. Yeah, sorry about that. I got as far as check_rules.pl with perl5.8 and running it against all the commits which changed the files it depends on and comparing the results with the old script; and then a major meltdown at work - primary SAN crashed :( - got in the way and I totally forgot about it... Thanks for picking up the ball and running with it.