You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by 孙 岚 <re...@hotmail.com> on 2016/09/03 08:27:35 UTC
[CONTRIBUTION]groovy-parser
Hi List,
The groovy-antlr4-grammar project is very hard to maintain and the performance is terrible, so I am rewriting the parser(https://github.com/danielsun1106/groovy-parser), which is based on the Java grammar provided by the antlr offical and introduced some good idea from the old grammar. The Java parser generated by antlr4 is proved efficient in the most of senarios, so Groovy parser will be efficient too if monitoring the performance from the beginning.
The groovy-parser project is now stand-alone in order to develop rapidly. When the project is mature enough, I will make a PR to replace groovy-antlr4-grammar. BTW, groovy-parser requires Java 8.
Currently groovy-parser supports all statements, most of expressions(except method call expression, command expression, etc.). I wish the groovy-parser could be done before the end of this year(2016).
Any suggestion and help is appreciated.
Cheers,
Daniel.Sun
Re: [CONTRIBUTION]groovy-parser
Posted by Jochen Theodorou <bl...@gmx.org>.
On 30.09.2016 10:03, daniel_sun wrote:
> Hi Sergei,
>
> I tried to add the statement to expression as an alternative, the
> antlr4 failed to generate sources becuase of the error "error(119):
> GroovyParser.g4::: The following sets of rules are mutually left-recursive
> [statement, expression, statementExpression]".
you cannot add statement as an alternative to expression if a expression
is also a statement. If any statement should be also an expression, then
statement and expression have to be merged
> Currenty the expression is quite huge, which is a potential cause of
> performance issue. so if we try our best to treat statement as expression in
> the future, the performance will be impacted somehow.
probably, yes
bye Jochen
Re: [CONTRIBUTION]groovy-parser
Posted by daniel_sun <re...@hotmail.com>.
Hi Sergei,
I tried to add the statement to expression as an alternative, the
antlr4 failed to generate sources becuase of the error "error(119):
GroovyParser.g4::: The following sets of rules are mutually left-recursive
[statement, expression, statementExpression]".
Currenty the expression is quite huge, which is a potential cause of
performance issue. so if we try our best to treat statement as expression in
the future, the performance will be impacted somehow.
Cheers,
Daniel.Sun
--
View this message in context: http://groovy.329449.n5.nabble.com/CONTRIBUTION-groovy-parser-tp5735184p5735699.html
Sent from the Groovy Dev mailing list archive at Nabble.com.
Re: [CONTRIBUTION]groovy-parser
Posted by Sergei Egorov <bs...@gmail.com>.
Hi all,
just curious - will it be possible to do "any statement is an expression"
with the new grammar?
That would be really awesome, especially for macro methods
BR,
Sergei
On Fri, Sep 30, 2016 at 10:31 AM daniel_sun <re...@hotmail.com> wrote:
> Hi Jochen,
>
> The groovy grammar bases on Java7
> grammar(https://github.com/antlr/grammars-v4/blob/master/java/Java.g4) at
> the beginning because I noticed the comment of Java8 grammar, which tells
> us
> Java7 has better performance.
>
> In addition, I refine the cache management of antlr4, so the memory
> will not grow forever and use the cache as far as it can. I find that about
> 20% time saved after the work done.
>
> Cheers,
> Daniel.Sun
>
>
>
> --
> View this message in context:
> http://groovy.329449.n5.nabble.com/CONTRIBUTION-groovy-parser-tp5735184p5735695.html
> Sent from the Groovy Dev mailing list archive at Nabble.com.
>
Re: [CONTRIBUTION]groovy-parser
Posted by daniel_sun <re...@hotmail.com>.
Hi Jochen,
The groovy grammar bases on Java7
grammar(https://github.com/antlr/grammars-v4/blob/master/java/Java.g4) at
the beginning because I noticed the comment of Java8 grammar, which tells us
Java7 has better performance.
In addition, I refine the cache management of antlr4, so the memory
will not grow forever and use the cache as far as it can. I find that about
20% time saved after the work done.
Cheers,
Daniel.Sun
--
View this message in context: http://groovy.329449.n5.nabble.com/CONTRIBUTION-groovy-parser-tp5735184p5735695.html
Sent from the Groovy Dev mailing list archive at Nabble.com.
Re: [CONTRIBUTION]groovy-parser
Posted by Jochen Theodorou <bl...@gmx.org>.
something maybe of interest from the javaparser gitter channel today:
ptitjes
Some numbers that are not on the bar graphs: Parsing JavaParser 2.5.1
with ANTLR4's Java 1.7 grammar takes **~115ms** and with ANTLR4's Java
1.8 grammar takes **15s** !! I had a discussion with Terence Parr, and
he told me the 1.7 grammar is highly optimized but nobody has optimized
the 1.8 one yet. Nevertheless, I find that 115ms is a lot, especially
considering it just spits out a parse tree and not the AST directly...
So if you base your grammar on the 1.8 one and the above is right, then
I am not sure you will accomplish better performance.
On 03.09.2016 10:27, wrote:
> Hi List,
>
>
> The groovy-antlr4-grammar project is very hard to maintain and
> the performance is terrible, so I am rewriting the
> parser(https://github.com/danielsun1106/groovy-parser), which is based
> on the Java grammar provided by the antlr offical and introduced some
> good idea from the old grammar. The Java parser generated by antlr4 is
> proved efficient in the most of senarios, so Groovy parser will be
> efficient too if monitoring the performance from the beginning.
>
>
> The groovy-parser project is now stand-alone in order to develop
> rapidly. When the project is mature enough, I will make a PR to replace
> groovy-antlr4-grammar. BTW, groovy-parser requires Java 8.
>
>
> Currently groovy-parser supports all statements, most of
> expressions(except method call expression, command expression, etc.). I
> wish the groovy-parser could be done before the end of this year(2016).
>
>
> Any suggestion and help is appreciated.
>
>
> Cheers,
>
> Daniel.Sun
>
Re: [CONTRIBUTION]groovy-parser
Posted by daniel_sun <re...@hotmail.com>.
Hi Guillaume,
The groovy parser is almost completed. I am parsing groovy files of
groovy project and comparing the old and new parser via the AST result. Some
bugs needs to fix. I wish the brand new parser can execute all test cases of
groovy project before the end of this year.
The fail directory contains all test cases *should* fail, and the
suggestion "2) stuff we would like to get parsed successfully" can be
treated as TODO list and commented in the *should* succeed test cases.
After all test cases pass, more features will be added, such as
friendly messages, lambda expression, do-while, indentical operation(===),
etc.
Cheers,
Daniel.Sun
--
View this message in context: http://groovy.329449.n5.nabble.com/CONTRIBUTION-groovy-parser-tp5735184p5735696.html
Sent from the Groovy Dev mailing list archive at Nabble.com.
Re: [CONTRIBUTION]groovy-parser
Posted by Guillaume Laforge <gl...@gmail.com>.
Indeed great to have a little update on the progress!
How is it possible to help? For example giving some new examples of code
and see how they are parsed (if parsed successfully or not, etc)?
Is the "fail" folder where we can contribute things that are not yet parsed
but should work ultimately, or is this for things we expect to to fail
parsing?
I was thinking that perhaps we need three distinct sets: 1) stuff we expect
to be parsed successfully, 2) stuff we would like to get parsed
successfully, and 3) stuff we don't want to be parsed, ie. that fail
parsing with proper error reporting?
Guillaume
On Tue, Sep 6, 2016 at 11:12 AM, Graeme Rocher <gr...@gmail.com>
wrote:
> Interesting project Daniel Sun, good luck with it and look forward to
> following your progress.
>
> Cheers
>
> On Sat, Sep 3, 2016 at 10:27 AM, 孙 岚 <re...@hotmail.com> wrote:
> > Hi List,
> >
> >
> > The groovy-antlr4-grammar project is very hard to maintain and the
> > performance is terrible, so I am rewriting the
> > parser(https://github.com/danielsun1106/groovy-parser), which is based
> on
> > the Java grammar provided by the antlr offical and introduced some good
> idea
> > from the old grammar. The Java parser generated by antlr4 is proved
> > efficient in the most of senarios, so Groovy parser will be efficient
> too if
> > monitoring the performance from the beginning.
> >
> >
> > The groovy-parser project is now stand-alone in order to develop
> > rapidly. When the project is mature enough, I will make a PR to replace
> > groovy-antlr4-grammar. BTW, groovy-parser requires Java 8.
> >
> >
> > Currently groovy-parser supports all statements, most of
> > expressions(except method call expression, command expression, etc.). I
> wish
> > the groovy-parser could be done before the end of this year(2016).
> >
> >
> > Any suggestion and help is appreciated.
> >
> >
> > Cheers,
> >
> > Daniel.Sun
>
>
>
> --
> Graeme Rocher
>
--
Guillaume Laforge
Apache Groovy committer & PMC Vice-President
Developer Advocate @ Google Cloud Platform
Blog: http://glaforge.appspot.com/
Social: @glaforge <http://twitter.com/glaforge> / Google+
<https://plus.google.com/u/0/114130972232398734985/posts>
Re: [CONTRIBUTION]groovy-parser
Posted by daniel_sun <re...@hotmail.com>.
Hi Graeme,
Thanks for your staring the project :)
Cheers,
Daniel.Sun
--
View this message in context: http://groovy.329449.n5.nabble.com/CONTRIBUTION-groovy-parser-tp5735184p5735698.html
Sent from the Groovy Dev mailing list archive at Nabble.com.
Re: [CONTRIBUTION]groovy-parser
Posted by Graeme Rocher <gr...@gmail.com>.
Interesting project Daniel Sun, good luck with it and look forward to
following your progress.
Cheers
On Sat, Sep 3, 2016 at 10:27 AM, 孙 岚 <re...@hotmail.com> wrote:
> Hi List,
>
>
> The groovy-antlr4-grammar project is very hard to maintain and the
> performance is terrible, so I am rewriting the
> parser(https://github.com/danielsun1106/groovy-parser), which is based on
> the Java grammar provided by the antlr offical and introduced some good idea
> from the old grammar. The Java parser generated by antlr4 is proved
> efficient in the most of senarios, so Groovy parser will be efficient too if
> monitoring the performance from the beginning.
>
>
> The groovy-parser project is now stand-alone in order to develop
> rapidly. When the project is mature enough, I will make a PR to replace
> groovy-antlr4-grammar. BTW, groovy-parser requires Java 8.
>
>
> Currently groovy-parser supports all statements, most of
> expressions(except method call expression, command expression, etc.). I wish
> the groovy-parser could be done before the end of this year(2016).
>
>
> Any suggestion and help is appreciated.
>
>
> Cheers,
>
> Daniel.Sun
--
Graeme Rocher