You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@groovy.apache.org by Jesper Steen Møller <je...@selskabet.org> on 2016/03/07 07:48:04 UTC
Progress on the Antlr4-based parser
Hi again Groovy-Dev
Here’s another update on the progress on the Antlr4 parser. I’m still exploring the limitations of the parser, partly by inspection and comparison with the Groovy documentation, and partly by experimentation (letting it loose on all the *.groovy files in the repository)
Since last update, I’ve fixed a number of issues:
assert statement (with either colon or comma as separator)
Added cast operator
super() or this()-call in constructors
Support identifiers as property names and map literal entry names
Incorporated Pavel’s ASTBuilder as Java. And un-machinetranslated in where I could, so it’s more idiomatic Java-ish.
Multiple declarations are now handled in locals and fields/properties.
Trailing commas are OK in lists and maps
Scripts may contain methods now
Updated Antlr4 version and made it configurable with the others in main build.gradle
Fixed the classname-detection login (Now I will never, ever forget binarySearch’s return value!
Allow F and D as suffix to integer literals
Allow static import and imports with aliases
Still outstanding:
Full Unicode letter support for identifiers
Proper unescaping of string literals
The grammar is too limited around call expressions trailing a non-path expression
Some combinations of parameterless calls and closure-only calls don’t work yet
Error messages look a lot different from the old parser - may need some qork
Haven’t looked at error recovery -(single token correction built into Antlr4 does some of the work)
Some newline handling around various operators need attention
So you can help me with a couple of answers:
Is it a goal to close the gap to Java’s more trickier syntax, such as placement-new and placement constructor calls? From what I can tell, we’d have to extend the AST too to support that. Has anyone ever used that except in compiler test cases?
./benchmark/bench/heapsort.groovy uses access modfiers on the script’s local variables — that’s not really allowed, is it? How should that work? It can’t get it to work in Groovy 2.4.x
As before, to play with it, try:
$ git clone -b antlr4 https://github.com/jespersm/groovy.git <https://github.com/jespersm/groovy.git>
$ cd groovy
$ gradle -PuseAntlr4=true console
Have fun.
-Jesper
Re: Progress on the Antlr4-based parser
Posted by Jochen Theodorou <bl...@gmx.org>.
On 07.03.2016 14:16, Jesper Steen Møller wrote:
[...]
> If you make a static inner subclass of the non-static inner class, the
> constructor must specify the outer instance.
of course...! Haven't thought about that one... I guess we miss that in
Groovy then... not sure though we need it, or if it is really good
design to do that... probably a no to both.
>>>>> * ./benchmark/bench/heapsort.groovy uses access modfiers on the
>>>>> script’s local variables — that’s not really allowed, is it? How
>>>>> should that work? It can’t get it to work in Groovy 2.4.x
>>>>
>>>> You mean like "public static final long IM = 139968"? It does not
>>>> really make a semantic sense to allow this. For convenient copy&paste
>>>> this could be allowed... if it poses no problem, I think it would be
>>>> nice to have. But it is not really required
>>>>
>>> But what should it mean? Should it become fields in the Script class, or
>>> just be stripped of access and staticness and introduced as locals in
>>> the main() ?
>>
>> yes
>>
>
> So - “Yes” to which side of the disjunction? Member or local?
sorry... ;) local variables. That's what you get without the modifiers
and we have not decided to do anything different through a modifier here
- afaik.
bye Jochen
Re: Progress on the Antlr4-based parser
Posted by Jesper Steen Møller <je...@selskabet.org>.
> On 7. mar. 2016, at 13.52, Jochen Theodorou <bl...@gmx.org> wrote:
>
> hehe... outer.super() give me a real WTF moment here ;) I have no idea what it does... even fantasy fails miserably here. It is actually more important to keep the grammar maintainable, then to add every obscure feature Java has to offer.
It’s really not too strange, compared to e.g. lambda method escape analysis ;-)
Each (nonstatic) inner class instance has an implicit reference to its enclosing (outer) instance, which it will get automatically if produced by a method of the outer instance. However, if you construct an inner class from somewhere else, you need to specify which outer instance it belongs to. That’s why we have “placement new”.
If you make a static inner subclass of the non-static inner class, the constructor must specify the outer instance. That’s why we need the outer super placement syntax. In effect:
C(A outer) {
outer.super();
}
translates into something like:
C(A outer) {
B.<init>(outer);
}
But I agree: Enough already! :-)
>
>>>> * ./benchmark/bench/heapsort.groovy uses access modfiers on the
>>>> script’s local variables — that’s not really allowed, is it? How
>>>> should that work? It can’t get it to work in Groovy 2.4.x
>>>
>>> You mean like "public static final long IM = 139968"? It does not
>>> really make a semantic sense to allow this. For convenient copy&paste
>>> this could be allowed... if it poses no problem, I think it would be
>>> nice to have. But it is not really required
>>>
>> But what should it mean? Should it become fields in the Script class, or
>> just be stripped of access and staticness and introduced as locals in
>> the main() ?
>
> yes
>
So - “Yes” to which side of the disjunction? Member or local?
>
>> Since it doesn’t work in Groovy now, I don’t understand why it’s in the
>> repo at all…
>
> I think this used to work before... which probably is why. If I would find now out why we do not allow it anymore…. on the other hand this is a strong indicator of not needing it as well... so ditch them ;) But "final" you will need.
It’s not a big issue, it’s just a matter of deciding which one, and perhaps updating the docs, which seem to forbid this.
Final is covered.
-Jesper
Re: Progress on the Antlr4-based parser
Posted by Jochen Theodorou <bl...@gmx.org>.
On 07.03.2016 12:29, Jesper Steen Møller wrote:
[...]
> Just what you always needed, right? I only learned about it from working
> on the JDT compiler.
Hehe, yeah, I did too learn some strange constructs of Java, just by
trying to mimic what Java does no the syntax level
> Also - I’m also -1 on introducing these pretty much useless Java
> features — it’s not “gratuitous incompatibility”, but major
> complications without any use. I’ll keep a list, but stop asking…
hehe... outer.super() give me a real WTF moment here ;) I have no idea
what it does... even fantasy fails miserably here. It is actually more
important to keep the grammar maintainable, then to add every obscure
feature Java has to offer.
>>> * ./benchmark/bench/heapsort.groovy uses access modfiers on the
>>> script’s local variables — that’s not really allowed, is it? How
>>> should that work? It can’t get it to work in Groovy 2.4.x
>>
>> You mean like "public static final long IM = 139968"? It does not
>> really make a semantic sense to allow this. For convenient copy&paste
>> this could be allowed... if it poses no problem, I think it would be
>> nice to have. But it is not really required
>>
> But what should it mean? Should it become fields in the Script class, or
> just be stripped of access and staticness and introduced as locals in
> the main() ?
yes
> Since it doesn’t work in Groovy now, I don’t understand why it’s in the
> repo at all…
I think this used to work before... which probably is why. If I would
find now out why we do not allow it anymore.... on the other hand this
is a strong indicator of not needing it as well... so ditch them ;) But
"final" you will need.
bye Jochen
Re: Progress on the Antlr4-based parser
Posted by Jesper Steen Møller <je...@selskabet.org>.
Hi Blackdrag
On 7. mar. 2016, at 10.02, Jochen Theodorou <bl...@gmx.org> wrote:
>
> On 07.03.2016 07:48, Jesper Steen Møller wrote:
> [...]
>>
>> So you can help me with a couple of answers:
>>
>> * Is it a goal to close the gap to Java’s more trickier syntax, such
>> as placement-new and placement constructor calls? From what I can
>> tell, we’d have to extend the AST too to support that. Has anyone
>> ever used that except in compiler test cases?
>
> I had to actually look up what you mean... so you can give type arguments in a constructor call? I did actually not know this. I would not give this one any priority, unless it is easy to do… but then I would not really give it an AST implementation yet
It’s certainly not something which is used a lot. Yes, you can declare type parameters for constructors like you can for methods. This is not supported in Groovy for now - neither is calling one.
Actually, placement new and placement constructor calls are different from explicit type arguments.
Placement new is used to specify the ‘outer’-instance for inner classes, and . I really dont think anyone does that in practise - here is an example:
public class A {
A(String name) {
this.me = name;
}
String me;
public class B {
String who() {
return "I'm a B inside " + me;
}
}
public static class C extends B {
C(A outer) {
outer.super(); // This is “placement constructor”, could be fudged in the AST as function call until late in compilation
}
String who() {
return "I'm a C inheiriting from this guy: " + super.who();
}
}
public static void main(String[] args) {
A a1 = new A("John");
A a2 = new A("Paul");
B b = a1.new B(); // This is "placement new" - may require AST for groovy, certainly grammar
System.out.println(b.who());
C c = new C(a2);
System.out.println(c.who());
}
}
Just what you always needed, right? I only learned about it from working on the JDT compiler.
Also - I’m also -1 on introducing these pretty much useless Java features — it’s not “gratuitous incompatibility”, but major complications without any use. I’ll keep a list, but stop asking…
>
>> * ./benchmark/bench/heapsort.groovy uses access modfiers on the
>> script’s local variables — that’s not really allowed, is it? How
>> should that work? It can’t get it to work in Groovy 2.4.x
>
> You mean like "public static final long IM = 139968"? It does not really make a semantic sense to allow this. For convenient copy&paste this could be allowed... if it poses no problem, I think it would be nice to have. But it is not really required
>
But what should it mean? Should it become fields in the Script class, or just be stripped of access and staticness and introduced as locals in the main() ?
Since it doesn’t work in Groovy now, I don’t understand why it’s in the repo at all…
-Jesper
Re: Progress on the Antlr4-based parser
Posted by Jochen Theodorou <bl...@gmx.org>.
On 07.03.2016 07:48, Jesper Steen Møller wrote:
[...]
>
> So you can help me with a couple of answers:
>
> * Is it a goal to close the gap to Java’s more trickier syntax, such
> as placement-new and placement constructor calls? From what I can
> tell, we’d have to extend the AST too to support that. Has anyone
> ever used that except in compiler test cases?
I had to actually look up what you mean... so you can give type
arguments in a constructor call? I did actually not know this. I would
not give this one any priority, unless it is easy to do... but then I
would not really give it an AST implementation yet
> * ./benchmark/bench/heapsort.groovy uses access modfiers on the
> script’s local variables — that’s not really allowed, is it? How
> should that work? It can’t get it to work in Groovy 2.4.x
You mean like "public static final long IM = 139968"? It does not really
make a semantic sense to allow this. For convenient copy&paste this
could be allowed... if it poses no problem, I think it would be nice to
have. But it is not really required
bye Jochen