You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@groovy.apache.org by "Winnebeck, Jason" <Ja...@windstream.com> on 2016/11/22 14:14:31 UTC
JDK8 Streams / Closure cast to interface
I love Groovy. I also love the new streams functionality in JDK 8. But, I am weary of the performance implications of Groovy + Streams, because to use streams you must use Groovy closures. I see the code generated creates a new closure instance then uses castToType to cast the closure to the JDK8 functionality interface. Does Groovy make this efficient, or is the proxy generation from this going to be excessive if running a lot of small stream operations?
I think an interesting feature of Groovy would be if it sees a closure cast implicitly or explicitly to a certain interface type, it could make the closure implement the interface. You'd still have the overhead of compile-time class generation versus lambdas, but at least you wouldn't have to create proxies, and maybe there is an improved chance of JIT inlining? Even if it was supported within a single statement, like "Function<String, String> x = {it.trim()}" or Stream.of("abc").map { it.trim() }, the closure with trim could implement Function. Of course for backwards compatibility the class could still extend Closure and still implement call methods, but also implement the method apply, which delegates to call.
Jason Winnebeck
Software Engineer III Contractor - IT Software Development | Windstream
600 Willowbrook Office Park, Rochester, NY 14450
Jason.Winnebeck@windstream.com<ma...@windstream.com> | windstreambusiness.com
o: 585.794-4585
This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments.
RE: JDK8 Streams / Closure cast to interface
Posted by "Winnebeck, Jason" <Ja...@windstream.com>.
It sounds like a good idea to support Java lambda syntax, or at least provide a way in Groovy to use Java lambdas. The only concern I have is that I still think the closure syntax is better, since you don't have parenthesis overload, and I still like the curly braces to distinguish that we are changing from a "parameter passing" context to an "expression" context. To be the lambda single expression syntax looks too much like parameter passing rather than defining a lambda.
stream.forEach((it) -> System.out.println(it))
stream.forEach{ System.out.println(it) }
I agree with Jochen about not having to create the Closure, if that is an option that sounds great. At first I thought it would be an observable (and therefore potentially breaking) change because the class would extend Closure, but then I realize when you pass a closure to a SAM method, the getClass is a java.lang.reflect.Proxy subclass instance. I have a hard time thinking that users would call Proxy.isProxyClass on the result, but who knows. But, I wonder if that potential breakage is worth it if Groovy could replace the closure that does not escape with a non-closure lambda (Java 8+) or non-closure inner class (Java 7 and earlier) when in static compilation mode. Since we know that the closure is to be assigned to a non-Closure type, we know setDelegate/setOwner can't be called, nor can setResolveStrategy. That situation should enable the Groovy static compiler to know whether or not use of Closure is required.
I think it's possible only under static compiler, because in dynamic mode, the runtime type, metaclass or runtime extension method could introduce an overload of the same method that takes Closure. I think only the static compiler can guarantee that the closure can be replaced with non-closure type without changing semantics (other than the change of getClass from proxy to something else).
Jason
-----Original Message-----
From: Graeme Rocher [mailto:graeme.rocher@gmail.com]
Sent: Wednesday, November 23, 2016 6:18 AM
To: users@groovy.apache.org
Subject: Re: JDK8 Streams / Closure cast to interface
Jochen said - “If you want this really efficient, you have to skip the generation of the Groovy Closure instance. This is doable and I plan to do this, but it will be a breaking change (all open blocks would be realised using the same class for example, having that instance would even become optional).”
Could I recommend that since the new parser now supports the lambda syntax that we add to Groovy the optimised version for lambdas and keep the closure behaviour as is to avoid a breaking change?
This would mean that there is a visual syntactical difference between closures and lambdas and the behaviour of lambdas will be closer to Java thus not surprising folks coming from Java.
We could then keep the behaviour of closures the same as it is now without it being a breaking change.
Thoughts?
On Wed, Nov 23, 2016 at 11:11 AM, Jochen Theodorou <bl...@gmx.org> wrote:
>
>
> On 22.11.2016 19:37, Winnebeck, Jason wrote:
>>
>> I was referring to a compile-time generation of the class -- that the
>> Closure itself that is normally generated implements the interface natively.
>
>
> Which means we are talking about direct assignments to local variables only?
> I mean the static compiler can do that in more cases, but frankly, why
> should the static compiler even bother with creating a Closure?
>
>> That would make it equivalent to anonymous class in Java 7 and
>> earlier for calling functional (or any SAM type) methods. That
>> wouldn't have any problems on Android, and should be as efficient as Java without lambdas?
>
>
> Android is ok, yes. As efficient as Java without lambdas... well..
> that I am not sure of. Even if you make it as an anonymous inner class
> that implements the interface and extends Closure, even if the
> interface method will just call doCall, you will still have to pay the
> initialization cost of the Closure, and Closure will inspect itself to
> set the maximum parameter number for example, you will still request a meta class and do some other things.
> So the init would not be as efficient. The method invocation should be
> similar to Java, if done from Java then, since there is no dynamic
> call. So here you would gain over todays Closure.
>
> But for typical usages of non-static Groovy the gain would be almost nil.
> Unless we can lift restrictions
>
>> I would assume the interface's method delegating to doCall would get
>> inlined. In other words, Groovy generating code like:
>>
>> class X {
>> public static void main(String[] args) {
>> Stream.of(1).forEach(new x__closure1(X.class, X.class));
>> }
>>
>> private static class x__closure1 extends Closure<Void>
>> implements Consumer<Integer> {
>> public x__closure1(Object owner, Object thisObject) {
>> super(owner, thisObject);
>> }
>>
>> void doCall(Integer x) {
>> System.out.println(x);
>> }
>>
>> @Override
>> public void accept(Integer x) {
>> doCall(x);
>> }
>> }
>> }
>>
>> From Groovy: Stream.of(1).forEach { println it }
>>
>> The new part being that Groovy added the accept method and implements
>> to the closure it already normally would have generated, and
>> castToType would not need to be called. All of the code manipulation
>> is done at compile-time so it is fully STC and Android compatible,
>> and no reflection is in use. You still have the a little more
>> overhead of Closure object compared to Java static inner class, but I
>> imagine this must be a lot less than proxy, but still allows Closure
>> to use the owner/delegate patterns that Groovy is known for, and I
>> assume would not affect backwards compatibility as superclass stays Closure.
>>
>> Of course, if it were possible for compiler to determine that the
>> closure is never using owner, delegate, or "thisObject", then it
>> could be possible to drop the "extends Closure" entirely if it can be
>> proven that the "closureness" of the object can never be observed.
>> But that's likely not possible as any method taking an interface
>> could choose to check for instanceof Closure and/or cast or do
>> something special if Groovy closure is passed in -- although is that
>> even possible today since Groovy actually passes in a proxy?
>
>
> It depends on if the implicit "this" is used or not. { println it }
> uses implicit this, thus cannot do it for sure. { this.println it },
> no implicit, thus can be optimized.
>
> I am wondering what would happen if we had 2 versions, one with
> implicit this delegation logic, the other not. Because if the usage is
> just an appended block and the target is just a functional interface,
> you will not need the version with delegate.
>
> bye Jochen
--
Graeme Rocher
This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments.
Re: JDK8 Streams / Closure cast to interface
Posted by Graeme Rocher <gr...@gmail.com>.
Jochen said - “If you want this really efficient, you have to skip the
generation of the Groovy Closure instance. This is doable and I plan
to do this, but it will be a breaking change (all open blocks would be
realised using the same class for example, having that instance would
even become optional).”
Could I recommend that since the new parser now supports the lambda
syntax that we add to Groovy the optimised version for lambdas and
keep the closure behaviour as is to avoid a breaking change?
This would mean that there is a visual syntactical difference between
closures and lambdas and the behaviour of lambdas will be closer to
Java thus not surprising folks coming from Java.
We could then keep the behaviour of closures the same as it is now
without it being a breaking change.
Thoughts?
On Wed, Nov 23, 2016 at 11:11 AM, Jochen Theodorou <bl...@gmx.org> wrote:
>
>
> On 22.11.2016 19:37, Winnebeck, Jason wrote:
>>
>> I was referring to a compile-time generation of the class -- that the
>> Closure itself that is normally generated implements the interface natively.
>
>
> Which means we are talking about direct assignments to local variables only?
> I mean the static compiler can do that in more cases, but frankly, why
> should the static compiler even bother with creating a Closure?
>
>> That would make it equivalent to anonymous class in Java 7 and earlier for
>> calling functional (or any SAM type) methods. That wouldn't have any
>> problems on Android, and should be as efficient as Java without lambdas?
>
>
> Android is ok, yes. As efficient as Java without lambdas... well.. that I am
> not sure of. Even if you make it as an anonymous inner class that implements
> the interface and extends Closure, even if the interface method will just
> call doCall, you will still have to pay the initialization cost of the
> Closure, and Closure will inspect itself to set the maximum parameter number
> for example, you will still request a meta class and do some other things.
> So the init would not be as efficient. The method invocation should be
> similar to Java, if done from Java then, since there is no dynamic call. So
> here you would gain over todays Closure.
>
> But for typical usages of non-static Groovy the gain would be almost nil.
> Unless we can lift restrictions
>
>> I would assume the interface's method delegating to doCall would get
>> inlined. In other words, Groovy generating code like:
>>
>> class X {
>> public static void main(String[] args) {
>> Stream.of(1).forEach(new x__closure1(X.class, X.class));
>> }
>>
>> private static class x__closure1 extends Closure<Void> implements
>> Consumer<Integer> {
>> public x__closure1(Object owner, Object thisObject) {
>> super(owner, thisObject);
>> }
>>
>> void doCall(Integer x) {
>> System.out.println(x);
>> }
>>
>> @Override
>> public void accept(Integer x) {
>> doCall(x);
>> }
>> }
>> }
>>
>> From Groovy: Stream.of(1).forEach { println it }
>>
>> The new part being that Groovy added the accept method and implements to
>> the closure it already normally would have generated, and castToType would
>> not need to be called. All of the code manipulation is done at compile-time
>> so it is fully STC and Android compatible, and no reflection is in use. You
>> still have the a little more overhead of Closure object compared to Java
>> static inner class, but I imagine this must be a lot less than proxy, but
>> still allows Closure to use the owner/delegate patterns that Groovy is known
>> for, and I assume would not affect backwards compatibility as superclass
>> stays Closure.
>>
>> Of course, if it were possible for compiler to determine that the closure
>> is never using owner, delegate, or "thisObject", then it could be possible
>> to drop the "extends Closure" entirely if it can be proven that the
>> "closureness" of the object can never be observed. But that's likely not
>> possible as any method taking an interface could choose to check for
>> instanceof Closure and/or cast or do something special if Groovy closure is
>> passed in -- although is that even possible today since Groovy actually
>> passes in a proxy?
>
>
> It depends on if the implicit "this" is used or not. { println it } uses
> implicit this, thus cannot do it for sure. { this.println it }, no implicit,
> thus can be optimized.
>
> I am wondering what would happen if we had 2 versions, one with implicit
> this delegation logic, the other not. Because if the usage is just an
> appended block and the target is just a functional interface, you will not
> need the version with delegate.
>
> bye Jochen
--
Graeme Rocher
Re: JDK8 Streams / Closure cast to interface
Posted by Jochen Theodorou <bl...@gmx.org>.
On 22.11.2016 19:37, Winnebeck, Jason wrote:
> I was referring to a compile-time generation of the class -- that the Closure itself that is normally generated implements the interface natively.
Which means we are talking about direct assignments to local variables
only? I mean the static compiler can do that in more cases, but frankly,
why should the static compiler even bother with creating a Closure?
> That would make it equivalent to anonymous class in Java 7 and earlier for calling functional (or any SAM type) methods. That wouldn't have any problems on Android, and should be as efficient as Java without lambdas?
Android is ok, yes. As efficient as Java without lambdas... well.. that
I am not sure of. Even if you make it as an anonymous inner class that
implements the interface and extends Closure, even if the interface
method will just call doCall, you will still have to pay the
initialization cost of the Closure, and Closure will inspect itself to
set the maximum parameter number for example, you will still request a
meta class and do some other things. So the init would not be as
efficient. The method invocation should be similar to Java, if done from
Java then, since there is no dynamic call. So here you would gain over
todays Closure.
But for typical usages of non-static Groovy the gain would be almost
nil. Unless we can lift restrictions
> I would assume the interface's method delegating to doCall would get inlined. In other words, Groovy generating code like:
>
> class X {
> public static void main(String[] args) {
> Stream.of(1).forEach(new x__closure1(X.class, X.class));
> }
>
> private static class x__closure1 extends Closure<Void> implements Consumer<Integer> {
> public x__closure1(Object owner, Object thisObject) {
> super(owner, thisObject);
> }
>
> void doCall(Integer x) {
> System.out.println(x);
> }
>
> @Override
> public void accept(Integer x) {
> doCall(x);
> }
> }
> }
>
> From Groovy: Stream.of(1).forEach { println it }
>
> The new part being that Groovy added the accept method and implements to the closure it already normally would have generated, and castToType would not need to be called. All of the code manipulation is done at compile-time so it is fully STC and Android compatible, and no reflection is in use. You still have the a little more overhead of Closure object compared to Java static inner class, but I imagine this must be a lot less than proxy, but still allows Closure to use the owner/delegate patterns that Groovy is known for, and I assume would not affect backwards compatibility as superclass stays Closure.
>
> Of course, if it were possible for compiler to determine that the closure is never using owner, delegate, or "thisObject", then it could be possible to drop the "extends Closure" entirely if it can be proven that the "closureness" of the object can never be observed. But that's likely not possible as any method taking an interface could choose to check for instanceof Closure and/or cast or do something special if Groovy closure is passed in -- although is that even possible today since Groovy actually passes in a proxy?
It depends on if the implicit "this" is used or not. { println it } uses
implicit this, thus cannot do it for sure. { this.println it }, no
implicit, thus can be optimized.
I am wondering what would happen if we had 2 versions, one with implicit
this delegation logic, the other not. Because if the usage is just an
appended block and the target is just a functional interface, you will
not need the version with delegate.
bye Jochen
RE: JDK8 Streams / Closure cast to interface
Posted by "Winnebeck, Jason" <Ja...@windstream.com>.
I was referring to a compile-time generation of the class -- that the Closure itself that is normally generated implements the interface natively. That would make it equivalent to anonymous class in Java 7 and earlier for calling functional (or any SAM type) methods. That wouldn't have any problems on Android, and should be as efficient as Java without lambdas? I would assume the interface's method delegating to doCall would get inlined. In other words, Groovy generating code like:
class X {
public static void main(String[] args) {
Stream.of(1).forEach(new x__closure1(X.class, X.class));
}
private static class x__closure1 extends Closure<Void> implements Consumer<Integer> {
public x__closure1(Object owner, Object thisObject) {
super(owner, thisObject);
}
void doCall(Integer x) {
System.out.println(x);
}
@Override
public void accept(Integer x) {
doCall(x);
}
}
}
From Groovy: Stream.of(1).forEach { println it }
The new part being that Groovy added the accept method and implements to the closure it already normally would have generated, and castToType would not need to be called. All of the code manipulation is done at compile-time so it is fully STC and Android compatible, and no reflection is in use. You still have the a little more overhead of Closure object compared to Java static inner class, but I imagine this must be a lot less than proxy, but still allows Closure to use the owner/delegate patterns that Groovy is known for, and I assume would not affect backwards compatibility as superclass stays Closure.
Of course, if it were possible for compiler to determine that the closure is never using owner, delegate, or "thisObject", then it could be possible to drop the "extends Closure" entirely if it can be proven that the "closureness" of the object can never be observed. But that's likely not possible as any method taking an interface could choose to check for instanceof Closure and/or cast or do something special if Groovy closure is passed in -- although is that even possible today since Groovy actually passes in a proxy?
Jason
-----Original Message-----
From: Jochen Theodorou [mailto:blackdrag@gmx.org]
Sent: Tuesday, November 22, 2016 12:01 PM
To: users@groovy.apache.org
Subject: Re: JDK8 Streams / Closure cast to interface
On 22.11.2016 15:14, Winnebeck, Jason wrote:
> I love Groovy. I also love the new streams functionality in JDK 8.
> But, I am weary of the performance implications of Groovy + Streams,
> because to use streams you must use Groovy closures. I see the code
> generated creates a new closure instance then uses castToType to cast
> the closure to the JDK8 functionality interface. Does Groovy make this
> efficient, or is the proxy generation from this going to be excessive
> if running a lot of small stream operations?
it is using a dynamic proxy with some optimized code paths.. such a proxy does for example not have to go through call to get to doCall. If you want this really efficient, you have to skip the generation of the Groovy Closure instance. This is doable and I plan to do this, but it will be a breaking change (all open blocks would be realized using the same class for example, having that instance would even become optional).
> I think an interesting feature of Groovy would be if it sees a closure
> cast implicitly or explicitly to a certain interface type, it could
> make the closure implement the interface. You'd still have the
> overhead of compile-time class generation versus lambdas, but at least
> you wouldn't have to create proxies, and maybe there is an improved
> chance of JIT inlining? Even if it was supported within a single
> statement, like "Function<String, String> x = {it.trim()}" or
> Stream.of("abc").map {
> it.trim() }, the closure with trim could implement Function. Of course
> for backwards compatibility the class could still extend Closure and
> still implement call methods, but also implement the method apply,
> which delegates to call.
The price question is now... is that going to be cheaper? runtime class generation is much more expensive than a reflective call - and considering Android, also much more problematic. And you have to be aware of the following... If I make a method call with an open block to a method taking a functional interface, the point in time for the conversion is after runtime method selection and before the actual invocation (or during invocation). At this point the original Closure instance already exists. Really, without test it is difficult to tell about the gain here
Real gains you get only with a much deeper integration
bye Jochen
This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments.
Re: JDK8 Streams / Closure cast to interface
Posted by Jochen Theodorou <bl...@gmx.org>.
On 22.11.2016 15:14, Winnebeck, Jason wrote:
> I love Groovy. I also love the new streams functionality in JDK 8. But,
> I am weary of the performance implications of Groovy + Streams, because
> to use streams you must use Groovy closures. I see the code generated
> creates a new closure instance then uses castToType to cast the closure
> to the JDK8 functionality interface. Does Groovy make this efficient, or
> is the proxy generation from this going to be excessive if running a lot
> of small stream operations?
it is using a dynamic proxy with some optimized code paths.. such a
proxy does for example not have to go through call to get to doCall. If
you want this really efficient, you have to skip the generation of the
Groovy Closure instance. This is doable and I plan to do this, but it
will be a breaking change (all open blocks would be realized using the
same class for example, having that instance would even become optional).
> I think an interesting feature of Groovy would be if it sees a closure
> cast implicitly or explicitly to a certain interface type, it could make
> the closure implement the interface. Youd still have the overhead of
> compile-time class generation versus lambdas, but at least you wouldnt
> have to create proxies, and maybe there is an improved chance of JIT
> inlining? Even if it was supported within a single statement, like
> Function<String, String> x = {it.trim()} or Stream.of(abc).map {
> it.trim() }, the closure with trim could implement Function. Of course
> for backwards compatibility the class could still extend Closure and
> still implement call methods, but also implement the method apply, which
> delegates to call.
The price question is now... is that going to be cheaper? runtime class
generation is much more expensive than a reflective call - and
considering Android, also much more problematic. And you have to be
aware of the following... If I make a method call with an open block to
a method taking a functional interface, the point in time for the
conversion is after runtime method selection and before the actual
invocation (or during invocation). At this point the original Closure
instance already exists. Really, without test it is difficult to tell
about the gain here
Real gains you get only with a much deeper integration
bye Jochen