Adding Kotlin support to the scripting bundle

[Mike Thomsen <mi...@gmail.com>]

I finally chased down the behavior differences that I found and
discussed with Matt in the previous PR. Kotlin's got very promising
(long term) JSR223 support but with a few caveats in the short term:

1. Its ScriptEngine does not support ScriptEngine#get properly, so any
component that uses get(String) to fetch an object defined in a script
won't work with Kotlin (ex InvokeScriptedProcessor).
2. Invocable vs Compilable have very different behaviors WRT bindings.
3. Until Kotlin's JSR223 engine matures, we'd probably need to limit
it to ExecuteScript with extra documentation explaining that it
doesn't behave quite as expected.

What I figured out, and it makes sense given how Kotlin works, is that
bindings cannot be used to provide dynamic injection of variables. So
just calling "session" as an implied ProcessSession won't work. You
have to do something like this:

val session = bindings["session"] as ProcessSession

Either on your own or in a setup block for every script.

I feel like there's value here, but wanted to throw it out there to
other devs before going down the process of finalizing the
ExecuteScript support and starting to exclude it from the other
components.

On a side note, we might want to add an InvokeCompiledScript processor
that's like InvokeScriptedProcessor but is designed for Kotlin, JRuby
and Groovy which work really well with taking scripts down to Java
byte code and working with streamlined fat jars. Could be another
angle on simplifying deployments of scripts and script-like
components.

Thanks,

Mike

Re: Adding Kotlin support to the scripting bundle

[Mike Thomsen <mi...@gmail.com>]

Full disclosure: I can barely write hello world in Kotlin right now.

Yeah, that seems like the right way to do it. What I found was that if
I did a bunch of preloads like setting up an implicit session,
context, REL_SUCCESS, REL_FAILURE by simply doing val session =
bindings[... for each import it worked very smoothly and gave the
ExecuteScript experience with Kotlin. I'll repurpose NIFI-4307 to
focus on this feature. I already have all of the test coverage ready
(I think) with my branch of ExecuteScript.

On Mon, Nov 7, 2022 at 2:37 PM David Handermann
<ex...@apache.org> wrote:
>
> Concurring with Matt, this sounds like a potential opportunity for a
> scripted Processor implementation specific to Kotlin.
>
> Regards,
> David Handermann
>
> On Mon, Nov 7, 2022 at 1:11 PM Matt Burgess <ma...@apache.org> wrote:
>
> > Mike,
> >
> > As an alternative it might be worth looking into a processor
> > specifically for Kotlin like we have with ExecuteGroovyScript (EGS).
> > ExecuteGroovyScript doesn't use JSR-223 and instead goes right for the
> > GroovyShell. This allows for cool things like Groovy-specific idioms
> > and helpful "meta-capabilities" such as redirect I/O to/from FlowFile
> > objects. Depending on how they've done their entry point(s), an
> > ExecuteKotlinScript processor could leverage compiled scripts without
> > needing the JSR-223 impl, as well as adding some Kotlin-ness to the
> > scripting capabilities. I hesitate to suggest this for any of the
> > other scripting languages but as Kotlin becomes more popular and given
> > its tight integration with the JVM (vs Jython for example), that might
> > be the way to go.
> >
> > Regards,
> > Matt
> >
> >
> > On Sun, Nov 6, 2022 at 6:16 PM Mike Thomsen <mi...@gmail.com>
> > wrote:
> > >
> > > I finally chased down the behavior differences that I found and
> > > discussed with Matt in the previous PR. Kotlin's got very promising
> > > (long term) JSR223 support but with a few caveats in the short term:
> > >
> > > 1. Its ScriptEngine does not support ScriptEngine#get properly, so any
> > > component that uses get(String) to fetch an object defined in a script
> > > won't work with Kotlin (ex InvokeScriptedProcessor).
> > > 2. Invocable vs Compilable have very different behaviors WRT bindings.
> > > 3. Until Kotlin's JSR223 engine matures, we'd probably need to limit
> > > it to ExecuteScript with extra documentation explaining that it
> > > doesn't behave quite as expected.
> > >
> > > What I figured out, and it makes sense given how Kotlin works, is that
> > > bindings cannot be used to provide dynamic injection of variables. So
> > > just calling "session" as an implied ProcessSession won't work. You
> > > have to do something like this:
> > >
> > > val session = bindings["session"] as ProcessSession
> > >
> > > Either on your own or in a setup block for every script.
> > >
> > > I feel like there's value here, but wanted to throw it out there to
> > > other devs before going down the process of finalizing the
> > > ExecuteScript support and starting to exclude it from the other
> > > components.
> > >
> > > On a side note, we might want to add an InvokeCompiledScript processor
> > > that's like InvokeScriptedProcessor but is designed for Kotlin, JRuby
> > > and Groovy which work really well with taking scripts down to Java
> > > byte code and working with streamlined fat jars. Could be another
> > > angle on simplifying deployments of scripts and script-like
> > > components.
> > >
> > > Thanks,
> > >
> > > Mike
> >

Re: Adding Kotlin support to the scripting bundle

[David Handermann <ex...@apache.org>]

Concurring with Matt, this sounds like a potential opportunity for a
scripted Processor implementation specific to Kotlin.

Regards,
David Handermann

On Mon, Nov 7, 2022 at 1:11 PM Matt Burgess <ma...@apache.org> wrote:

> Mike,
>
> As an alternative it might be worth looking into a processor
> specifically for Kotlin like we have with ExecuteGroovyScript (EGS).
> ExecuteGroovyScript doesn't use JSR-223 and instead goes right for the
> GroovyShell. This allows for cool things like Groovy-specific idioms
> and helpful "meta-capabilities" such as redirect I/O to/from FlowFile
> objects. Depending on how they've done their entry point(s), an
> ExecuteKotlinScript processor could leverage compiled scripts without
> needing the JSR-223 impl, as well as adding some Kotlin-ness to the
> scripting capabilities. I hesitate to suggest this for any of the
> other scripting languages but as Kotlin becomes more popular and given
> its tight integration with the JVM (vs Jython for example), that might
> be the way to go.
>
> Regards,
> Matt
>
>
> On Sun, Nov 6, 2022 at 6:16 PM Mike Thomsen <mi...@gmail.com>
> wrote:
> >
> > I finally chased down the behavior differences that I found and
> > discussed with Matt in the previous PR. Kotlin's got very promising
> > (long term) JSR223 support but with a few caveats in the short term:
> >
> > 1. Its ScriptEngine does not support ScriptEngine#get properly, so any
> > component that uses get(String) to fetch an object defined in a script
> > won't work with Kotlin (ex InvokeScriptedProcessor).
> > 2. Invocable vs Compilable have very different behaviors WRT bindings.
> > 3. Until Kotlin's JSR223 engine matures, we'd probably need to limit
> > it to ExecuteScript with extra documentation explaining that it
> > doesn't behave quite as expected.
> >
> > What I figured out, and it makes sense given how Kotlin works, is that
> > bindings cannot be used to provide dynamic injection of variables. So
> > just calling "session" as an implied ProcessSession won't work. You
> > have to do something like this:
> >
> > val session = bindings["session"] as ProcessSession
> >
> > Either on your own or in a setup block for every script.
> >
> > I feel like there's value here, but wanted to throw it out there to
> > other devs before going down the process of finalizing the
> > ExecuteScript support and starting to exclude it from the other
> > components.
> >
> > On a side note, we might want to add an InvokeCompiledScript processor
> > that's like InvokeScriptedProcessor but is designed for Kotlin, JRuby
> > and Groovy which work really well with taking scripts down to Java
> > byte code and working with streamlined fat jars. Could be another
> > angle on simplifying deployments of scripts and script-like
> > components.
> >
> > Thanks,
> >
> > Mike
>

Re: Adding Kotlin support to the scripting bundle

[Matt Burgess <ma...@apache.org>]

Mike,

As an alternative it might be worth looking into a processor
specifically for Kotlin like we have with ExecuteGroovyScript (EGS).
ExecuteGroovyScript doesn't use JSR-223 and instead goes right for the
GroovyShell. This allows for cool things like Groovy-specific idioms
and helpful "meta-capabilities" such as redirect I/O to/from FlowFile
objects. Depending on how they've done their entry point(s), an
ExecuteKotlinScript processor could leverage compiled scripts without
needing the JSR-223 impl, as well as adding some Kotlin-ness to the
scripting capabilities. I hesitate to suggest this for any of the
other scripting languages but as Kotlin becomes more popular and given
its tight integration with the JVM (vs Jython for example), that might
be the way to go.

Regards,
Matt


On Sun, Nov 6, 2022 at 6:16 PM Mike Thomsen <mi...@gmail.com> wrote:
>
> I finally chased down the behavior differences that I found and
> discussed with Matt in the previous PR. Kotlin's got very promising
> (long term) JSR223 support but with a few caveats in the short term:
>
> 1. Its ScriptEngine does not support ScriptEngine#get properly, so any
> component that uses get(String) to fetch an object defined in a script
> won't work with Kotlin (ex InvokeScriptedProcessor).
> 2. Invocable vs Compilable have very different behaviors WRT bindings.
> 3. Until Kotlin's JSR223 engine matures, we'd probably need to limit
> it to ExecuteScript with extra documentation explaining that it
> doesn't behave quite as expected.
>
> What I figured out, and it makes sense given how Kotlin works, is that
> bindings cannot be used to provide dynamic injection of variables. So
> just calling "session" as an implied ProcessSession won't work. You
> have to do something like this:
>
> val session = bindings["session"] as ProcessSession
>
> Either on your own or in a setup block for every script.
>
> I feel like there's value here, but wanted to throw it out there to
> other devs before going down the process of finalizing the
> ExecuteScript support and starting to exclude it from the other
> components.
>
> On a side note, we might want to add an InvokeCompiledScript processor
> that's like InvokeScriptedProcessor but is designed for Kotlin, JRuby
> and Groovy which work really well with taking scripts down to Java
> byte code and working with streamlined fat jars. Could be another
> angle on simplifying deployments of scripts and script-like
> components.
>
> Thanks,
>
> Mike