You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by happy smith <ha...@gmail.com> on 2019/01/30 18:28:45 UTC
ScriptedLookupService with Python
Dear users,
I would like to use ScriptedLookupService with Python, but I am not
succeeding in get any scripts working .
On the basis of an example that I found for groovy, I am trying something
like this:
import org.apache.nifi.lookup.LookupService as LookupService
from org.apache.nifi.processor import PropertyDescriptor
class SequenceLookupService(LookupService):
propertyDescriptor = PropertyDescriptor.Builder().name("Database Connection
Pooling Service").description("The Controller Service that is used to
obtain connection to
database").required(True).identifiesControllerService(LookupService).build()
def OnEnabled():
pass
def OnDisabled():
pass
def initialize():
pass
def lookup(coordinates):
return "test"
def getPropertyDescriptor(name):
return propertyDescriptor
def getPropertyDescriptors():
return [propertyDescriptor]
lookupService = SequenceLookupService()
Could any of you point me to some examples of how to use python for the
ScriptedLookupService ?
Thanks in advance for any support.
Re: ScriptedLookupService with Python
Posted by Mike Thomsen <mi...@gmail.com>.
I agree, and filed a Jira ticket for it:
https://issues.apache.org/jira/browse/NIFI-5995
FWIW, I've used Groovy for this sort of thing a lot. You really can't go
wrong with that choice.
On Thu, Jan 31, 2019 at 4:14 AM happy smith <ha...@gmail.com> wrote:
> Thanks a lot for the quick answer. Even if I am not familiar with Groovy,
> I will try the Groovy script :). I think it is the best solution.
> Really thanks for your support :)
>
> PS: Probably, waiting for the fix of the bug [1], it would be better to
> update the documentation related to the ScriptedLookupService and highlight
> that for the moment the service cannot be used with python.
>
> On Wed, 30 Jan 2019 at 20:18, Matt Burgess <ma...@apache.org> wrote:
>
>> Short answer: Unfortunately this is not currently possible for Jython
>> in NiFi 1.7.0+ because of a Jython bug [1] that never made it into a
>> release.
>>
>> Longer answer: NIFI-5287 [2], released as part of NiFi 1.7.0,
>> introduced a new default interface method for LookupService, in order
>> to avoid breaking API compatibility for those with custom
>> LookupServices. This exposes the bug in [1] for the Jython script
>> engine. At a major release point (NiFi 2.0) we could refactor the NiFi
>> codebase to remove the default interface method, update all internal
>> implementations, and announce that the LookupService API has changed
>> and thus custom implementations would have to be updated. Not sure if
>> we can get away with that for minor releases or not, usually breaking
>> API compatibility is a no-no except for major releases.
>>
>> Possible workarounds:
>> - I haven't tried with a NiFi between versions 1.3.0 (when the
>> service was introduced) and 1.7.0 (when [2] was introduced) but it may
>> be possible as the interface didn't have any default methods then. For
>> your script, you'll want to change "from org.apache.nifi.processor
>> import PropertyDescriptor" to "from org.apache.nifi.components import
>> PropertyDescriptor". Also the OnEnabled and OnDisabled methods should
>> be named onEnabled and onDisabled respectively, and you'll want to
>> implement the remaining interface methods such as getValueType() and
>> getRequiredKeys().
>> - Port your script to Groovy, although since you started with a
>> Groovy example and ported to Jython I'm guessing this isn't an option
>> :) However you would find that Groovy is much faster than Jython, and
>> there is a working example in the unit tests [3].
>> - If you are trying to script up a DB LookupService, there are some
>> similar examples at [4] and [5], and I can't find Joey's in his GitHub
>> but I know he was working on it at one point too.
>>
>> Regards,
>> Matt
>>
>> [1] https://bugs.jython.org/issue2403
>> [2] https://issues.apache.org/jira/browse/NIFI-5287
>> [3]
>> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/resources/groovy/test_lookup_inline.groovy
>> [4]
>> https://funnifi.blogspot.com/2018/08/database-sequence-lookup-with.html
>> [5]
>> https://github.com/brettryan/nifi-drunken-bundle/blob/dev/nifi-drunken-services/src/main/java/com/drunkendev/nifi/services/SQLLookupService.java
>>
>> On Wed, Jan 30, 2019 at 1:29 PM happy smith <ha...@gmail.com>
>> wrote:
>> >
>> > Dear users,
>> >
>> > I would like to use ScriptedLookupService with Python, but I am not
>> succeeding in get any scripts working .
>> >
>> > On the basis of an example that I found for groovy, I am trying
>> something like this:
>> >
>> > import org.apache.nifi.lookup.LookupService as LookupService
>> > from org.apache.nifi.processor import PropertyDescriptor
>> >
>> > class SequenceLookupService(LookupService):
>> > propertyDescriptor = PropertyDescriptor.Builder().name("Database
>> Connection Pooling Service").description("The Controller Service that is
>> used to obtain connection to
>> database").required(True).identifiesControllerService(LookupService).build()
>> >
>> > def OnEnabled():
>> > pass
>> >
>> > def OnDisabled():
>> > pass
>> >
>> > def initialize():
>> > pass
>> >
>> > def lookup(coordinates):
>> > return "test"
>> >
>> > def getPropertyDescriptor(name):
>> > return propertyDescriptor
>> >
>> > def getPropertyDescriptors():
>> > return [propertyDescriptor]
>> >
>> >
>> > lookupService = SequenceLookupService()
>> >
>> > Could any of you point me to some examples of how to use python for the
>> ScriptedLookupService ?
>> > Thanks in advance for any support.
>>
>
Re: ScriptedLookupService with Python
Posted by happy smith <ha...@gmail.com>.
Thanks a lot for the quick answer. Even if I am not familiar with Groovy, I
will try the Groovy script :). I think it is the best solution.
Really thanks for your support :)
PS: Probably, waiting for the fix of the bug [1], it would be better to
update the documentation related to the ScriptedLookupService and highlight
that for the moment the service cannot be used with python.
On Wed, 30 Jan 2019 at 20:18, Matt Burgess <ma...@apache.org> wrote:
> Short answer: Unfortunately this is not currently possible for Jython
> in NiFi 1.7.0+ because of a Jython bug [1] that never made it into a
> release.
>
> Longer answer: NIFI-5287 [2], released as part of NiFi 1.7.0,
> introduced a new default interface method for LookupService, in order
> to avoid breaking API compatibility for those with custom
> LookupServices. This exposes the bug in [1] for the Jython script
> engine. At a major release point (NiFi 2.0) we could refactor the NiFi
> codebase to remove the default interface method, update all internal
> implementations, and announce that the LookupService API has changed
> and thus custom implementations would have to be updated. Not sure if
> we can get away with that for minor releases or not, usually breaking
> API compatibility is a no-no except for major releases.
>
> Possible workarounds:
> - I haven't tried with a NiFi between versions 1.3.0 (when the
> service was introduced) and 1.7.0 (when [2] was introduced) but it may
> be possible as the interface didn't have any default methods then. For
> your script, you'll want to change "from org.apache.nifi.processor
> import PropertyDescriptor" to "from org.apache.nifi.components import
> PropertyDescriptor". Also the OnEnabled and OnDisabled methods should
> be named onEnabled and onDisabled respectively, and you'll want to
> implement the remaining interface methods such as getValueType() and
> getRequiredKeys().
> - Port your script to Groovy, although since you started with a
> Groovy example and ported to Jython I'm guessing this isn't an option
> :) However you would find that Groovy is much faster than Jython, and
> there is a working example in the unit tests [3].
> - If you are trying to script up a DB LookupService, there are some
> similar examples at [4] and [5], and I can't find Joey's in his GitHub
> but I know he was working on it at one point too.
>
> Regards,
> Matt
>
> [1] https://bugs.jython.org/issue2403
> [2] https://issues.apache.org/jira/browse/NIFI-5287
> [3]
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/resources/groovy/test_lookup_inline.groovy
> [4]
> https://funnifi.blogspot.com/2018/08/database-sequence-lookup-with.html
> [5]
> https://github.com/brettryan/nifi-drunken-bundle/blob/dev/nifi-drunken-services/src/main/java/com/drunkendev/nifi/services/SQLLookupService.java
>
> On Wed, Jan 30, 2019 at 1:29 PM happy smith <ha...@gmail.com>
> wrote:
> >
> > Dear users,
> >
> > I would like to use ScriptedLookupService with Python, but I am not
> succeeding in get any scripts working .
> >
> > On the basis of an example that I found for groovy, I am trying
> something like this:
> >
> > import org.apache.nifi.lookup.LookupService as LookupService
> > from org.apache.nifi.processor import PropertyDescriptor
> >
> > class SequenceLookupService(LookupService):
> > propertyDescriptor = PropertyDescriptor.Builder().name("Database
> Connection Pooling Service").description("The Controller Service that is
> used to obtain connection to
> database").required(True).identifiesControllerService(LookupService).build()
> >
> > def OnEnabled():
> > pass
> >
> > def OnDisabled():
> > pass
> >
> > def initialize():
> > pass
> >
> > def lookup(coordinates):
> > return "test"
> >
> > def getPropertyDescriptor(name):
> > return propertyDescriptor
> >
> > def getPropertyDescriptors():
> > return [propertyDescriptor]
> >
> >
> > lookupService = SequenceLookupService()
> >
> > Could any of you point me to some examples of how to use python for the
> ScriptedLookupService ?
> > Thanks in advance for any support.
>
Re: ScriptedLookupService with Python
Posted by Matt Burgess <ma...@apache.org>.
Short answer: Unfortunately this is not currently possible for Jython
in NiFi 1.7.0+ because of a Jython bug [1] that never made it into a
release.
Longer answer: NIFI-5287 [2], released as part of NiFi 1.7.0,
introduced a new default interface method for LookupService, in order
to avoid breaking API compatibility for those with custom
LookupServices. This exposes the bug in [1] for the Jython script
engine. At a major release point (NiFi 2.0) we could refactor the NiFi
codebase to remove the default interface method, update all internal
implementations, and announce that the LookupService API has changed
and thus custom implementations would have to be updated. Not sure if
we can get away with that for minor releases or not, usually breaking
API compatibility is a no-no except for major releases.
Possible workarounds:
- I haven't tried with a NiFi between versions 1.3.0 (when the
service was introduced) and 1.7.0 (when [2] was introduced) but it may
be possible as the interface didn't have any default methods then. For
your script, you'll want to change "from org.apache.nifi.processor
import PropertyDescriptor" to "from org.apache.nifi.components import
PropertyDescriptor". Also the OnEnabled and OnDisabled methods should
be named onEnabled and onDisabled respectively, and you'll want to
implement the remaining interface methods such as getValueType() and
getRequiredKeys().
- Port your script to Groovy, although since you started with a
Groovy example and ported to Jython I'm guessing this isn't an option
:) However you would find that Groovy is much faster than Jython, and
there is a working example in the unit tests [3].
- If you are trying to script up a DB LookupService, there are some
similar examples at [4] and [5], and I can't find Joey's in his GitHub
but I know he was working on it at one point too.
Regards,
Matt
[1] https://bugs.jython.org/issue2403
[2] https://issues.apache.org/jira/browse/NIFI-5287
[3] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/test/resources/groovy/test_lookup_inline.groovy
[4] https://funnifi.blogspot.com/2018/08/database-sequence-lookup-with.html
[5] https://github.com/brettryan/nifi-drunken-bundle/blob/dev/nifi-drunken-services/src/main/java/com/drunkendev/nifi/services/SQLLookupService.java
On Wed, Jan 30, 2019 at 1:29 PM happy smith <ha...@gmail.com> wrote:
>
> Dear users,
>
> I would like to use ScriptedLookupService with Python, but I am not succeeding in get any scripts working .
>
> On the basis of an example that I found for groovy, I am trying something like this:
>
> import org.apache.nifi.lookup.LookupService as LookupService
> from org.apache.nifi.processor import PropertyDescriptor
>
> class SequenceLookupService(LookupService):
> propertyDescriptor = PropertyDescriptor.Builder().name("Database Connection Pooling Service").description("The Controller Service that is used to obtain connection to database").required(True).identifiesControllerService(LookupService).build()
>
> def OnEnabled():
> pass
>
> def OnDisabled():
> pass
>
> def initialize():
> pass
>
> def lookup(coordinates):
> return "test"
>
> def getPropertyDescriptor(name):
> return propertyDescriptor
>
> def getPropertyDescriptors():
> return [propertyDescriptor]
>
>
> lookupService = SequenceLookupService()
>
> Could any of you point me to some examples of how to use python for the ScriptedLookupService ?
> Thanks in advance for any support.