You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by bsabin <bo...@gmail.com> on 2016/09/09 14:25:25 UTC
nifi 1.0 extending a standard processor
Hi,
I have a custom processor that extends the MergeContent proc and when NiFi
starts I have this error in logs:
2016-09-09 18:17:00,607 ERROR [main] org.apache.nifi.NiFi Failure to launch
NiFi due to java.util.ServiceConfigurationError:
org.apache.nifi.processor.Processor: Provider
org.apache.nifi.processors.standard.DetectDuplicate could not be
instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor:
Provider org.apache.nifi.processors.standard.DetectDuplicate could not be
instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
~[na:1.8.0_101]
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
~[na:1.8.0_101]
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
~[na:1.8.0_101]
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
~[na:1.8.0_101]
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
~[na:1.8.0_101]
at
org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:107)
~[nifi-nar-utils-1.0.0.jar:1.0.0]
at
org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:88)
~[nifi-nar-utils-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.<init>(NiFi.java:135)
~[nifi-runtime-1.0.0.jar:1.0.0]
at org.apache.nifi.NiFi.main(NiFi.java:243)
~[nifi-runtime-1.0.0.jar:1.0.0]
Caused by: java.lang.NoClassDefFoundError:
org/apache/nifi/distributed/cache/client/Serializer
at java.lang.Class.getDeclaredConstructors0(Native Method)
~[na:1.8.0_101]
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
~[na:1.8.0_101]
at java.lang.Class.getConstructor0(Class.java:3075) ~[na:1.8.0_101]
at java.lang.Class.newInstance(Class.java:412) ~[na:1.8.0_101]
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
~[na:1.8.0_101]
... 6 common frames omitted
Caused by: java.lang.ClassNotFoundException:
org.apache.nifi.distributed.cache.client.Serializer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
~[na:1.8.0_101]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
~[na:1.8.0_101]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
~[na:1.8.0_101]
On Nifi 0.6.1/ 0.7.0 it worked, but when I've tried to use PutSQL proc, I
could not see in the property list the Controller Service populated.
Is there a proper way to extend the behaviour of a standard processor ?
Thanks
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: nifi 1.0 extending a standard processor
Posted by peter-ho <pe...@live.com>.
my project has changed quite a bit, so can't really tell the root cause or
resolution anymore. Also moved from maven to gradle. If you post yours, I
can help identify a problem.
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306p15470.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: nifi 1.0 extending a standard processor
Posted by vamsi77874 <va...@gmail.com>.
Could post your pom file, I am having same issue
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306p15469.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: nifi 1.0 extending a standard processor
Posted by peter-ho <pe...@live.com>.
Thanks for your help, Koji.
That fixed the problem.
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306p13931.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: nifi 1.0 extending a standard processor
Posted by Koji Kawamura <ij...@gmail.com>.
Hi Peter,
Would you check nifi-mysftp-processors nar to see if there's any nar
dependency by any chance?
I tested following dependency setting and got the same error:
- nifi-B-processors-nar
- nar dependency to nifi-A-processors-nar
- dependency to nifi-B-processors
- nifi-B-processors
- nar dependency to nifi-C-processors-nar (nar dependency shouldn't
be used in a project that is not a 'nar' project)
Thanks,
Koji
On Sat, Nov 19, 2016 at 3:20 AM, peter-ho <pe...@live.com> wrote:
> After adding the nar as dependency to the nar pom.xml, I'm getting the
> following error:
>
> [ERROR] Failed to execute goal
> org.apache.nifi:nifi-nar-maven-plugin:1.1.0:nar (default-nar) on project
> discover-mysftp-nar: Error assembling NAR: Each NAR represents a
> ClassLoader. A NAR dependency allows that NAR's ClassLoader to be used as
> the parent of this NAR's ClassLoader. As a result, only a single NAR
> dependency is allowed. -> [Help 1]
>
> Have you seen it? I tried adding <scope>provided</scope> to
> nifi-standard-nar in nar's pom, which got it to build, but when nifi starts,
> it can't initialize the processor complaining class def not found for the
> classes I refer to.
>
> I didn't add the project/artifact to assembly, as I'm not building
> everything, just planning to add an extra nar. My nar's pom:
>
> <dependencies>
> <dependency>
> <groupId>com.nifi</groupId>
> <artifactId>nifi-mysftp-processors</artifactId>
> <version>1.0-SNAPSHOT</version>
> </dependency>
> <dependency>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-standard-nar</artifactId>
> <type>nar</type>
> </dependency>
> </dependencies>
>
>
> Thanks
> Peter
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306p13924.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: nifi 1.0 extending a standard processor
Posted by peter-ho <pe...@live.com>.
After adding the nar as dependency to the nar pom.xml, I'm getting the
following error:
[ERROR] Failed to execute goal
org.apache.nifi:nifi-nar-maven-plugin:1.1.0:nar (default-nar) on project
discover-mysftp-nar: Error assembling NAR: Each NAR represents a
ClassLoader. A NAR dependency allows that NAR's ClassLoader to be used as
the parent of this NAR's ClassLoader. As a result, only a single NAR
dependency is allowed. -> [Help 1]
Have you seen it? I tried adding <scope>provided</scope> to
nifi-standard-nar in nar's pom, which got it to build, but when nifi starts,
it can't initialize the processor complaining class def not found for the
classes I refer to.
I didn't add the project/artifact to assembly, as I'm not building
everything, just planning to add an extra nar. My nar's pom:
<dependencies>
<dependency>
<groupId>com.nifi</groupId>
<artifactId>nifi-mysftp-processors</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-nar</artifactId>
<type>nar</type>
</dependency>
</dependencies>
Thanks
Peter
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306p13924.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: nifi 1.0 extending a standard processor
Posted by Koji Kawamura <ij...@apache.org>.
Hello,
Have you been able to find a way to extend the standard processors?
I was wondering what would be the proper way to do that. So I tested few things:
1. Create new custom nar which depends on nifi-standard-processors
First, I tried to add new nar bundle, let's say "nifi-stdpllus-bundle"
following "Maven Projects for Extensions" [1]. Then add a dependency
to nifi-standard-processors so that I can write a processor extends
MergeContent. When I compiled it and tried to start NiFi, I got the
exact same error that you reported [2].
It indicates that the newly added custom nar "nifi-stdplus-bundle"
made NiFi to load standard processors such as DetectDuplicate from
nifi-stdplus-bundle class loader, because the custom nar has
nifi-standard-processors.jar in it.
DetectDuplicate uses
org/apache/nifi/distributed/cache/client/Serializer, which resides in
another jar, nifi-distributed-cache-client-service-api.jar, which is
not contained in the custom jar.
I assume you were in this situation.
2. Add nar reference from custom nar to standard nar
The reason why nifi-standard-processors can use
nifi-distributed-cache-client-service-api.jar is that
nifi-standard-nar has nar reference to nifi-standard-services-api-nar
(which has Serializer) in its pom.
So, I added the same nar dependency in 'nifi-stdplus-bundle/pom.xml'.
Then NiFi started to launch without the previous error.
However, since standard processors are in two nars (standard and
custom), NiFi shows warning messages [3], and also it increases file
size.
3. Specify 'provided' scope
In order to avoid having standard processors in the custom nar file, I
used 'provided' scope for the nifi-standard-processors dependency in
nifi-stdplus-processors. Please check this commit as an example [4].
Hope this helps.
BTW, I'm interested in why you had to extend MergeContent processor in
the first place. Would you elaborate what was the shortcomings? I
wonder if it can be merged into the project. Having required change in
master branch is the best way to extend (enhance) a standard processor
:)
[1] https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
[2]
2016-09-12 11:07:46,861 ERROR [main] org.apache.nifi.NiFi Failure to
launch NiFi due to java.util.ServiceConfigurationError:
org.apache.nifi.processor.Processor: Provider
org.apache.nifi.processors.standard.DetectDuplicate could not be
instantiated
java.util.ServiceConfigurationError:
org.apache.nifi.processor.Processor: Provider
org.apache.nifi.processors.standard.DetectDuplicate could not be
instantiated
[3]
2016-09-12 12:00:43,981 WARN [main]
org.apache.nifi.nar.ExtensionManager Attempt was made to load
org.apache.nifi.processors.standard.AttributesToJSON from
org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-standard-nar-0.7.0.nar-unpacked]
but that class name is already loaded/registered from
org.apache.nifi.nar.NarClassLoader[./work/nar/extensions/nifi-stdplus-nar-0.7.0.nar-unpacked].
This may cause unpredictable behavior. Order of NARs is not
guaranteed.
[4] https://github.com/ijokarumawak/nifi/commit/134ff5dfc144abe463630ec9d4ab332bd16f22ac
Thanks,
Koji
On Fri, Sep 9, 2016 at 11:25 PM, bsabin <bo...@gmail.com> wrote:
> Hi,
>
> I have a custom processor that extends the MergeContent proc and when NiFi
> starts I have this error in logs:
>
> 2016-09-09 18:17:00,607 ERROR [main] org.apache.nifi.NiFi Failure to launch
> NiFi due to java.util.ServiceConfigurationError:
> org.apache.nifi.processor.Processor: Provider
> org.apache.nifi.processors.standard.DetectDuplicate could not be
> instantiated
> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor:
> Provider org.apache.nifi.processors.standard.DetectDuplicate could not be
> instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> ~[na:1.8.0_101]
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> ~[na:1.8.0_101]
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> ~[na:1.8.0_101]
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> ~[na:1.8.0_101]
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> ~[na:1.8.0_101]
> at
> org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:107)
> ~[nifi-nar-utils-1.0.0.jar:1.0.0]
> at
> org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:88)
> ~[nifi-nar-utils-1.0.0.jar:1.0.0]
> at org.apache.nifi.NiFi.<init>(NiFi.java:135)
> ~[nifi-runtime-1.0.0.jar:1.0.0]
> at org.apache.nifi.NiFi.main(NiFi.java:243)
> ~[nifi-runtime-1.0.0.jar:1.0.0]
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/nifi/distributed/cache/client/Serializer
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> ~[na:1.8.0_101]
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> ~[na:1.8.0_101]
> at java.lang.Class.getConstructor0(Class.java:3075) ~[na:1.8.0_101]
> at java.lang.Class.newInstance(Class.java:412) ~[na:1.8.0_101]
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ~[na:1.8.0_101]
> ... 6 common frames omitted
> Caused by: java.lang.ClassNotFoundException:
> org.apache.nifi.distributed.cache.client.Serializer
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_101]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> ~[na:1.8.0_101]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ~[na:1.8.0_101]
>
>
> On Nifi 0.6.1/ 0.7.0 it worked, but when I've tried to use PutSQL proc, I
> could not see in the property list the Controller Service populated.
> Is there a proper way to extend the behaviour of a standard processor ?
>
> Thanks
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/nifi-1-0-extending-a-standard-processor-tp13306.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.