You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Phil H <gi...@gmail.com> on 2017/11/06 16:58:41 UTC
Classloader issues
Hi guys,
I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
Pulling my hair out here - would love some help!
Cheers,
Phil
Re: Classloader issues
Posted by Bryan Bende <bb...@gmail.com>.
Correct :)
Technically we do support deploying extensions in JARs, but if you did
that you would only be able to utilize JARs that are directly in lib,
and since nifi-utils is not one of those, it explains why your
processor coming from the JAR cannot find StandardValidators.
Long story short, I think removing the JAR and copying in your NAR should work.
-Bryan
On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gi...@gmail.com> wrote:
> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>
>
>
>> On 7 Nov 2017, at 5:17 am, Phil H <gi...@gmail.com> wrote:
>>
>> Thanks Bryan,
>>
>> This new NAR does not appear in the extensions directory (my other working ones do).
>>
>> As for your second question
>>
>> [phil@localhost JSONCondenser]$ ls ~/nifi-1.3.0/lib/ | grep jar
>> javax.servlet-api-3.1.0.jar
>> jcl-over-slf4j-1.7.25.jar
>> jetty-schemas-3.1.jar
>> jul-to-slf4j-1.7.25.jar
>> log4j-over-slf4j-1.7.25.jar
>> logback-classic-1.2.3.jar
>> logback-core-1.2.3.jar
>> nifi-api-1.3.0.jar
>> nifi-framework-api-1.3.0.jar
>> nifi-JSONCondenser-processors-0.1.jar
>> nifi-nar-utils-1.3.0.jar
>> nifi-properties-1.3.0.jar
>> nifi-runtime-1.3.0.jar
>> slf4j-api-1.7.25.jar
>>
>>
>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <bb...@gmail.com> wrote:
>>>
>>> Thanks for the poms.
>>>
>>> Can you provide the output of listing
>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>> ?
>>>
>>> and also the output of listing JARs that are in NiFi's lib directory?
>>> ls -l NIFI_HOME/lib/ | grep jar
>>>
>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>> that no other unexpected JARs are in your lib directory.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <parent>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-nar-bundles</artifactId>
>>>> <version>1.3.0</version>
>>>> </parent>
>>>>
>>>> <repositories>
>>>> <repository>
>>>> <id>project.local</id>
>>>> <name>projects</name>
>>>> <url>file:${project.basedir}/repo</url>
>>>> </repository>
>>>> </repositories>
>>>>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>JSONCondenser</artifactId>
>>>> <version>0.1</version>
>>>> <packaging>pom</packaging>
>>>>
>>>> <modules>
>>>> <module>nifi-JSONCondenser-processors</module>
>>>> <module>nifi-JSONCondenser-nar</module>
>>>> </modules>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.json</groupId>
>>>> <artifactId>JSON</artifactId>
>>>> <version>1.0</version>
>>>> </dependency>
>>>> </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <parent>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>JSONCondenser</artifactId>
>>>> <version>0.1</version>
>>>> </parent>
>>>>
>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>> <packaging>jar</packaging>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-api</artifactId>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-utils</artifactId>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-mock</artifactId>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.slf4j</groupId>
>>>> <artifactId>slf4j-simple</artifactId>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>junit</groupId>
>>>> <artifactId>junit</artifactId>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </project>
>>>>
>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <parent>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>JSONCondenser</artifactId>
>>>> <version>0.1</version>
>>>> </parent>
>>>>
>>>> <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>> <version>0.1</version>
>>>> <packaging>nar</packaging>
>>>> <properties>
>>>> <maven.javadoc.skip>true</maven.javadoc.skip>
>>>> <source.skip>true</source.skip>
>>>> </properties>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>> <version>0.1</version>
>>>> </dependency>
>>>> </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>>
>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>
>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>> and bundle?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Bryan
>>>>>
>>>>>
>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>
>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>>>>
>>>>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>
>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>> .Builder().name("ID_PATH")
>>>>>>> .displayName("JSON ID Path")
>>>>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>> .required(true)
>>>>>>> .build();
>>>>>>>
>>>>>>>
>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>> at java.lang.Class.newInstance(Class.java:442)
>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>> ... 7 common frames omitted
>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>> ... 14 common frames omitted
>>>>>>>
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>
>>>>>>>> What version are you on?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>>>>> Hi guys,
>>>>>>>>>
>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>
>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>
>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>
>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Phil
>>>>>>>
>>>>>>
>>>>
>>
>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
While we're talking about this, is there a simple way to have maven put the NAR file into my NiFi lib directory automatically as part of the "mvn install" script?
Cheers,
Phil
> On 7 Nov 2017, at 05:22, Joe Witt <jo...@gmail.com> wrote:
>
> Phil - you are the first person to ever make that mistake.
>
> Nope, not true :) No worries and thanks for following up. I believe
> once we have the extension registry in play this sort of thing will be
> much easier/simpler.
>
> Thanks
>
>> On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gi...@gmail.com> wrote:
>> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>>
>>
>>
>>> On 7 Nov 2017, at 5:17 am, Phil H <gi...@gmail.com> wrote:
>>>
>>> Thanks Bryan,
>>>
>>> This new NAR does not appear in the extensions directory (my other working ones do).
>>>
>>> As for your second question
>>>
>>> [phil@localhost JSONCondenser]$ ls ~/nifi-1.3.0/lib/ | grep jar
>>> javax.servlet-api-3.1.0.jar
>>> jcl-over-slf4j-1.7.25.jar
>>> jetty-schemas-3.1.jar
>>> jul-to-slf4j-1.7.25.jar
>>> log4j-over-slf4j-1.7.25.jar
>>> logback-classic-1.2.3.jar
>>> logback-core-1.2.3.jar
>>> nifi-api-1.3.0.jar
>>> nifi-framework-api-1.3.0.jar
>>> nifi-JSONCondenser-processors-0.1.jar
>>> nifi-nar-utils-1.3.0.jar
>>> nifi-properties-1.3.0.jar
>>> nifi-runtime-1.3.0.jar
>>> slf4j-api-1.7.25.jar
>>>
>>>
>>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>
>>>> Thanks for the poms.
>>>>
>>>> Can you provide the output of listing
>>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>>> ?
>>>>
>>>> and also the output of listing JARs that are in NiFi's lib directory?
>>>> ls -l NIFI_HOME/lib/ | grep jar
>>>>
>>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>>> that no other unexpected JARs are in your lib directory.
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
>>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>>
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <parent>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-nar-bundles</artifactId>
>>>>> <version>1.3.0</version>
>>>>> </parent>
>>>>>
>>>>> <repositories>
>>>>> <repository>
>>>>> <id>project.local</id>
>>>>> <name>projects</name>
>>>>> <url>file:${project.basedir}/repo</url>
>>>>> </repository>
>>>>> </repositories>
>>>>>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>JSONCondenser</artifactId>
>>>>> <version>0.1</version>
>>>>> <packaging>pom</packaging>
>>>>>
>>>>> <modules>
>>>>> <module>nifi-JSONCondenser-processors</module>
>>>>> <module>nifi-JSONCondenser-nar</module>
>>>>> </modules>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.json</groupId>
>>>>> <artifactId>JSON</artifactId>
>>>>> <version>1.0</version>
>>>>> </dependency>
>>>>> </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <parent>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>JSONCondenser</artifactId>
>>>>> <version>0.1</version>
>>>>> </parent>
>>>>>
>>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>> <packaging>jar</packaging>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-api</artifactId>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-utils</artifactId>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-mock</artifactId>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.slf4j</groupId>
>>>>> <artifactId>slf4j-simple</artifactId>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>junit</groupId>
>>>>> <artifactId>junit</artifactId>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </project>
>>>>>
>>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <parent>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>JSONCondenser</artifactId>
>>>>> <version>0.1</version>
>>>>> </parent>
>>>>>
>>>>> <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>> <version>0.1</version>
>>>>> <packaging>nar</packaging>
>>>>> <properties>
>>>>> <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>> <source.skip>true</source.skip>
>>>>> </properties>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>> <version>0.1</version>
>>>>> </dependency>
>>>>> </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>>>
>>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>>
>>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>>> and bundle?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Bryan
>>>>>>
>>>>>>
>>>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>>
>>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>> .Builder().name("ID_PATH")
>>>>>>>> .displayName("JSON ID Path")
>>>>>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>> .required(true)
>>>>>>>> .build();
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>> at java.lang.Class.newInstance(Class.java:442)
>>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>> ... 7 common frames omitted
>>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>> ... 14 common frames omitted
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>>
>>>>>>>>> What version are you on?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>>
>>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>>
>>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>>
>>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Phil
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
Yep, replacing the JAR with the NAR (and then re-connecting it through the UI) worked. As a nice postscript to the story, the previously untested filtering algorithm i had written worked as expected first time too :-)
Thanks so much for your help, team!
> On 7 Nov 2017, at 5:22 am, Joe Witt <jo...@gmail.com> wrote:
>
> Phil - you are the first person to ever make that mistake.
>
> Nope, not true :) No worries and thanks for following up. I believe
> once we have the extension registry in play this sort of thing will be
> much easier/simpler.
>
> Thanks
>
> On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gi...@gmail.com> wrote:
>> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>>
>>
>>
>>> On 7 Nov 2017, at 5:17 am, Phil H <gi...@gmail.com> wrote:
>>>
>>> Thanks Bryan,
>>>
>>> This new NAR does not appear in the extensions directory (my other working ones do).
>>>
>>> As for your second question
>>>
>>> [phil@localhost JSONCondenser]$ ls ~/nifi-1.3.0/lib/ | grep jar
>>> javax.servlet-api-3.1.0.jar
>>> jcl-over-slf4j-1.7.25.jar
>>> jetty-schemas-3.1.jar
>>> jul-to-slf4j-1.7.25.jar
>>> log4j-over-slf4j-1.7.25.jar
>>> logback-classic-1.2.3.jar
>>> logback-core-1.2.3.jar
>>> nifi-api-1.3.0.jar
>>> nifi-framework-api-1.3.0.jar
>>> nifi-JSONCondenser-processors-0.1.jar
>>> nifi-nar-utils-1.3.0.jar
>>> nifi-properties-1.3.0.jar
>>> nifi-runtime-1.3.0.jar
>>> slf4j-api-1.7.25.jar
>>>
>>>
>>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>
>>>> Thanks for the poms.
>>>>
>>>> Can you provide the output of listing
>>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>>> ?
>>>>
>>>> and also the output of listing JARs that are in NiFi's lib directory?
>>>> ls -l NIFI_HOME/lib/ | grep jar
>>>>
>>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>>> that no other unexpected JARs are in your lib directory.
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
>>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>>
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <parent>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-nar-bundles</artifactId>
>>>>> <version>1.3.0</version>
>>>>> </parent>
>>>>>
>>>>> <repositories>
>>>>> <repository>
>>>>> <id>project.local</id>
>>>>> <name>projects</name>
>>>>> <url>file:${project.basedir}/repo</url>
>>>>> </repository>
>>>>> </repositories>
>>>>>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>JSONCondenser</artifactId>
>>>>> <version>0.1</version>
>>>>> <packaging>pom</packaging>
>>>>>
>>>>> <modules>
>>>>> <module>nifi-JSONCondenser-processors</module>
>>>>> <module>nifi-JSONCondenser-nar</module>
>>>>> </modules>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.json</groupId>
>>>>> <artifactId>JSON</artifactId>
>>>>> <version>1.0</version>
>>>>> </dependency>
>>>>> </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <parent>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>JSONCondenser</artifactId>
>>>>> <version>0.1</version>
>>>>> </parent>
>>>>>
>>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>> <packaging>jar</packaging>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-api</artifactId>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-utils</artifactId>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.apache.nifi</groupId>
>>>>> <artifactId>nifi-mock</artifactId>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.slf4j</groupId>
>>>>> <artifactId>slf4j-simple</artifactId>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>junit</groupId>
>>>>> <artifactId>junit</artifactId>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </project>
>>>>>
>>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <parent>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>JSONCondenser</artifactId>
>>>>> <version>0.1</version>
>>>>> </parent>
>>>>>
>>>>> <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>> <version>0.1</version>
>>>>> <packaging>nar</packaging>
>>>>> <properties>
>>>>> <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>> <source.skip>true</source.skip>
>>>>> </properties>
>>>>>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>com.jidmu</groupId>
>>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>> <version>0.1</version>
>>>>> </dependency>
>>>>> </dependencies>
>>>>>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>>>
>>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>>
>>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>>> and bundle?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Bryan
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>>
>>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>> .Builder().name("ID_PATH")
>>>>>>>> .displayName("JSON ID Path")
>>>>>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>> .required(true)
>>>>>>>> .build();
>>>>>>>>
>>>>>>>>
>>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>> at java.lang.Class.newInstance(Class.java:442)
>>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>> ... 7 common frames omitted
>>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>> ... 14 common frames omitted
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>>
>>>>>>>>> What version are you on?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>>
>>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>>
>>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>>
>>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Phil
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>
Re: Classloader issues
Posted by Joe Witt <jo...@gmail.com>.
Phil - you are the first person to ever make that mistake.
Nope, not true :) No worries and thanks for following up. I believe
once we have the extension registry in play this sort of thing will be
much easier/simpler.
Thanks
On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gi...@gmail.com> wrote:
> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
>
>
>
>> On 7 Nov 2017, at 5:17 am, Phil H <gi...@gmail.com> wrote:
>>
>> Thanks Bryan,
>>
>> This new NAR does not appear in the extensions directory (my other working ones do).
>>
>> As for your second question
>>
>> [phil@localhost JSONCondenser]$ ls ~/nifi-1.3.0/lib/ | grep jar
>> javax.servlet-api-3.1.0.jar
>> jcl-over-slf4j-1.7.25.jar
>> jetty-schemas-3.1.jar
>> jul-to-slf4j-1.7.25.jar
>> log4j-over-slf4j-1.7.25.jar
>> logback-classic-1.2.3.jar
>> logback-core-1.2.3.jar
>> nifi-api-1.3.0.jar
>> nifi-framework-api-1.3.0.jar
>> nifi-JSONCondenser-processors-0.1.jar
>> nifi-nar-utils-1.3.0.jar
>> nifi-properties-1.3.0.jar
>> nifi-runtime-1.3.0.jar
>> slf4j-api-1.7.25.jar
>>
>>
>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <bb...@gmail.com> wrote:
>>>
>>> Thanks for the poms.
>>>
>>> Can you provide the output of listing
>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>> ?
>>>
>>> and also the output of listing JARs that are in NiFi's lib directory?
>>> ls -l NIFI_HOME/lib/ | grep jar
>>>
>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>> that no other unexpected JARs are in your lib directory.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>>
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <parent>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-nar-bundles</artifactId>
>>>> <version>1.3.0</version>
>>>> </parent>
>>>>
>>>> <repositories>
>>>> <repository>
>>>> <id>project.local</id>
>>>> <name>projects</name>
>>>> <url>file:${project.basedir}/repo</url>
>>>> </repository>
>>>> </repositories>
>>>>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>JSONCondenser</artifactId>
>>>> <version>0.1</version>
>>>> <packaging>pom</packaging>
>>>>
>>>> <modules>
>>>> <module>nifi-JSONCondenser-processors</module>
>>>> <module>nifi-JSONCondenser-nar</module>
>>>> </modules>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.json</groupId>
>>>> <artifactId>JSON</artifactId>
>>>> <version>1.0</version>
>>>> </dependency>
>>>> </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <parent>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>JSONCondenser</artifactId>
>>>> <version>0.1</version>
>>>> </parent>
>>>>
>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>> <packaging>jar</packaging>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-api</artifactId>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-utils</artifactId>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.nifi</groupId>
>>>> <artifactId>nifi-mock</artifactId>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.slf4j</groupId>
>>>> <artifactId>slf4j-simple</artifactId>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>junit</groupId>
>>>> <artifactId>junit</artifactId>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </project>
>>>>
>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <parent>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>JSONCondenser</artifactId>
>>>> <version>0.1</version>
>>>> </parent>
>>>>
>>>> <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>> <version>0.1</version>
>>>> <packaging>nar</packaging>
>>>> <properties>
>>>> <maven.javadoc.skip>true</maven.javadoc.skip>
>>>> <source.skip>true</source.skip>
>>>> </properties>
>>>>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>com.jidmu</groupId>
>>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>> <version>0.1</version>
>>>> </dependency>
>>>> </dependencies>
>>>>
>>>> </project>
>>>>
>>>>
>>>>
>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>>
>>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>>
>>>>> Can you share your project, or the pom files for the processors, NAR,
>>>>> and bundle?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Bryan
>>>>>
>>>>>
>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>>
>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>>>>
>>>>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>>
>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>> .Builder().name("ID_PATH")
>>>>>>> .displayName("JSON ID Path")
>>>>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>> .required(true)
>>>>>>> .build();
>>>>>>>
>>>>>>>
>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>> at java.lang.Class.newInstance(Class.java:442)
>>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>> ... 7 common frames omitted
>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>> ... 14 common frames omitted
>>>>>>>
>>>>>>>
>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>
>>>>>>>> What version are you on?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>>>>> Hi guys,
>>>>>>>>>
>>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>
>>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>>
>>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>>
>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Phil
>>>>>>>
>>>>>>
>>>>
>>
>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather than the NAR file
> On 7 Nov 2017, at 5:17 am, Phil H <gi...@gmail.com> wrote:
>
> Thanks Bryan,
>
> This new NAR does not appear in the extensions directory (my other working ones do).
>
> As for your second question
>
> [phil@localhost JSONCondenser]$ ls ~/nifi-1.3.0/lib/ | grep jar
> javax.servlet-api-3.1.0.jar
> jcl-over-slf4j-1.7.25.jar
> jetty-schemas-3.1.jar
> jul-to-slf4j-1.7.25.jar
> log4j-over-slf4j-1.7.25.jar
> logback-classic-1.2.3.jar
> logback-core-1.2.3.jar
> nifi-api-1.3.0.jar
> nifi-framework-api-1.3.0.jar
> nifi-JSONCondenser-processors-0.1.jar
> nifi-nar-utils-1.3.0.jar
> nifi-properties-1.3.0.jar
> nifi-runtime-1.3.0.jar
> slf4j-api-1.7.25.jar
>
>
>> On 7 Nov 2017, at 5:13 am, Bryan Bende <bb...@gmail.com> wrote:
>>
>> Thanks for the poms.
>>
>> Can you provide the output of listing
>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>> ?
>>
>> and also the output of listing JARs that are in NiFi's lib directory?
>> ls -l NIFI_HOME/lib/ | grep jar
>>
>> Want to verify that nifi-utils JAR is actually in your NAR and also
>> that no other unexpected JARs are in your lib directory.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
>>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>>
>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>> <modelVersion>4.0.0</modelVersion>
>>>
>>> <parent>
>>> <groupId>org.apache.nifi</groupId>
>>> <artifactId>nifi-nar-bundles</artifactId>
>>> <version>1.3.0</version>
>>> </parent>
>>>
>>> <repositories>
>>> <repository>
>>> <id>project.local</id>
>>> <name>projects</name>
>>> <url>file:${project.basedir}/repo</url>
>>> </repository>
>>> </repositories>
>>>
>>> <groupId>com.jidmu</groupId>
>>> <artifactId>JSONCondenser</artifactId>
>>> <version>0.1</version>
>>> <packaging>pom</packaging>
>>>
>>> <modules>
>>> <module>nifi-JSONCondenser-processors</module>
>>> <module>nifi-JSONCondenser-nar</module>
>>> </modules>
>>>
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.json</groupId>
>>> <artifactId>JSON</artifactId>
>>> <version>1.0</version>
>>> </dependency>
>>> </dependencies>
>>>
>>> </project>
>>>
>>>
>>>
>>> ./nifi-JSONCondenser-processors/pom.xml
>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>> <modelVersion>4.0.0</modelVersion>
>>>
>>> <parent>
>>> <groupId>com.jidmu</groupId>
>>> <artifactId>JSONCondenser</artifactId>
>>> <version>0.1</version>
>>> </parent>
>>>
>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>> <packaging>jar</packaging>
>>>
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.nifi</groupId>
>>> <artifactId>nifi-api</artifactId>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.nifi</groupId>
>>> <artifactId>nifi-utils</artifactId>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.nifi</groupId>
>>> <artifactId>nifi-mock</artifactId>
>>> <scope>test</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.slf4j</groupId>
>>> <artifactId>slf4j-simple</artifactId>
>>> <scope>test</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>junit</groupId>
>>> <artifactId>junit</artifactId>
>>> <scope>test</scope>
>>> </dependency>
>>> </dependencies>
>>> </project>
>>>
>>> ./nifi-JSONCondenser-nar/pom.xml
>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>> <modelVersion>4.0.0</modelVersion>
>>>
>>> <parent>
>>> <groupId>com.jidmu</groupId>
>>> <artifactId>JSONCondenser</artifactId>
>>> <version>0.1</version>
>>> </parent>
>>>
>>> <artifactId>nifi-JSONCondenser-nar</artifactId>
>>> <version>0.1</version>
>>> <packaging>nar</packaging>
>>> <properties>
>>> <maven.javadoc.skip>true</maven.javadoc.skip>
>>> <source.skip>true</source.skip>
>>> </properties>
>>>
>>> <dependencies>
>>> <dependency>
>>> <groupId>com.jidmu</groupId>
>>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>>> <version>0.1</version>
>>> </dependency>
>>> </dependencies>
>>>
>>> </project>
>>>
>>>
>>>
>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>>>
>>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>>
>>>> Can you share your project, or the pom files for the processors, NAR,
>>>> and bundle?
>>>>
>>>> Thanks,
>>>>
>>>> Bryan
>>>>
>>>>
>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>>
>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>>>
>>>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>>
>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>> .Builder().name("ID_PATH")
>>>>>> .displayName("JSON ID Path")
>>>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>> .required(true)
>>>>>> .build();
>>>>>>
>>>>>>
>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>> at java.lang.Class.newInstance(Class.java:442)
>>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>> ... 7 common frames omitted
>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>> ... 14 common frames omitted
>>>>>>
>>>>>>
>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>>>
>>>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>>> goes through to see where it goes wonky.
>>>>>>>
>>>>>>> What version are you on?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>>>> Hi guys,
>>>>>>>>
>>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>
>>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>>
>>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>>
>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Phil
>>>>>>
>>>>>
>>>
>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
Thanks Bryan,
This new NAR does not appear in the extensions directory (my other working ones do).
As for your second question
[phil@localhost JSONCondenser]$ ls ~/nifi-1.3.0/lib/ | grep jar
javax.servlet-api-3.1.0.jar
jcl-over-slf4j-1.7.25.jar
jetty-schemas-3.1.jar
jul-to-slf4j-1.7.25.jar
log4j-over-slf4j-1.7.25.jar
logback-classic-1.2.3.jar
logback-core-1.2.3.jar
nifi-api-1.3.0.jar
nifi-framework-api-1.3.0.jar
nifi-JSONCondenser-processors-0.1.jar
nifi-nar-utils-1.3.0.jar
nifi-properties-1.3.0.jar
nifi-runtime-1.3.0.jar
slf4j-api-1.7.25.jar
> On 7 Nov 2017, at 5:13 am, Bryan Bende <bb...@gmail.com> wrote:
>
> Thanks for the poms.
>
> Can you provide the output of listing
> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
> ?
>
> and also the output of listing JARs that are in NiFi's lib directory?
> ls -l NIFI_HOME/lib/ | grep jar
>
> Want to verify that nifi-utils JAR is actually in your NAR and also
> that no other unexpected JARs are in your lib directory.
>
> Thanks,
>
> Bryan
>
> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
>> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>>
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>> <modelVersion>4.0.0</modelVersion>
>>
>> <parent>
>> <groupId>org.apache.nifi</groupId>
>> <artifactId>nifi-nar-bundles</artifactId>
>> <version>1.3.0</version>
>> </parent>
>>
>> <repositories>
>> <repository>
>> <id>project.local</id>
>> <name>projects</name>
>> <url>file:${project.basedir}/repo</url>
>> </repository>
>> </repositories>
>>
>> <groupId>com.jidmu</groupId>
>> <artifactId>JSONCondenser</artifactId>
>> <version>0.1</version>
>> <packaging>pom</packaging>
>>
>> <modules>
>> <module>nifi-JSONCondenser-processors</module>
>> <module>nifi-JSONCondenser-nar</module>
>> </modules>
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.json</groupId>
>> <artifactId>JSON</artifactId>
>> <version>1.0</version>
>> </dependency>
>> </dependencies>
>>
>> </project>
>>
>>
>>
>> ./nifi-JSONCondenser-processors/pom.xml
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>> <modelVersion>4.0.0</modelVersion>
>>
>> <parent>
>> <groupId>com.jidmu</groupId>
>> <artifactId>JSONCondenser</artifactId>
>> <version>0.1</version>
>> </parent>
>>
>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>> <packaging>jar</packaging>
>>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.nifi</groupId>
>> <artifactId>nifi-api</artifactId>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.nifi</groupId>
>> <artifactId>nifi-utils</artifactId>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.nifi</groupId>
>> <artifactId>nifi-mock</artifactId>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.slf4j</groupId>
>> <artifactId>slf4j-simple</artifactId>
>> <scope>test</scope>
>> </dependency>
>> <dependency>
>> <groupId>junit</groupId>
>> <artifactId>junit</artifactId>
>> <scope>test</scope>
>> </dependency>
>> </dependencies>
>> </project>
>>
>> ./nifi-JSONCondenser-nar/pom.xml
>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>> <modelVersion>4.0.0</modelVersion>
>>
>> <parent>
>> <groupId>com.jidmu</groupId>
>> <artifactId>JSONCondenser</artifactId>
>> <version>0.1</version>
>> </parent>
>>
>> <artifactId>nifi-JSONCondenser-nar</artifactId>
>> <version>0.1</version>
>> <packaging>nar</packaging>
>> <properties>
>> <maven.javadoc.skip>true</maven.javadoc.skip>
>> <source.skip>true</source.skip>
>> </properties>
>>
>> <dependencies>
>> <dependency>
>> <groupId>com.jidmu</groupId>
>> <artifactId>nifi-JSONCondenser-processors</artifactId>
>> <version>0.1</version>
>> </dependency>
>> </dependencies>
>>
>> </project>
>>
>>
>>
>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>>
>>> It is most likely an issue with the Maven configuration in one of your modules.
>>>
>>> Can you share your project, or the pom files for the processors, NAR,
>>> and bundle?
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>>
>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>>
>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>>
>>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>>
>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>> .Builder().name("ID_PATH")
>>>>> .displayName("JSON ID Path")
>>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>> .required(true)
>>>>> .build();
>>>>>
>>>>>
>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>> at java.lang.Class.newInstance(Class.java:442)
>>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>> ... 7 common frames omitted
>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>> ... 14 common frames omitted
>>>>>
>>>>>
>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>>
>>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>>> goes through to see where it goes wonky.
>>>>>>
>>>>>> What version are you on?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>
>>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>>
>>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>>
>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Phil
>>>>>
>>>>
>>
Re: Classloader issues
Posted by Bryan Bende <bb...@gmail.com>.
Thanks for the poms.
Can you provide the output of listing
NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
?
and also the output of listing JARs that are in NiFi's lib directory?
ls -l NIFI_HOME/lib/ | grep jar
Want to verify that nifi-utils JAR is actually in your NAR and also
that no other unexpected JARs are in your lib directory.
Thanks,
Bryan
On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gi...@gmail.com> wrote:
> ./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
>
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
> <modelVersion>4.0.0</modelVersion>
>
> <parent>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-nar-bundles</artifactId>
> <version>1.3.0</version>
> </parent>
>
> <repositories>
> <repository>
> <id>project.local</id>
> <name>projects</name>
> <url>file:${project.basedir}/repo</url>
> </repository>
> </repositories>
>
> <groupId>com.jidmu</groupId>
> <artifactId>JSONCondenser</artifactId>
> <version>0.1</version>
> <packaging>pom</packaging>
>
> <modules>
> <module>nifi-JSONCondenser-processors</module>
> <module>nifi-JSONCondenser-nar</module>
> </modules>
>
> <dependencies>
> <dependency>
> <groupId>org.json</groupId>
> <artifactId>JSON</artifactId>
> <version>1.0</version>
> </dependency>
> </dependencies>
>
> </project>
>
>
>
> ./nifi-JSONCondenser-processors/pom.xml
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
> <modelVersion>4.0.0</modelVersion>
>
> <parent>
> <groupId>com.jidmu</groupId>
> <artifactId>JSONCondenser</artifactId>
> <version>0.1</version>
> </parent>
>
> <artifactId>nifi-JSONCondenser-processors</artifactId>
> <packaging>jar</packaging>
>
> <dependencies>
> <dependency>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-api</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-utils</artifactId>
> </dependency>
> <dependency>
> <groupId>org.apache.nifi</groupId>
> <artifactId>nifi-mock</artifactId>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-simple</artifactId>
> <scope>test</scope>
> </dependency>
> <dependency>
> <groupId>junit</groupId>
> <artifactId>junit</artifactId>
> <scope>test</scope>
> </dependency>
> </dependencies>
> </project>
>
> ./nifi-JSONCondenser-nar/pom.xml
> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
> <modelVersion>4.0.0</modelVersion>
>
> <parent>
> <groupId>com.jidmu</groupId>
> <artifactId>JSONCondenser</artifactId>
> <version>0.1</version>
> </parent>
>
> <artifactId>nifi-JSONCondenser-nar</artifactId>
> <version>0.1</version>
> <packaging>nar</packaging>
> <properties>
> <maven.javadoc.skip>true</maven.javadoc.skip>
> <source.skip>true</source.skip>
> </properties>
>
> <dependencies>
> <dependency>
> <groupId>com.jidmu</groupId>
> <artifactId>nifi-JSONCondenser-processors</artifactId>
> <version>0.1</version>
> </dependency>
> </dependencies>
>
> </project>
>
>
>
>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>>
>> It is most likely an issue with the Maven configuration in one of your modules.
>>
>> Can you share your project, or the pom files for the processors, NAR,
>> and bundle?
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>>
>>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>>
>>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>>
>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>> .Builder().name("ID_PATH")
>>>> .displayName("JSON ID Path")
>>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>> .required(true)
>>>> .build();
>>>>
>>>>
>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>> at java.lang.Class.newInstance(Class.java:442)
>>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>> ... 7 common frames omitted
>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>> ... 14 common frames omitted
>>>>
>>>>
>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>>
>>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>>> to add a debugger at runtime and examine the context classloader as it
>>>>> goes through to see where it goes wonky.
>>>>>
>>>>> What version are you on?
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>
>>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>>
>>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>>
>>>>>> Pulling my hair out here - would love some help!
>>>>>>
>>>>>> Cheers,
>>>>>> Phil
>>>>
>>>
>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
./pom.xml (Note that this issue occurred BEFORE I added the external JAR dependency, and I have that exact same org.json dependency in a bunch of other processors I have written without issue)
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-bundles</artifactId>
<version>1.3.0</version>
</parent>
<repositories>
<repository>
<id>project.local</id>
<name>projects</name>
<url>file:${project.basedir}/repo</url>
</repository>
</repositories>
<groupId>com.jidmu</groupId>
<artifactId>JSONCondenser</artifactId>
<version>0.1</version>
<packaging>pom</packaging>
<modules>
<module>nifi-JSONCondenser-processors</module>
<module>nifi-JSONCondenser-nar</module>
</modules>
<dependencies>
<dependency>
<groupId>org.json</groupId>
<artifactId>JSON</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</project>
./nifi-JSONCondenser-processors/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.jidmu</groupId>
<artifactId>JSONCondenser</artifactId>
<version>0.1</version>
</parent>
<artifactId>nifi-JSONCondenser-processors</artifactId>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-utils</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mock</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
./nifi-JSONCondenser-nar/pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.jidmu</groupId>
<artifactId>JSONCondenser</artifactId>
<version>0.1</version>
</parent>
<artifactId>nifi-JSONCondenser-nar</artifactId>
<version>0.1</version>
<packaging>nar</packaging>
<properties>
<maven.javadoc.skip>true</maven.javadoc.skip>
<source.skip>true</source.skip>
</properties>
<dependencies>
<dependency>
<groupId>com.jidmu</groupId>
<artifactId>nifi-JSONCondenser-processors</artifactId>
<version>0.1</version>
</dependency>
</dependencies>
</project>
> On 7 Nov 2017, at 4:39 am, Bryan Bende <bb...@gmail.com> wrote:
>
> It is most likely an issue with the Maven configuration in one of your modules.
>
> Can you share your project, or the pom files for the processors, NAR,
> and bundle?
>
> Thanks,
>
> Bryan
>
>
> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>>
>>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>>
>>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>>
>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>> .Builder().name("ID_PATH")
>>> .displayName("JSON ID Path")
>>> .description("The path to the JSON attribute that represents the unique ID for the object")
>>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>> .required(true)
>>> .build();
>>>
>>>
>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>> at java.lang.Class.newInstance(Class.java:442)
>>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>> ... 7 common frames omitted
>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>> ... 14 common frames omitted
>>>
>>>
>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>>
>>>> Can you share the code by chance for a review? Otherwise, you'll want
>>>> to add a debugger at runtime and examine the context classloader as it
>>>> goes through to see where it goes wonky.
>>>>
>>>> What version are you on?
>>>>
>>>> Thanks
>>>>
>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>>> Hi guys,
>>>>>
>>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>
>>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>>
>>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>>
>>>>> Pulling my hair out here - would love some help!
>>>>>
>>>>> Cheers,
>>>>> Phil
>>>
>>
Re: Classloader issues
Posted by Bryan Bende <bb...@gmail.com>.
It is most likely an issue with the Maven configuration in one of your modules.
Can you share your project, or the pom files for the processors, NAR,
and bundle?
Thanks,
Bryan
On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gi...@gmail.com> wrote:
> Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
>
>> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>>
>> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>>
>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>> .Builder().name("ID_PATH")
>> .displayName("JSON ID Path")
>> .description("The path to the JSON attribute that represents the unique ID for the object")
>> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>> .required(true)
>> .build();
>>
>>
>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>> at org.apache.nifi.NiFi.main(NiFi.java:267)
>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>> at java.lang.Class.newInstance(Class.java:442)
>> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>> ... 7 common frames omitted
>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> ... 14 common frames omitted
>>
>>
>>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>>
>>> Can you share the code by chance for a review? Otherwise, you'll want
>>> to add a debugger at runtime and examine the context classloader as it
>>> goes through to see where it goes wonky.
>>>
>>> What version are you on?
>>>
>>> Thanks
>>>
>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>>> Hi guys,
>>>>
>>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>
>>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>>
>>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>>
>>>> Pulling my hair out here - would love some help!
>>>>
>>>> Cheers,
>>>> Phil
>>
>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
Nifi version is 1.3.0, running on Java 1.8.0_131, running on a out-of-the-box CentOS (if that’s relevant)
> On 7 Nov 2017, at 4:17 am, Phil H <gi...@gmail.com> wrote:
>
> I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
>
> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
> .Builder().name("ID_PATH")
> .displayName("JSON ID Path")
> .description("The path to the JSON attribute that represents the unique ID for the object")
> .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
> .required(true)
> .build();
>
>
> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
> at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
> at org.apache.nifi.NiFi.main(NiFi.java:267)
> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
> at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at java.lang.Class.newInstance(Class.java:442)
> at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 7 common frames omitted
> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 14 common frames omitted
>
>
>> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>>
>> Can you share the code by chance for a review? Otherwise, you'll want
>> to add a debugger at runtime and examine the context classloader as it
>> goes through to see where it goes wonky.
>>
>> What version are you on?
>>
>> Thanks
>>
>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>>> Hi guys,
>>>
>>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>
>>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>>
>>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>>
>>> Pulling my hair out here - would love some help!
>>>
>>> Cheers,
>>> Phil
>
Re: Classloader issues
Posted by Phil H <gi...@gmail.com>.
I added the StandardValidators reference back in and the original error reoccurs. The offending code (which compiles fine using the same JDK) is totally standard stuff:
public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
.Builder().name("ID_PATH")
.displayName("JSON ID Path")
.description("The path to the JSON attribute that represents the unique ID for the object")
.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
.required(true)
.build();
2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
at org.apache.nifi.NiFi.<init>(NiFi.java:160)
at org.apache.nifi.NiFi.main(NiFi.java:267)
Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
... 7 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 14 common frames omitted
> On 7 Nov 2017, at 4:02 am, Joe Witt <jo...@gmail.com> wrote:
>
> Can you share the code by chance for a review? Otherwise, you'll want
> to add a debugger at runtime and examine the context classloader as it
> goes through to see where it goes wonky.
>
> What version are you on?
>
> Thanks
>
> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
>> Hi guys,
>>
>> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>>
>> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>>
>> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>>
>> Pulling my hair out here - would love some help!
>>
>> Cheers,
>> Phil
Re: Classloader issues
Posted by Joe Witt <jo...@gmail.com>.
Can you share the code by chance for a review? Otherwise, you'll want
to add a debugger at runtime and examine the context classloader as it
goes through to see where it goes wonky.
What version are you on?
Thanks
On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
> Hi guys,
>
> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>
> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>
> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>
> Pulling my hair out here - would love some help!
>
> Cheers,
> Phil
Re: Classloader issues
Posted by Matt Burgess <ma...@apache.org>.
Phil,
Any chance you could share your NAR and processor POMs? Your processor
POM should have a dependency on nifi-utils. What are the other classes
that aren't found? I'm curious if they are in the same JAR/NAR or not.
Do you have a parent NAR set (in your bundle's NAR POM)? Also once
your NAR is built, does it appear to have the correct contents? You
can check with "jar tvf /path/to/my/nifi-bundle.nar"
Regards,
Matt
On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gi...@gmail.com> wrote:
> Hi guys,
>
> I have just (today) started having issues with a new processor I've written where a seemingly random class (e.g.: StandardValidators) fails to load when NiFi is initializing. I've created this processor like all my others (using the maven archetype) and the VM I'm running it on has not changed versions of any software.
>
> I removed any reference to StandardValidators, and the NAR then loaded successfully. As I added some more functionality, I then had the same ClassNotFoundException occur with a different random class.
>
> Because I have used the maven archetype, all my other processors obviously use StandardValidators, so it's not like the class itself is somehow corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing as this ClassNotFoundException).
>
> Pulling my hair out here - would love some help!
>
> Cheers,
> Phil