You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Scott Wagner <sw...@beenverified.com> on 2017/07/05 15:53:14 UTC
Custom NAR interfering with BundleUtils.findBundleForType
Hey all,
I have been running on NiFi 1.1.1 for a while, and just started
working on migrating to 1.3.0. However, when I tried starting up, I was
getting unhandled exceptions during startup of Jetty. I have traced
this down to something being wrong with a custom NAR file that we built
for 1.1.1.
The problem is when BundleUtils.findBundleForType is being called,
it is finding 2 bundles for every internal NiFi processor class - the
correct NAR that it comes with in 1.3.0, and my custom NAR file as
well. I'm sure that I have done something wrong in the packaging for my
custom NAR file, but I don't know what it could be. It was originally
built from the NiFi NAR archetype late last year.
If anyone can point in me in the direction of what I'm doing wrong,
I would greatly appreciate it. I'm not a maven/packaging expert by any
means so no advice is too basic.
Thanks!
- Scott
Re: Custom NAR interfering with BundleUtils.findBundleForType
Posted by Scott Wagner <sw...@beenverified.com>.
To provide closure to this thread:
Bryan, the exception indeed was being thrown there, but the root cause
of that was because "Ghost" processors were being created in the place
of the actual processors because my custom NAR was confusing
BuildUtils.findBundleForType.
Joe, thank you very much for the suggestion of what to look into. We
had copied most of the InvokeHTTP processor into our NAR with some
modifications, but it was still including references to classes in
nifi-standard-processors which was being dragged in through the pom. I
was able to extricate the 2 classes I need and copy them into my own
project to at least get NiFi up and running; while this is not the
correct permanent solution, I was able to move past the startup problems.
Thanks for being part of such a great and responsive community.
- Scott
> Bryan Bende <ma...@gmail.com>
> Wednesday, July 5, 2017 12:07 PM
> Scott,
>
> Thanks for providing the stacktrace... do any of your custom
> processors use the @DefaultSchedule annotation? and if so, do any of
> them set the tasks to a number less than 1?
>
> The exception you are getting is from some code that is preventing
> using 0 or negative number of tasks for a processor that is not
> scheduled for event driven, basically meaning event driven is the only
> one where it would make sense to have less than 1 task.
>
> What Joe mentioned about including other standard processors in your
> NAR could very well still be a problem, but that stacktrace might be
> something else.
>
> -Bryan
>
> Scott Wagner <ma...@beenverified.com>
> Wednesday, July 5, 2017 11:44 AM
> Hi Joe,
>
> We are extending AbstractProcessor for our processors, and
> AbstractControllerService for our controller service. However, we did
> include the InvokeHTTP processor with some modifications that are
> referencing some other classes that are in the
> nifi-processors-standard JAR. I will look into breaking those out to
> remove that dependency.
>
> The actual error that we are getting is below:
>
> 2017-07-04 12:28:29,076 WARN [main]
> org.apache.nifi.web.server.JettyServer Failed to start web server...
> shutting down.
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:426)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576)
> at
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at org.eclipse.jetty.server.Server.start(Server.java:452)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at org.eclipse.jetty.server.Server.doStart(Server.java:419)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.apache.nifi.web.server.JettyServer.start(JettyServer.java:705)
> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
> at org.apache.nifi.NiFi.main(NiFi.java:267)
> Caused by: java.lang.IllegalArgumentException: null
> at
> org.apache.nifi.controller.StandardProcessorNode.setMaxConcurrentTasks(StandardProcessorNode.java:620)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:985)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1064)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:312)
> ... 33 common frames omitted
>
>
>
> Joe Witt <ma...@gmail.com>
> Wednesday, July 5, 2017 10:57 AM
> Scott
>
> In your custom NAR are you extending some other processor/component
> such as one found in standard processors? You'll really need to break
> that chain and not pull in the same components.
>
> Can you share the actual error being provided at startup/
>
> Thanks
> joe
> Scott Wagner <ma...@beenverified.com>
> Wednesday, July 5, 2017 10:53 AM
> Hey all,
>
> I have been running on NiFi 1.1.1 for a while, and just started
> working on migrating to 1.3.0. However, when I tried starting up, I
> was getting unhandled exceptions during startup of Jetty. I have
> traced this down to something being wrong with a custom NAR file that
> we built for 1.1.1.
>
> The problem is when BundleUtils.findBundleForType is being called,
> it is finding 2 bundles for every internal NiFi processor class - the
> correct NAR that it comes with in 1.3.0, and my custom NAR file as
> well. I'm sure that I have done something wrong in the packaging for
> my custom NAR file, but I don't know what it could be. It was
> originally built from the NiFi NAR archetype late last year.
>
> If anyone can point in me in the direction of what I'm doing
> wrong, I would greatly appreciate it. I'm not a maven/packaging
> expert by any means so no advice is too basic.
>
> Thanks!
>
> - Scott
Re: Custom NAR interfering with BundleUtils.findBundleForType
Posted by Bryan Bende <bb...@gmail.com>.
Scott,
Thanks for providing the stacktrace... do any of your custom
processors use the @DefaultSchedule annotation? and if so, do any of
them set the tasks to a number less than 1?
The exception you are getting is from some code that is preventing
using 0 or negative number of tasks for a processor that is not
scheduled for event driven, basically meaning event driven is the only
one where it would make sense to have less than 1 task.
What Joe mentioned about including other standard processors in your
NAR could very well still be a problem, but that stacktrace might be
something else.
-Bryan
On Wed, Jul 5, 2017 at 12:44 PM, Scott Wagner <sw...@beenverified.com> wrote:
> Hi Joe,
>
> We are extending AbstractProcessor for our processors, and
> AbstractControllerService for our controller service. However, we did
> include the InvokeHTTP processor with some modifications that are
> referencing some other classes that are in the nifi-processors-standard JAR.
> I will look into breaking those out to remove that dependency.
>
> The actual error that we are getting is below:
>
> 2017-07-04 12:28:29,076 WARN [main] org.apache.nifi.web.server.JettyServer
> Failed to start web server... shutting down.
> org.apache.nifi.controller.serialization.FlowSynchronizationException:
> java.lang.IllegalArgumentException
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:426)
> at
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576)
> at
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
> at
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722)
> at
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533)
> at
> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
> at org.eclipse.jetty.server.Server.start(Server.java:452)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
> at org.eclipse.jetty.server.Server.doStart(Server.java:419)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.apache.nifi.web.server.JettyServer.start(JettyServer.java:705)
> at org.apache.nifi.NiFi.<init>(NiFi.java:160)
> at org.apache.nifi.NiFi.main(NiFi.java:267)
> Caused by: java.lang.IllegalArgumentException: null
> at
> org.apache.nifi.controller.StandardProcessorNode.setMaxConcurrentTasks(StandardProcessorNode.java:620)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:985)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1064)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
> at
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:312)
> ... 33 common frames omitted
>
>
>> Joe Witt <ma...@gmail.com>
>> Wednesday, July 5, 2017 10:57 AM
>> Scott
>>
>> In your custom NAR are you extending some other processor/component
>> such as one found in standard processors? You'll really need to break
>> that chain and not pull in the same components.
>>
>> Can you share the actual error being provided at startup/
>>
>> Thanks
>> joe
>> Scott Wagner <ma...@beenverified.com>
>> Wednesday, July 5, 2017 10:53 AM
>> Hey all,
>>
>> I have been running on NiFi 1.1.1 for a while, and just started
>> working on migrating to 1.3.0. However, when I tried starting up, I was
>> getting unhandled exceptions during startup of Jetty. I have traced this
>> down to something being wrong with a custom NAR file that we built for
>> 1.1.1.
>>
>> The problem is when BundleUtils.findBundleForType is being called, it
>> is finding 2 bundles for every internal NiFi processor class - the correct
>> NAR that it comes with in 1.3.0, and my custom NAR file as well. I'm sure
>> that I have done something wrong in the packaging for my custom NAR file,
>> but I don't know what it could be. It was originally built from the NiFi
>> NAR archetype late last year.
>>
>> If anyone can point in me in the direction of what I'm doing wrong, I
>> would greatly appreciate it. I'm not a maven/packaging expert by any means
>> so no advice is too basic.
>>
>> Thanks!
>>
>> - Scott
>
>
Re: Custom NAR interfering with BundleUtils.findBundleForType
Posted by Scott Wagner <sw...@beenverified.com>.
Hi Joe,
We are extending AbstractProcessor for our processors, and
AbstractControllerService for our controller service. However, we did
include the InvokeHTTP processor with some modifications that are
referencing some other classes that are in the nifi-processors-standard
JAR. I will look into breaking those out to remove that dependency.
The actual error that we are getting is below:
2017-07-04 12:28:29,076 WARN [main]
org.apache.nifi.web.server.JettyServer Failed to start web server...
shutting down.
org.apache.nifi.controller.serialization.FlowSynchronizationException:
java.lang.IllegalArgumentException
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:426)
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1576)
at
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:722)
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:533)
at
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
at
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
at
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
at org.eclipse.jetty.server.Server.start(Server.java:452)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
at org.eclipse.jetty.server.Server.doStart(Server.java:419)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.apache.nifi.web.server.JettyServer.start(JettyServer.java:705)
at org.apache.nifi.NiFi.<init>(NiFi.java:160)
at org.apache.nifi.NiFi.main(NiFi.java:267)
Caused by: java.lang.IllegalArgumentException: null
at
org.apache.nifi.controller.StandardProcessorNode.setMaxConcurrentTasks(StandardProcessorNode.java:620)
at
org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:985)
at
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1064)
at
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
at
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
at
org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1183)
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:312)
... 33 common frames omitted
> Joe Witt <ma...@gmail.com>
> Wednesday, July 5, 2017 10:57 AM
> Scott
>
> In your custom NAR are you extending some other processor/component
> such as one found in standard processors? You'll really need to break
> that chain and not pull in the same components.
>
> Can you share the actual error being provided at startup/
>
> Thanks
> joe
> Scott Wagner <ma...@beenverified.com>
> Wednesday, July 5, 2017 10:53 AM
> Hey all,
>
> I have been running on NiFi 1.1.1 for a while, and just started
> working on migrating to 1.3.0. However, when I tried starting up, I
> was getting unhandled exceptions during startup of Jetty. I have
> traced this down to something being wrong with a custom NAR file that
> we built for 1.1.1.
>
> The problem is when BundleUtils.findBundleForType is being called,
> it is finding 2 bundles for every internal NiFi processor class - the
> correct NAR that it comes with in 1.3.0, and my custom NAR file as
> well. I'm sure that I have done something wrong in the packaging for
> my custom NAR file, but I don't know what it could be. It was
> originally built from the NiFi NAR archetype late last year.
>
> If anyone can point in me in the direction of what I'm doing
> wrong, I would greatly appreciate it. I'm not a maven/packaging
> expert by any means so no advice is too basic.
>
> Thanks!
>
> - Scott
Re: Custom NAR interfering with BundleUtils.findBundleForType
Posted by Joe Witt <jo...@gmail.com>.
Scott
In your custom NAR are you extending some other processor/component
such as one found in standard processors? You'll really need to break
that chain and not pull in the same components.
Can you share the actual error being provided at startup/
Thanks
joe
On Wed, Jul 5, 2017 at 11:53 AM, Scott Wagner <sw...@beenverified.com> wrote:
> Hey all,
>
> I have been running on NiFi 1.1.1 for a while, and just started working
> on migrating to 1.3.0. However, when I tried starting up, I was getting
> unhandled exceptions during startup of Jetty. I have traced this down to
> something being wrong with a custom NAR file that we built for 1.1.1.
>
> The problem is when BundleUtils.findBundleForType is being called, it is
> finding 2 bundles for every internal NiFi processor class - the correct NAR
> that it comes with in 1.3.0, and my custom NAR file as well. I'm sure that
> I have done something wrong in the packaging for my custom NAR file, but I
> don't know what it could be. It was originally built from the NiFi NAR
> archetype late last year.
>
> If anyone can point in me in the direction of what I'm doing wrong, I
> would greatly appreciate it. I'm not a maven/packaging expert by any means
> so no advice is too basic.
>
> Thanks!
>
> - Scott