You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Chris Teoh <ch...@gmail.com> on 2015/08/30 08:21:03 UTC
PutHDFS irrecoverably stops NiFi from loading workflow
Hey folks,
I added PutHDFS into my workflow, pointed it to core-site.xml and
hdfs-site.xml and it crashed. Restarting doesn't seem to help as the
browser will say NiFi is loading, and then fails to load.
How do I remove the PutHDFS processor without the web UI?
Errors in the log are:-
o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
repository due to java.lang.OutOfMemoryError: PermGen space
o.apache.nifi.processors.hadoop.PutHDFS
[PutHDFS[id=4cc42c80-e37f-4122-a6f2-0c3b61ffc00b]] Failed to write to HDFS
due to {} java.lang.OutOfMemoryError: PermGen space at
java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_67] at
java.lang.ClassLoader.defineClass(ClassLoader.java:800) ~[na:1.7.0_67] at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
~[na:1.7.0_67] at
java.net.URLClassLoader.defineClass(URLClassLoader.java:449) ~[na:1.7.0_67]
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
~[na:1.7.0_67] at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
~[na:1.7.0_67] at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
~[na:1.7.0_67] at java.security.AccessController.doPrivileged(Native
Method) ~[na:1.7.0_67] at
java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67]
at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67] at
java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67] at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:755)
~[hadoop-hdfs-2.6.0.jar: na] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
~[na:1.7.0_67] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.7.0_67] at java.lang.reflect.Method.invoke(Method.java:606)
~[na:1.7.0_67] at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
~[hadoop-common-2.6.0.jar:na] at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
~[hadoop-common-2.6.0.jar:na] at
com.sun.proxy.$Proxy156.getFileInfo(Unknown Source) ~[na:na] at
org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
~[hadoop-hdfs-2.6.0.jar:na] at
org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
~[hadoop-hdfs-2.6.0.jar:na] at
org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
~[hadoop-hdfs-2.6.0.jar:na] at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
~[hadoop-common-2.6.0.jar:na] at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
~[hadoop-hdfs-2.6.0.jar:na] at
org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:232)
~[nifi-hdfs-processors-0.2.1.jar:0.2.1] at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-0.2.1.jar:0.2.1] at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
~[na:na] at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
~[na:na] at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
~[na:na] at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
~[na:na] at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_67] at
java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
~[na:1.7.0_67]
Re: PutHDFS irrecoverably stops NiFi from loading workflow
Posted by Chris Teoh <ch...@gmail.com>.
Hey all,
So here's the error:-
ERROR [NiFi Web Server-381780] org.apache.nifi.NiFi An Unknown Error
Occurred in Thread Thread[NiFi Web Server-381780,5,main]:
java.lang.OutOfMemoryError: PermGen space
The processors continue to run as I can see them continue to write entries
into the nifi-app.log file.
This error seemed to persist even when I set the PermGen values to 2048M.
In this scenario is there a way to remove a processor when the web UI is
unavailable?
Kind Regards
Chris
On Mon, 31 Aug 2015 at 02:27 Joe Witt <jo...@gmail.com> wrote:
> Not from a NiFi perspective. Java 8 works great. As a community
> though we plan to keep supporting Java 7 users as best we can until
> NiFi 1.0.0 happens.
>
> On Sun, Aug 30, 2015 at 12:10 PM, Rick Braddy <rb...@softnas.com> wrote:
> > Is there any reason to just use Java 8 so it's not an issue?
> >
> >
> >
> >> On Aug 30, 2015, at 5:54 AM, Joe Witt <jo...@gmail.com> wrote:
> >>
> >> Chris
> >>
> >> Alan nailed it. In bootstrap.conf just uncomment these two lines and
> >> restart nifi.
> >>
> >> #java.arg.11=-XX:PermSize=128M
> >> #java.arg.12=-XX:MaxPermSize=128M
> >>
> >>
> >> But you are not the first run to run into this stumbling block so have
> >> created this jira [1].
> >>
> >> Thanks
> >> Joe
> >>
> >> [1] https://issues.apache.org/jira/browse/NIFI-909
> >>
> >>> On Sun, Aug 30, 2015 at 2:58 AM, Alan Jackoway <al...@cloudera.com>
> wrote:
> >>> Have you tried setting the PermGen size with something like
> >>> -XX:MaxPermSize=512m ? If you look in the file bootstrap.conf you
> should see
> >>> an example of how to set it.
> >>>
> >>> I ask because that error doesn't look specific to PutHDFS to me. If
> the real
> >>> problem is PermGen space, you may go through a lot of work trying to
> remove
> >>> that processor and then have this happen again when you add something
> else.
> >>>
> >>> Let us know if bootstrap.conf works for you,
> >>> Alan
> >>>
> >>>> On Aug 30, 2015 7:21 AM, "Chris Teoh" <ch...@gmail.com> wrote:
> >>>>
> >>>> Hey folks,
> >>>>
> >>>> I added PutHDFS into my workflow, pointed it to core-site.xml and
> >>>> hdfs-site.xml and it crashed. Restarting doesn't seem to help as the
> browser
> >>>> will say NiFi is loading, and then fails to load.
> >>>>
> >>>> How do I remove the PutHDFS processor without the web UI?
> >>>>
> >>>> Errors in the log are:-
> >>>> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
> >>>> repository due to java.lang.OutOfMemoryError: PermGen space
> >>>>
> >>>> o.apache.nifi.processors.hadoop.PutHDFS
> >>>> [PutHDFS[id=4cc42c80-e37f-4122-a6f2-0c3b61ffc00b]] Failed to write to
> HDFS
> >>>> due to {} java.lang.OutOfMemoryError: PermGen space at
> >>>> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_67] at
> >>>> java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> ~[na:1.7.0_67] at
> >>>>
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> >>>> ~[na:1.7.0_67] at
> >>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> ~[na:1.7.0_67]
> >>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> ~[na:1.7.0_67]
> >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> ~[na:1.7.0_67] at
> >>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67]
> at
> >>>> java.security.AccessController.doPrivileged(Native Method)
> ~[na:1.7.0_67] at
> >>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> ~[na:1.7.0_67] at
> >>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67]
> at
> >>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67]
> at
> >>>>
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:755)
> >>>> ~[hadoop-hdfs-2.6.0.jar: na] at
> >>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.7.0_67]
> >>>> at
> >>>>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>>> ~[na:1.7.0_67] at
> >>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>> ~[na:1.7.0_67] at java.lang.reflect.Method.invoke(Method.java:606)
> >>>> ~[na:1.7.0_67] at
> >>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> >>>> ~[hadoop-common-2.6.0.jar:na] at
> >>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> >>>> ~[hadoop-common-2.6.0.jar:na] at
> com.sun.proxy.$Proxy156.getFileInfo(Unknown
> >>>> Source) ~[na:na] at
> >>>> org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
> >>>> ~[hadoop-hdfs-2.6.0.jar:na] at
> >>>>
> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
> >>>> ~[hadoop-hdfs-2.6.0.jar:na] at
> >>>>
> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
> >>>> ~[hadoop-hdfs-2.6.0.jar:na] at
> >>>>
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> >>>> ~[hadoop-common-2.6.0.jar:na] at
> >>>>
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
> >>>> ~[hadoop-hdfs-2.6.0.jar:na] at
> >>>> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:232)
> >>>> ~[nifi-hdfs-processors-0.2.1.jar:0.2.1] at
> >>>>
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> >>>> ~[nifi-api-0.2.1.jar:0.2.1] at
> >>>>
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
> >>>> ~[na:na] at
> >>>>
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
> >>>> ~[na:na] at
> >>>>
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> >>>> ~[na:na] at
> >>>>
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
> >>>> ~[na:na] at
> >>>>
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> >>>> ~[na:1.7.0_67] at
> >>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> >>>> ~[na:1.7.0_67]
>
Re: PutHDFS irrecoverably stops NiFi from loading workflow
Posted by Joe Witt <jo...@gmail.com>.
Not from a NiFi perspective. Java 8 works great. As a community
though we plan to keep supporting Java 7 users as best we can until
NiFi 1.0.0 happens.
On Sun, Aug 30, 2015 at 12:10 PM, Rick Braddy <rb...@softnas.com> wrote:
> Is there any reason to just use Java 8 so it's not an issue?
>
>
>
>> On Aug 30, 2015, at 5:54 AM, Joe Witt <jo...@gmail.com> wrote:
>>
>> Chris
>>
>> Alan nailed it. In bootstrap.conf just uncomment these two lines and
>> restart nifi.
>>
>> #java.arg.11=-XX:PermSize=128M
>> #java.arg.12=-XX:MaxPermSize=128M
>>
>>
>> But you are not the first run to run into this stumbling block so have
>> created this jira [1].
>>
>> Thanks
>> Joe
>>
>> [1] https://issues.apache.org/jira/browse/NIFI-909
>>
>>> On Sun, Aug 30, 2015 at 2:58 AM, Alan Jackoway <al...@cloudera.com> wrote:
>>> Have you tried setting the PermGen size with something like
>>> -XX:MaxPermSize=512m ? If you look in the file bootstrap.conf you should see
>>> an example of how to set it.
>>>
>>> I ask because that error doesn't look specific to PutHDFS to me. If the real
>>> problem is PermGen space, you may go through a lot of work trying to remove
>>> that processor and then have this happen again when you add something else.
>>>
>>> Let us know if bootstrap.conf works for you,
>>> Alan
>>>
>>>> On Aug 30, 2015 7:21 AM, "Chris Teoh" <ch...@gmail.com> wrote:
>>>>
>>>> Hey folks,
>>>>
>>>> I added PutHDFS into my workflow, pointed it to core-site.xml and
>>>> hdfs-site.xml and it crashed. Restarting doesn't seem to help as the browser
>>>> will say NiFi is loading, and then fails to load.
>>>>
>>>> How do I remove the PutHDFS processor without the web UI?
>>>>
>>>> Errors in the log are:-
>>>> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
>>>> repository due to java.lang.OutOfMemoryError: PermGen space
>>>>
>>>> o.apache.nifi.processors.hadoop.PutHDFS
>>>> [PutHDFS[id=4cc42c80-e37f-4122-a6f2-0c3b61ffc00b]] Failed to write to HDFS
>>>> due to {} java.lang.OutOfMemoryError: PermGen space at
>>>> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_67] at
>>>> java.lang.ClassLoader.defineClass(ClassLoader.java:800) ~[na:1.7.0_67] at
>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>> ~[na:1.7.0_67] at
>>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) ~[na:1.7.0_67]
>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) ~[na:1.7.0_67]
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) ~[na:1.7.0_67] at
>>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67] at
>>>> java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67] at
>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67] at
>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67] at
>>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67] at
>>>> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:755)
>>>> ~[hadoop-hdfs-2.6.0.jar: na] at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> ~[na:1.7.0_67] at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> ~[na:1.7.0_67] at java.lang.reflect.Method.invoke(Method.java:606)
>>>> ~[na:1.7.0_67] at
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
>>>> ~[hadoop-common-2.6.0.jar:na] at
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>>>> ~[hadoop-common-2.6.0.jar:na] at com.sun.proxy.$Proxy156.getFileInfo(Unknown
>>>> Source) ~[na:na] at
>>>> org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
>>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>>> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
>>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>>> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
>>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>>>> ~[hadoop-common-2.6.0.jar:na] at
>>>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
>>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>>> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:232)
>>>> ~[nifi-hdfs-processors-0.2.1.jar:0.2.1] at
>>>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>>>> ~[nifi-api-0.2.1.jar:0.2.1] at
>>>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
>>>> ~[na:na] at
>>>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
>>>> ~[na:na] at
>>>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
>>>> ~[na:na] at
>>>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
>>>> ~[na:na] at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>> ~[na:1.7.0_67] at
>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>>> ~[na:1.7.0_67]
Re: PutHDFS irrecoverably stops NiFi from loading workflow
Posted by Rick Braddy <rb...@softnas.com>.
Is there any reason to just use Java 8 so it's not an issue?
> On Aug 30, 2015, at 5:54 AM, Joe Witt <jo...@gmail.com> wrote:
>
> Chris
>
> Alan nailed it. In bootstrap.conf just uncomment these two lines and
> restart nifi.
>
> #java.arg.11=-XX:PermSize=128M
> #java.arg.12=-XX:MaxPermSize=128M
>
>
> But you are not the first run to run into this stumbling block so have
> created this jira [1].
>
> Thanks
> Joe
>
> [1] https://issues.apache.org/jira/browse/NIFI-909
>
>> On Sun, Aug 30, 2015 at 2:58 AM, Alan Jackoway <al...@cloudera.com> wrote:
>> Have you tried setting the PermGen size with something like
>> -XX:MaxPermSize=512m ? If you look in the file bootstrap.conf you should see
>> an example of how to set it.
>>
>> I ask because that error doesn't look specific to PutHDFS to me. If the real
>> problem is PermGen space, you may go through a lot of work trying to remove
>> that processor and then have this happen again when you add something else.
>>
>> Let us know if bootstrap.conf works for you,
>> Alan
>>
>>> On Aug 30, 2015 7:21 AM, "Chris Teoh" <ch...@gmail.com> wrote:
>>>
>>> Hey folks,
>>>
>>> I added PutHDFS into my workflow, pointed it to core-site.xml and
>>> hdfs-site.xml and it crashed. Restarting doesn't seem to help as the browser
>>> will say NiFi is loading, and then fails to load.
>>>
>>> How do I remove the PutHDFS processor without the web UI?
>>>
>>> Errors in the log are:-
>>> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
>>> repository due to java.lang.OutOfMemoryError: PermGen space
>>>
>>> o.apache.nifi.processors.hadoop.PutHDFS
>>> [PutHDFS[id=4cc42c80-e37f-4122-a6f2-0c3b61ffc00b]] Failed to write to HDFS
>>> due to {} java.lang.OutOfMemoryError: PermGen space at
>>> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_67] at
>>> java.lang.ClassLoader.defineClass(ClassLoader.java:800) ~[na:1.7.0_67] at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>> ~[na:1.7.0_67] at
>>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) ~[na:1.7.0_67]
>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) ~[na:1.7.0_67]
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) ~[na:1.7.0_67] at
>>> java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67] at
>>> java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67] at
>>> java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67] at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67] at
>>> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67] at
>>> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:755)
>>> ~[hadoop-hdfs-2.6.0.jar: na] at
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> ~[na:1.7.0_67] at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> ~[na:1.7.0_67] at java.lang.reflect.Method.invoke(Method.java:606)
>>> ~[na:1.7.0_67] at
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
>>> ~[hadoop-common-2.6.0.jar:na] at
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>>> ~[hadoop-common-2.6.0.jar:na] at com.sun.proxy.$Proxy156.getFileInfo(Unknown
>>> Source) ~[na:na] at
>>> org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>>> ~[hadoop-common-2.6.0.jar:na] at
>>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
>>> ~[hadoop-hdfs-2.6.0.jar:na] at
>>> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:232)
>>> ~[nifi-hdfs-processors-0.2.1.jar:0.2.1] at
>>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>>> ~[nifi-api-0.2.1.jar:0.2.1] at
>>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
>>> ~[na:na] at
>>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
>>> ~[na:na] at
>>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
>>> ~[na:na] at
>>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
>>> ~[na:na] at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>> ~[na:1.7.0_67] at
>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>>> ~[na:1.7.0_67]
Re: PutHDFS irrecoverably stops NiFi from loading workflow
Posted by Joe Witt <jo...@gmail.com>.
Chris
Alan nailed it. In bootstrap.conf just uncomment these two lines and
restart nifi.
#java.arg.11=-XX:PermSize=128M
#java.arg.12=-XX:MaxPermSize=128M
But you are not the first run to run into this stumbling block so have
created this jira [1].
Thanks
Joe
[1] https://issues.apache.org/jira/browse/NIFI-909
On Sun, Aug 30, 2015 at 2:58 AM, Alan Jackoway <al...@cloudera.com> wrote:
> Have you tried setting the PermGen size with something like
> -XX:MaxPermSize=512m ? If you look in the file bootstrap.conf you should see
> an example of how to set it.
>
> I ask because that error doesn't look specific to PutHDFS to me. If the real
> problem is PermGen space, you may go through a lot of work trying to remove
> that processor and then have this happen again when you add something else.
>
> Let us know if bootstrap.conf works for you,
> Alan
>
> On Aug 30, 2015 7:21 AM, "Chris Teoh" <ch...@gmail.com> wrote:
>>
>> Hey folks,
>>
>> I added PutHDFS into my workflow, pointed it to core-site.xml and
>> hdfs-site.xml and it crashed. Restarting doesn't seem to help as the browser
>> will say NiFi is loading, and then fails to load.
>>
>> How do I remove the PutHDFS processor without the web UI?
>>
>> Errors in the log are:-
>> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
>> repository due to java.lang.OutOfMemoryError: PermGen space
>>
>> o.apache.nifi.processors.hadoop.PutHDFS
>> [PutHDFS[id=4cc42c80-e37f-4122-a6f2-0c3b61ffc00b]] Failed to write to HDFS
>> due to {} java.lang.OutOfMemoryError: PermGen space at
>> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_67] at
>> java.lang.ClassLoader.defineClass(ClassLoader.java:800) ~[na:1.7.0_67] at
>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> ~[na:1.7.0_67] at
>> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) ~[na:1.7.0_67]
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) ~[na:1.7.0_67]
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) ~[na:1.7.0_67] at
>> java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67] at
>> java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67] at
>> java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67] at
>> java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67] at
>> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67] at
>> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:755)
>> ~[hadoop-hdfs-2.6.0.jar: na] at
>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> ~[na:1.7.0_67] at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> ~[na:1.7.0_67] at java.lang.reflect.Method.invoke(Method.java:606)
>> ~[na:1.7.0_67] at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
>> ~[hadoop-common-2.6.0.jar:na] at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>> ~[hadoop-common-2.6.0.jar:na] at com.sun.proxy.$Proxy156.getFileInfo(Unknown
>> Source) ~[na:na] at
>> org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
>> ~[hadoop-hdfs-2.6.0.jar:na] at
>> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
>> ~[hadoop-hdfs-2.6.0.jar:na] at
>> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
>> ~[hadoop-hdfs-2.6.0.jar:na] at
>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>> ~[hadoop-common-2.6.0.jar:na] at
>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
>> ~[hadoop-hdfs-2.6.0.jar:na] at
>> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:232)
>> ~[nifi-hdfs-processors-0.2.1.jar:0.2.1] at
>> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>> ~[nifi-api-0.2.1.jar:0.2.1] at
>> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
>> ~[na:na] at
>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
>> ~[na:na] at
>> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
>> ~[na:na] at
>> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
>> ~[na:na] at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> ~[na:1.7.0_67] at
>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>> ~[na:1.7.0_67]
Re: PutHDFS irrecoverably stops NiFi from loading workflow
Posted by Alan Jackoway <al...@cloudera.com>.
Have you tried setting the PermGen size with something like
-XX:MaxPermSize=512m ? If you look in the file bootstrap.conf you should
see an example of how to set it.
I ask because that error doesn't look specific to PutHDFS to me. If the
real problem is PermGen space, you may go through a lot of work trying to
remove that processor and then have this happen again when you add
something else.
Let us know if bootstrap.conf works for you,
Alan
On Aug 30, 2015 7:21 AM, "Chris Teoh" <ch...@gmail.com> wrote:
> Hey folks,
>
> I added PutHDFS into my workflow, pointed it to core-site.xml and
> hdfs-site.xml and it crashed. Restarting doesn't seem to help as the
> browser will say NiFi is loading, and then fails to load.
>
> How do I remove the PutHDFS processor without the web UI?
>
> Errors in the log are:-
> o.a.n.p.PersistentProvenanceRepository Failed to rollover Provenance
> repository due to java.lang.OutOfMemoryError: PermGen space
>
> o.apache.nifi.processors.hadoop.PutHDFS
> [PutHDFS[id=4cc42c80-e37f-4122-a6f2-0c3b61ffc00b]] Failed to write to HDFS
> due to {} java.lang.OutOfMemoryError: PermGen space at
> java.lang.ClassLoader.defineClass1(Native Method) ~[na:1.7.0_67] at
> java.lang.ClassLoader.defineClass(ClassLoader.java:800) ~[na:1.7.0_67] at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> ~[na:1.7.0_67] at
> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) ~[na:1.7.0_67]
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> ~[na:1.7.0_67] at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> ~[na:1.7.0_67] at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> ~[na:1.7.0_67] at java.security.AccessController.doPrivileged(Native
> Method) ~[na:1.7.0_67] at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67] at
> java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67] at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:755)
> ~[hadoop-hdfs-2.6.0.jar: na] at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> ~[na:1.7.0_67] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.7.0_67] at java.lang.reflect.Method.invoke(Method.java:606)
> ~[na:1.7.0_67] at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> ~[hadoop-common-2.6.0.jar:na] at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> ~[hadoop-common-2.6.0.jar:na] at
> com.sun.proxy.$Proxy156.getFileInfo(Unknown Source) ~[na:na] at
> org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1988)
> ~[hadoop-hdfs-2.6.0.jar:na] at
> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
> ~[hadoop-hdfs-2.6.0.jar:na] at
> org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
> ~[hadoop-hdfs-2.6.0.jar:na] at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> ~[hadoop-common-2.6.0.jar:na] at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
> ~[hadoop-hdfs-2.6.0.jar:na] at
> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:232)
> ~[nifi-hdfs-processors-0.2.1.jar:0.2.1] at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-0.2.1.jar:0.2.1] at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1077)
> ~[na:na] at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:127)
> ~[na:na] at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
> ~[na:na] at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
> ~[na:na] at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_67] at
> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> ~[na:1.7.0_67]
>