You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com> on 2016/03/23 23:07:40 UTC

Has anyone this error signature in PutKafka

2016-03-23 21:29:22,383 ERROR [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] failed to process session due to java.lang.NegativeArraySizeException: java.lang.NegativeArraySizeException
2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] Processor Administratively Yielded for 1 sec due to processing failure
2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] due to uncaught Exception: java.lang.NegativeArraySizeException
2016-03-23 21:29:22,392 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
java.lang.NegativeArraySizeException: null
        at org.apache.nifi.processors.kafka.PutKafka.onTrigger(PutKafka.java:451) ~[na:na]
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) ~[nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]

I’m running HDF 1.1.1.0-12 (not sure which NiFi version that maps to) and Kafka 0.9.0

Thanks,
Chris

Re: Has anyone this error signature in PutKafka

Posted by Oleg Zhurakousky <oz...@hortonworks.com>.

Interesting question and the only way I can answer it is that it shouldn’t, but would need more context

Oleg

> On Mar 23, 2016, at 6:27 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
> 
> Thanks, Oleg.
> 
> FWIW the traceback seems to correspond to this line in PutKafka.java
> 
> final byte[] value = new byte[(int) flowFile.getSize()];
> 
> 
> Can the flow file size be negative?
> 
> 
> 
> On 3/23/16, 6:24 PM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:
> 
>> Chris
>> 
>> Yes we have (can’t remember the details though). There was a lot of work around NiFi Kafka support lately and it somewhat culminated today. So. . .
>> 1. We’ve downgraded back to 0.8.2 https://issues.apache.org/jira/browse/NIFI-1629. You can see details in JIRA
>> 2. We’ve done some refactoring in PutKafka to address this https://issues.apache.org/jira/browse/NIFI-1645 which was merged to the master few hours ago
>> 
>> The 0.6.0 release is imminent, so I’d suggest to either wait few days or if you can build from master that would be ideal and would help us tremendously with additional validation.
>> Let me know if you need any help with building.
>> 
>> Cheers
>> Oleg
>>> On Mar 23, 2016, at 6:07 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>>> 
>>> 2016-03-23 21:29:22,383 ERROR [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] failed to process session due to java.lang.NegativeArraySizeException: java.lang.NegativeArraySizeException
>>> 2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] Processor Administratively Yielded for 1 sec due to processing failure
>>> 2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] due to uncaught Exception: java.lang.NegativeArraySizeException
>>> 2016-03-23 21:29:22,392 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
>>> java.lang.NegativeArraySizeException: null
>>>       at org.apache.nifi.processors.kafka.PutKafka.onTrigger(PutKafka.java:451) ~[na:na]
>>>       at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) ~[nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>>       at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>>       at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>>       at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>>       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>>>       at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45]
>>>       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45]
>>>       at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45]
>>>       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
>>>       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
>>>       at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
>>> 
>>> I’m running HDF 1.1.1.0-12 (not sure which NiFi version that maps to) and Kafka 0.9.0
>>> 
>>> Thanks,
>>> Chris
>>

Re: Has anyone this error signature in PutKafka

Posted by "McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote)" <ch...@hpe.com>.

Thanks, Oleg.

FWIW the traceback seems to correspond to this line in PutKafka.java

 final byte[] value = new byte[(int) flowFile.getSize()];


Can the flow file size be negative?



On 3/23/16, 6:24 PM, "Oleg Zhurakousky" <oz...@hortonworks.com> wrote:

>Chris
>
>Yes we have (can’t remember the details though). There was a lot of work around NiFi Kafka support lately and it somewhat culminated today. So. . .
>1. We’ve downgraded back to 0.8.2 https://issues.apache.org/jira/browse/NIFI-1629. You can see details in JIRA
>2. We’ve done some refactoring in PutKafka to address this https://issues.apache.org/jira/browse/NIFI-1645 which was merged to the master few hours ago
>
>The 0.6.0 release is imminent, so I’d suggest to either wait few days or if you can build from master that would be ideal and would help us tremendously with additional validation.
>Let me know if you need any help with building.
>
>Cheers
>Oleg
>> On Mar 23, 2016, at 6:07 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
>> 
>> 2016-03-23 21:29:22,383 ERROR [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] failed to process session due to java.lang.NegativeArraySizeException: java.lang.NegativeArraySizeException
>> 2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] Processor Administratively Yielded for 1 sec due to processing failure
>> 2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] due to uncaught Exception: java.lang.NegativeArraySizeException
>> 2016-03-23 21:29:22,392 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
>> java.lang.NegativeArraySizeException: null
>>        at org.apache.nifi.processors.kafka.PutKafka.onTrigger(PutKafka.java:451) ~[na:na]
>>        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) ~[nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45]
>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45]
>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45]
>>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
>>        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
>> 
>> I’m running HDF 1.1.1.0-12 (not sure which NiFi version that maps to) and Kafka 0.9.0
>> 
>> Thanks,
>> Chris
>

Re: Has anyone this error signature in PutKafka

Posted by Oleg Zhurakousky <oz...@hortonworks.com>.

Chris

Yes we have (can’t remember the details though). There was a lot of work around NiFi Kafka support lately and it somewhat culminated today. So. . .
1. We’ve downgraded back to 0.8.2 https://issues.apache.org/jira/browse/NIFI-1629. You can see details in JIRA
2. We’ve done some refactoring in PutKafka to address this https://issues.apache.org/jira/browse/NIFI-1645 which was merged to the master few hours ago

The 0.6.0 release is imminent, so I’d suggest to either wait few days or if you can build from master that would be ideal and would help us tremendously with additional validation.
Let me know if you need any help with building.

Cheers
Oleg
> On Mar 23, 2016, at 6:07 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <ch...@hpe.com> wrote:
> 
> 2016-03-23 21:29:22,383 ERROR [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] failed to process session due to java.lang.NegativeArraySizeException: java.lang.NegativeArraySizeException
> 2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.apache.nifi.processors.kafka.PutKafka PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] Processor Administratively Yielded for 1 sec due to processing failure
> 2016-03-23 21:29:22,383 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding PutKafka[id=f66e4092-946e-3338-93d6-7ea3a56bfd20] due to uncaught Exception: java.lang.NegativeArraySizeException
> 2016-03-23 21:29:22,392 WARN [Timer-Driven Process Thread-5] o.a.n.c.t.ContinuallyRunProcessorTask
> java.lang.NegativeArraySizeException: null
>        at org.apache.nifi.processors.kafka.PutKafka.onTrigger(PutKafka.java:451) ~[na:na]
>        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) ~[nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>        at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) [nifi-framework-core-1.1.1.0-12.jar:1.1.1.0-12]
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_45]
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_45]
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_45]
>        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]
>        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> 
> I’m running HDF 1.1.1.0-12 (not sure which NiFi version that maps to) and Kafka 0.9.0
> 
> Thanks,
> Chris