[jira] [Commented] (NIFI-1534) PutKafka fails when Run Duration > 0

["Oleg Zhurakousky (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154376#comment-15154376 ] 

Oleg Zhurakousky commented on NIFI-1534:
----------------------------------------

This si a real catch 22 situation.  Basically PutKafka is annotated with _@TriggerWhenEmpty_ and _@SupportsBatching_. On top of that the implementation logic is structured that there may be a significant delay (even without batching) on when the FlowFile is transferred to a Relationship and the actual session for which messages have been sent is committed which in a way exposes another bug. For example imagine your task runs once every hour. Even without batching the FlowFile will be transferred and session will be committed an hour after the actual message have been sent. That is because the logic in onTrigger() processes internal queue first before building a new one and since internal queue doesn't exist on the first invocation of _onTrigger()_, nothing is being transferred and/or committed on the first invocation of _onTrigger()_. And with each subsequent invocation the component is always one step behind. 

The above is also at the core of immediate issue since on the first invocation of the _onTrigger()_ nothing is transferred to any relationship, but given the support for batching (via _@SupportsBatching_) and *Run-duration > 0* the ContinuallyRunningProcessTask attempts to commit session after each invocation of _onTrigger()_ resulting in "_transfer relationship not specified_"

So for the time being the workaround is to not set Run-duration to any value > 0.


> PutKafka fails when Run Duration > 0
> ------------------------------------
>
>                 Key: NIFI-1534
>                 URL: https://issues.apache.org/jira/browse/NIFI-1534
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Oleg Zhurakousky
>            Assignee: Oleg Zhurakousky
>             Fix For: 0.5.1
>
>
> The following error shows up in the bulletin
> {code}
> PutKafka[id=80e8946f-349c-4716-9da9-5e766582f788] Failed to process session due to org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=27030bf8-c217-4b59-a0ba-46f1b30ff093,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1455807429050-246, container=default, section=246], offset=563929, length=431],offset=0,name=hh77.33,size=431] is not known in this session
> {code}
> After some debugging I also see this
> {code}
> ERROR [Timer-Driven Process Thread-14] o.apache.nifi.processors.kafka.PutKafka
> org.apache.nifi.processor.exception.FlowFileHandlingException: StandardFlowFileRecord[uuid=dd373d71-ebd2-432e-bcf5-9980d7df4da4,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1454688235111-5, container=content2, section=5], offset=0, length=4778852],offset=0,name=boo,size=4778852] transfer relationship not specified
>         at org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:210) 
>         at org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:280)
>         at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:179)
> . . . .
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)