PublishKafka_0_10 transfer relationship not specified

[Konstantin Knauf <ko...@tngtech.com>]

Hi everyone,

I am currently doing a small PoC, if we can use Apache NiFi as
integration component for our application.

Right now, I just read files from local disk and write them to a Kafka
topic (0.10.0.1):

GetFile -> PublishKafka_0_10 (terminates success and failure)

After a couple of files and a few million messages the PublishKafka_0_10
processors run into:

2017-06-16 12:05:07,972 ERROR [Timer-Driven Process Thread-10]
o.a.n.p.kafka.pubsub.PublishKafka_0_10
PublishKafka_0_10[id=b0578b3d-015c-100
0-e685-07f301c90901]
PublishKafka_0_10[id=b0578b3d-015c-1000-e685-07f301c90901] failed to
process due to org.apache.nifi.processor.exception
.FlowFileHandlingException:
StandardFlowFileRecord[uuid=c150df70-8f89-4f6b-9a2d-5be4df726df0,claim=StandardContentClaim
[resourceClaim=Stand
ardResourceClaim[id=1497607276559-29, container=default, section=29],
offset=0, length=0],offset=0,name=file5.csv,size=0] transfer relations
hip not specified; rolling back session: {}

You can find more logs attached. The processor never recovers. The Kafka
Cluster is fine.

Any pointer to what is going on or what I might be doing wrong is highly
appreciated.

Cheers and thank you,

Konstantin

-- 
Konstantin Knauf * konstantin.knauf@tngtech.com * +49-174-3413182
TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
Sitz: Unterföhring * Amtsgericht München * HRB 135082

Re: PublishKafka_0_10 transfer relationship not specified

[Mark Payne <ma...@hotmail.com>]

Hi Knauf,

What version of NiFi are you running?

Are the logs that you sent selectively pruned from your overall nifi-app.log, or did you just grab
a big section of the nifi-app.log file and send that? The reason that I ask is that quite often when
we see a 'transfer relationship not specified' error, it is the result of a bug in error handling. E.g.,
the FlowFile should be routed to failure for some other reason, and the part of the code that
transfers the FlowFile to the failure relationship has a problem. So I am curious if there are any other
log messages indicating what may have gone wrong?

Thanks
-Mark



> On Jun 16, 2017, at 6:13 AM, Konstantin Knauf <ko...@tngtech.com> wrote:
> 
> Hi everyone,
> 
> I am currently doing a small PoC, if we can use Apache NiFi as
> integration component for our application.
> 
> Right now, I just read files from local disk and write them to a Kafka
> topic (0.10.0.1):
> 
> GetFile -> PublishKafka_0_10 (terminates success and failure)
> 
> After a couple of files and a few million messages the PublishKafka_0_10
> processors run into:
> 
> 2017-06-16 12:05:07,972 ERROR [Timer-Driven Process Thread-10]
> o.a.n.p.kafka.pubsub.PublishKafka_0_10
> PublishKafka_0_10[id=b0578b3d-015c-100
> 0-e685-07f301c90901]
> PublishKafka_0_10[id=b0578b3d-015c-1000-e685-07f301c90901] failed to
> process due to org.apache.nifi.processor.exception
> .FlowFileHandlingException:
> StandardFlowFileRecord[uuid=c150df70-8f89-4f6b-9a2d-5be4df726df0,claim=StandardContentClaim
> [resourceClaim=Stand
> ardResourceClaim[id=1497607276559-29, container=default, section=29],
> offset=0, length=0],offset=0,name=file5.csv,size=0] transfer relations
> hip not specified; rolling back session: {}
> 
> You can find more logs attached. The processor never recovers. The Kafka
> Cluster is fine.
> 
> Any pointer to what is going on or what I might be doing wrong is highly
> appreciated.
> 
> Cheers and thank you,
> 
> Konstantin
> 
> -- 
> Konstantin Knauf * konstantin.knauf@tngtech.com * +49-174-3413182
> TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring
> Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke
> Sitz: Unterföhring * Amtsgericht München * HRB 135082
> <kafka-error-log.txt>