You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@nifi.apache.org by "Mark Payne (JIRA)" <ji...@apache.org> on 2016/10/04 19:23:20 UTC

[jira] [Created] (NIFI-2865) Address issues of PublishKafka blocking when having trouble communicating with Kafka broker and improve performance

Mark Payne created NIFI-2865:
--------------------------------

             Summary: Address issues of PublishKafka blocking when having trouble communicating with Kafka broker and improve performance
                 Key: NIFI-2865
                 URL: https://issues.apache.org/jira/browse/NIFI-2865
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
            Reporter: Mark Payne
            Assignee: Mark Payne
             Fix For: 1.1.0


When NiFi is unable to communicate properly with the Kafka broker, we see the NiFi threads occasionally block. This should be resolvable by calling the wakeup() method of the client. Additionally, if Kafka takes too long to respond, we should be able to route the FlowFile to failure and move on.

PublishKafka has a nice feature that allows a demarcated stream to be sent as separate messages, so that a large number of messages can be sent as a single FlowFile. However, in the case of individual messages per FlowFile, the performance could be improved by batching together multiple FlowFiles per session



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)