You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2024/09/21 21:46:00 UTC

[jira] [Commented] (NIFI-13784) Improve Kafka Processors' handling

    [ https://issues.apache.org/jira/browse/NIFI-13784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883548#comment-17883548 ] 

ASF subversion and git services commented on NIFI-13784:
--------------------------------------------------------

Commit 6ddae78b6fe6b2fbbcbd75fc5ed0d62d30c1c29b in nifi's branch refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6ddae78b6f ]

NIFI-13784 Fixed Kafka Publishing Behavior creating multiple Producers (#9297)

Fixed error handling in several situations. Cleaned up leaky abstraction. Some code cleanup, fixed default values for time-based properties to use 'sec' instead of 's' to adhere to typical conventions.

Signed-off-by: David Handermann <ex...@apache.org>

> Improve Kafka Processors' handling
> ----------------------------------
>
>                 Key: NIFI-13784
>                 URL: https://issues.apache.org/jira/browse/NIFI-13784
>             Project: Apache NiFi
>          Issue Type: Bug
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 2.0.0-M5
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> I ran into some issues with PublishKafka not properly handling timeouts. Any time there is a timeout, I encounter an error indicating that the incoming FlowFile was not transferred to any relationship. This then yields the processor, resulting in poor performance.
> Looking further, I'm finding other issues that need to be addressed also, such as PublishKafka creating a new Kafka Producer every time it is triggered to run. This is expensive always but far moreso when using transactional producers.
> The Consumer has a leaky abstraction, in that the ConsumerService returned extends AutoCloseable but then documents that processors should not close it, only the KafkaConnectionService should. In that case, ConsumerService should not implement AutoCloseable. We should never expose methods that we don't want called.
> Error handling is not properly implemented in the case of transaction commits timing out.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)