[jira] [Created] (NIFI-12306) ConsumeAzureEventHub should not log partition ownership changes at error level

["Peter Turcsanyi (Jira)" <ji...@apache.org>]

Peter Turcsanyi created NIFI-12306:
--------------------------------------

             Summary: ConsumeAzureEventHub should not log partition ownership changes at error level
                 Key: NIFI-12306
                 URL: https://issues.apache.org/jira/browse/NIFI-12306
             Project: Apache NiFi
          Issue Type: Improvement
            Reporter: Peter Turcsanyi
            Assignee: Peter Turcsanyi


Azure Event Hub client instances run a load balancing algorithm which checks the distribution of the partitions among the consumer group members and can rebalance the partition assignments.

This algorithm runs independently on each client node and if the node owns less partitions than the fair share (typically after start-up), it "steals" a partition from another node.

The other node only recognizes it via the broken connection (2 consumers from the same consumer group cannot consume the same partition at the same time) so basically it is an error event on that side. It seems it is by design and will not change in the Azure client lib.

However, we can handle this type of "error" differently on the NiFi side and do not log it at error level but only info. The current error bulletin is quite confusing for the user.

Error message:
{code:java}
2023-11-01 13:19:46,613 ERROR [partition-pump-1-22] o.a.n.p.a.eventhub.ConsumeAzureEventHub ConsumeAzureEventHub[id=7d57cb75-018b-1000-0000-000042a1ec17] Receive Events failed Namespace [turcsanyi-eventhub-ns.servicebus.windows.net] Event Hub [turcsanyi-eventhub-2] Consumer Group [$default] Partition [1]
com.azure.core.amqp.exception.AmqpException: New receiver '7661792b-acf1-49b8-9474-2ac06ebc22d9' with higher epoch of '0' is created hence current receiver '8bec0a38-2d58-46d1-ac37-4cb80bd7ba29' with epoch '0' is getting disconnected. If you are recreating the receiver, make sure a higher epoch is used. TrackingId:ddb15716000068bf0010179a6542425c_G13S3_B13S1, SystemTracker:turcsanyi-eventhub-ns:eventhub:turcsanyi-eventhub-2~16383|$default, Timestamp:2023-11-01T12:19:46, errorContext[NAMESPACE: turcsanyi-eventhub-ns.servicebus.windows.net. ERROR CONTEXT: N/A, PATH: turcsanyi-eventhub-2/ConsumerGroups/$default/Partitions/1, REFERENCE_ID: 1_b1a39c_1698841180105, LINK_CREDIT: 300]
	at com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:85)
	at com.azure.core.amqp.implementation.handler.LinkHandler.handleRemoteLinkClosed(LinkHandler.java:124)
  ...
{code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)