[GitHub] [nifi] exceptionfactory commented on pull request #6319: NIFI-10381 Refactor Azure Event Hubs components with current SDK

[GitBox <gi...@apache.org>]

exceptionfactory commented on PR #6319:
URL: https://github.com/apache/nifi/pull/6319#issuecomment-1253251075

   Thanks for the feedback and testing @turcsanyip!
   
   I pushed an update incorporating your suggestions for credential handling in `PutAzureEventHub` and also addressed issues with Event Position handling.
   
   For `GetAzureEventHub`, the initial enqueued time determination happens once as suggested, and subsequent executions use the `PartitionProperties` to determine the last enqueued time. This approach should be more robust for handling gaps between processor execution.
   
   For `ConsumeAzureEventHub`, the new  `LegacyBlobStorageEventPositionProvider` is responsible for locating existing checkpoint information and returning it when found. The implementation follows the storage approach used in [AzureStorageCheckpointLeaseManager](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/eventhubs/microsoft-azure-eventhubs-eph/src/main/java/com/microsoft/azure/eventprocessorhost/AzureStorageCheckpointLeaseManager.java) from the legacy SDK. The implementation will read existing sequence numbers as use them to derive a starting Event Position when found. The provider also deletes the blob since the consumer will subsequently store checkpoint information using the new SDK strategy. This approach should handle the basic migration scenario when upgrading NiFi to a version that includes these changes. It is probably worth including a note in the NiFi migration guidance for awareness.
   
   Please let me know if you notice any additional issues.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org