[GitHub] [hudi] melin commented on issue #4938: [SUPPORT]Add snapshot,  send event notification, user can customize listener

[GitBox <gi...@apache.org>]

melin commented on issue #4938:
URL: https://github.com/apache/hudi/issues/4938#issuecomment-1057570270


   > sorry, I am not very clear on the requirement. We already have commit callbacks. Would that suffice or are you looking for something else. can you throw some light please.
   
   There is a need. The upstream system periodically writes Hudi data to the object storage. The downstream system needs to know the changes of Hudi data. Currently, the upstream system can only periodically query Hudi snapshot information to determine whether Hudi table data has changed. Then the incremental calculation of the job is triggered. If there is a snapshot change notification mechanism (including partition change information). Users can customize the Listener to write the change message to Kafka, and the downstream system only needs to fetch the message from Kafka to know the data change.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org