You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "gardenia (via GitHub)" <gi...@apache.org> on 2026/06/15 12:19:41 UTC
[PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint [ozone]
gardenia opened a new pull request, #10515:
URL: https://github.com/apache/ozone/pull/10515
Please describe your PR in detail:
* Persist the checkpoint within the Ozone filesystem itself so that another OM can pick up if the leader changes.
* The approach is to use the ozone filesystem itself to ensure distributed storage.
* The mechanic is to store the checkpoint within bucket metadata properties
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15562
## How was this patch tested?
Unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint [ozone]
Posted by "gardenia (via GitHub)" <gi...@apache.org>.
gardenia commented on PR #10515:
URL: https://github.com/apache/ozone/pull/10515#issuecomment-4883083416
> @gardenia , the progress indicator is the txID, key is "notification-checkpoint", value is txID. How about use the OM metadata table to persist this value? metadata is defined in OMDBDefinition.java, currently it's used to persist data like OM layout version, ranger ozone service version, it can perfectly save the "notification-checkpoint" -> txID data, Or we can introduce a new table to save it, just like the statefulServiceConfig table in SCM, which can be used by any feature of SCM.
Thanks for your suggestions. I liked the idea that using something like SetBucketPropertyRequest meant we can piggyback on existing HA storage and ratis requests but I can see how your approach would be cleaner.
If we went with your suggestion I'm assuming we would need a need a new Ratis request/response to distribute this to other OMs (similar to OMSetRangerServiceVersionRequest) ? e.g. SaveCheckpointRequest ? Or is there something else to piggy back on for that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint (via Ozone filesystem bucket properties) [ozone]
Posted by "gardenia (via GitHub)" <gi...@apache.org>.
gardenia closed pull request #10515: HDDS-15562. EventNotification: persistence strategy for consumer checkpoint (via Ozone filesystem bucket properties)
URL: https://github.com/apache/ozone/pull/10515
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on PR #10515:
URL: https://github.com/apache/ozone/pull/10515#issuecomment-4874076396
@gardenia , the progress indicator is the txID, key is "notification-checkpoint", value is txID. How about use the OM metadata table to persist this value? metadata is defined in OMDBDefinition.java, currently it's used to persist data like OM layout version, ranger ozone service version, it can perfectly save the "notification-checkpoint" -> txID data, Or we can introduce a new table to save it, just like the statefulServiceConfig table in SCM, which can be used by any feature of SCM.
Second, this indicator tries to prevent OM to handle events which have already been handled before OM start or OM leader transfer, but since the event model is at least once, the receiver need to have built-in capability to handle resent event notification. So the checkpoint loading or saving fails, we should still go ahead and send events out. Checkpoint is nice to have, not mandatory, it's failure should lead to the event process task function fail.
Third, the event framework supports multiple plugins, each plugin runs independently. The current checkpoint and #10516 are specific to Kafka. If there is another plugin loaded, this plugin may also has it's checkpoint, and it needs to coordinate with Kafka about the purge, in case one plugin moving fast, while another one moving relatively slow. It looks like the current framework doesn't support this if we really want to support multiple plugins running together.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint [ozone]
Posted by "gardenia (via GitHub)" <gi...@apache.org>.
gardenia commented on PR #10515:
URL: https://github.com/apache/ozone/pull/10515#issuecomment-4709952325
@ChenSammi FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint (via Ozone filesystem bucket properties) [ozone]
Posted by "gardenia (via GitHub)" <gi...@apache.org>.
gardenia commented on PR #10515:
URL: https://github.com/apache/ozone/pull/10515#issuecomment-4912979859
superseded by https://github.com/apache/ozone/pull/10515
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15562. EventNotification: persistence strategy for consumer checkpoint (via Ozone filesystem bucket properties) [ozone]
Posted by "gardenia (via GitHub)" <gi...@apache.org>.
gardenia commented on PR #10515:
URL: https://github.com/apache/ozone/pull/10515#issuecomment-4912949350
> @gardenia , the progress indicator is the txID, key is "notification-checkpoint", value is txID. How about use the OM metadata table to persist this value? metadata is defined in OMDBDefinition.java, currently it's used to persist data like OM layout version, ranger ozone service version, it can perfectly save the "notification-checkpoint" -> txID data, Or we can introduce a new table to save it, just like the statefulServiceConfig table in SCM, which can be used by any feature of SCM.
>
> Second, this indicator tries to prevent OM to handle events which have already been handled before OM start or OM leader transfer, but since the event model is at least once, the receiver need to have built-in capability to handle resent event notification. So the checkpoint loading or saving fails, we should still go ahead and send events out. Checkpoint is nice to have, not mandatory, it's failure should lead to the event process task function fail. Also we need to consider if event notification feature is enabled -> checkpoint is created and saved -> event notification is disabled, who is going to clean the checkpoint. If later event notification is enabled again, what will be the impact of this uncleaned checkpoint?
>
> Third, the event framework supports multiple plugins, each plugin runs independently. The current checkpoint and #10516 are specific to Kafka. If there is another plugin loaded, this plugin may also has it's checkpoint, and it needs to coordinate with Kafka about the purge, in case one plugin moving fast, while another one moving relatively slow. It looks like the current framework doesn't support this if we really want to support multiple plugins running together.
>
> BTW, I turned https://issues.apache.org/jira/browse/HDDS-13513 into an umbrella JIRA, and converted all existing event notification JIRA as the sub JIRA of [HDDS-13513](https://issues.apache.org/jira/browse/HDDS-13513). Let's create new sub JIRAs of [HDDS-13513](https://issues.apache.org/jira/browse/HDDS-13513) in future, so that we can easily track all event notification patches under one page.
@ChenSammi I have opened a new PR which intends to implement the strategy you suggest for using RocksDB to store the checkpoint and address the other issues you mentioned (different checkpoint names per plugin and progress not stalled by a checkpoint write failure):
https://github.com/apache/ozone/pull/10689
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org