You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/12/09 18:31:24 UTC
[GitHub] [ozone] smengcl opened a new pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
smengcl opened a new pull request #1680:
URL: https://github.com/apache/ozone/pull/1680
## What changes were proposed in this pull request?
1. Reduce new object creation when Recon is consuming batches OM events (`consumeOMEvents()`)
2. Improve `OMUpdateEventBatch#filter` to reduce memory usage.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4570
## How was this patch tested?
All existing UTs shall pass.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758104985
Marking this PR as ready for review.
Existing UTs should cover. No new UTs added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] arp7 edited a comment on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
arp7 edited a comment on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758779987
~@smengcl can you check if the integration test failures are related to the patch?~
Looks like @prashantpogde already checked that. @avijayanhwx does your review comment need to be addressed prior to merge? Wasn't sure since you also added a +1. 🙂
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-759621606
Thanks @prashantpogde and @avijayanhwx for reviewing. Got a green run on the latest commit. I will merge this shortly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758860212
> Here the problem is that we fetch too much DB update events to apply to recon db, right? How about doing the update event size limitation each time we get from OM? I see currently we just fetch all update events since from last sequence number.
btw just filed https://issues.apache.org/jira/browse/HDDS-4678 for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] avijayanhwx commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758863839
> ~@smengcl can you check if the integration test failures are related to the patch?~
>
> Looks like @prashantpogde already checked that. @avijayanhwx does your review comment need to be addressed prior to merge? Wasn't sure since you also added a +1. 🙂
I am OK with the approach in this patch. Just had a minor suggestion for @smengcl :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] arp7 commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
arp7 commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758779987
@smengcl can you check if the integration test failures are related to the patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] prashantpogde commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
prashantpogde commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758342436
The changes look good.
is it possible to get some stats as to how much memory footprint is reduced for some sample run (pre/post these changes) ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl merged pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl merged pull request #1680:
URL: https://github.com/apache/ozone/pull/1680
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] prashantpogde commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
prashantpogde commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758346097
Ci failures are not related with the changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-742730703
> Here the problem is that we fetch too much DB update events to apply to recon db, right? How about doing the update event size limitation each time we get from OM? I see currently we just fetch all update events since from last sequence number.
Getting smaller batches is indeed another viable approach.
The current plan is to reduce the number of array copy when Recon is processing those events. At the moment the memory usage is very bloated (may use >2.0x memory).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758833816
Thanks @prashantpogde and @arp7 for checking the CI. `TestContainerStateMachine` and `TestKeyInputStream` looks unrelated. I'm running them locally to add confidence to that.
I'm removing `ReconOmTask#getTaskTables` from the interface as @avijayanhwx suggested. Should just take another few hours to get a new CI run done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] smengcl edited a comment on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
smengcl edited a comment on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-758104985
Marking this PR as ready for review.
Existing UTs should cover the test. -- No new UT added. Fixed 2 existing UTs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] avijayanhwx commented on pull request #1680: HDDS-4570. Reduce memory footprint of OMUpdateEventBatch for Recon
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on pull request #1680:
URL: https://github.com/apache/ozone/pull/1680#issuecomment-742742345
@linyiqun A good point. Currently, we can control the number of events implicitly through the Recon-OM delta updates sync interval. We are also bound by the RPC buffer limit (~128MB). But, in the future we may add more downstream tasks in Recon to act on OM data, and we cannot scale the number of copies of the event data based on number of tasks :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org