You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "szetszwo (via GitHub)" <gi...@apache.org> on 2023/04/20 14:51:57 UTC
[GitHub] [ozone] szetszwo commented on pull request #4572: HDDS-8427. Optimize multipartUpload memory usage
szetszwo commented on PR #4572:
URL: https://github.com/apache/ozone/pull/4572#issuecomment-1516471425
@xichen01 , thanks for working on this!
- Because of the underlying O(n^2) problem, this memory improvement is minor since it does not reduce the size or the number of the `PartKeyInfo` objects (~1.5 KB each).
- For 10,000 parts, there are 50,005,000 `PartKeyInfo` objects as you mentioned, and the size is ~71GB. The improvement here is 1.7GB (=2.17GB - 491MB), which is around 2.3% (=1.7/(71 + 2.17)).
- Once the O(n^2) problem has been fixed, the `TreeMap` problem in this pull request will disappear.
In the change here, you have avoided copying a `TreeMap` in a `OmMultipartKeyInfo` constructor. How about keep the `TreeMap` and avoid the copying, but not adding `IntSparseMap`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org