You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/11/01 01:48:54 UTC

[GitHub] [ozone] duongkame opened a new pull request, #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

duongkame opened a new pull request, #3921:
URL: https://github.com/apache/ozone/pull/3921

   ## What changes were proposed in this pull request?
   This PR brings back the mechanism to share LRU cache between RocksDB in datanodes
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-7362
   
   ## How was this patch tested?
   
   Standard CI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] duongkame commented on pull request #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

Posted by GitBox <gi...@apache.org>.
duongkame commented on PR #3921:
URL: https://github.com/apache/ozone/pull/3921#issuecomment-1297924839

   @lokeshj1703 @ChenSammi As mentioned in the [original discussion](https://github.com/apache/ozone/pull/3673#issuecomment-1297455939), please have a look at this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] captainzmc commented on pull request #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

Posted by GitBox <gi...@apache.org>.
captainzmc commented on PR #3921:
URL: https://github.com/apache/ozone/pull/3921#issuecomment-1300464712

   Hi @ChenSammi, If this PR can reduce the problem of too many LRUCache objects in DN. I think we can cherry-pick it to branch-1.3.  If there is no problem I will turn off revert PR HDDS-7442.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] DaveTeng0 commented on pull request #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

Posted by GitBox <gi...@apache.org>.
DaveTeng0 commented on PR #3921:
URL: https://github.com/apache/ozone/pull/3921#issuecomment-1297921062

   hmm just for myself learning~ when is the timing a DatanodeDBProfile object being created?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] ChenSammi merged pull request #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

Posted by GitBox <gi...@apache.org>.
ChenSammi merged PR #3921:
URL: https://github.com/apache/ozone/pull/3921


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] duongkame commented on pull request #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

Posted by GitBox <gi...@apache.org>.
duongkame commented on PR #3921:
URL: https://github.com/apache/ozone/pull/3921#issuecomment-1301101845

   > Thanks @duongkame for the fix. The change make sense. In addition, Did you check the dump of DN? Just make sure if there are still many LRUCache objects.
   
   Yes, I did. Please see below:
   ```
   duong@fubar ~ % jmap -histo:live 81172 | grep Managed
    144:           104           7488  org.apache.hadoop.hdds.utils.db.managed.ManagedColumnFamilyOptions
    388:             4           1056  org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelImpl
    400:            21           1008  org.apache.hadoop.hdds.utils.db.managed.ManagedDBOptions
    548:             4            544  org.apache.hadoop.hdds.utils.db.managed.ManagedBlockBasedTableConfig
    572:            21            504  org.apache.hadoop.hdds.utils.db.managed.ManagedCheckpoint
    573:            21            504  org.apache.hadoop.hdds.utils.db.managed.ManagedRocksDB
    574:            21            504  org.apache.hadoop.hdds.utils.db.managed.ManagedWriteOptions
   1340:             4             96  org.apache.hadoop.hdds.utils.db.managed.ManagedBloomFilter
   1341:             4             96  org.apache.hadoop.hdds.utils.db.managed.ManagedLRUCache
   1398:             4             96  org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelImpl$1ChannelCallTracerFactory
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


[GitHub] [ozone] ChenSammi commented on pull request #3921: HDDS-7362. Bring back the mechanism to share LRU cache between RocksDB in datanodes

Posted by GitBox <gi...@apache.org>.
ChenSammi commented on PR #3921:
URL: https://github.com/apache/ozone/pull/3921#issuecomment-1302929454

   LGTM + 1.  Thanks @duongkame .


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org