You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "weimingdiit (via GitHub)" <gi...@apache.org> on 2024/08/24 09:46:04 UTC
[PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
weimingdiit opened a new pull request, #7117:
URL: https://github.com/apache/ozone/pull/7117
## What changes were proposed in this pull request?
Make RocksDB cache configurable in Ozone Manager.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11230
## How was this patch tested?
unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #7117:
URL: https://github.com/apache/ozone/pull/7117#discussion_r1742196007
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -159,6 +159,16 @@ public final class OzoneConfigKeys {
public static final String
OZONE_METADATA_STORE_ROCKSDB_CF_WRITE_BUFFER_SIZE_DEFAULT = "128MB";
+ public static final String OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE =
+ "ozone.metastore.rocksdb.block.cache.size";
+ public static final String
+ OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE_DEFAULT = "256MB";
Review Comment:
That seems fine to me. I don't have data on the size of serialized proto values but that could help approximate the number of keys the cache would hold.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2319741029
> Hi @weimingdiit thanks for looking in to this. Maybe it would be better to support RocksDB config files directly instead of needing to expose each config through the Ozone configuration? See the [RocksDB docs](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) on this for reference. We could first load the options set by the user in this file, and then override any in Ozone that must be set a certain way, for example creating column families that don't exist.
Thank you very much for your comments，this is a good idea. However, for the current version, could we first make some of the more performance-critical parameters configurable, such as the ones I mentioned? I will continue to follow up on this issue and create a new issue to directly support RocksDB configuration files, so we don't have to rely on Ozone configuration. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on code in PR #7117:
URL: https://github.com/apache/ozone/pull/7117#discussion_r1743143822
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -159,6 +159,16 @@ public final class OzoneConfigKeys {
public static final String
OZONE_METADATA_STORE_ROCKSDB_CF_WRITE_BUFFER_SIZE_DEFAULT = "128MB";
+ public static final String OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE =
+ "ozone.metastore.rocksdb.block.cache.size";
+ public static final String
+ OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE_DEFAULT = "256MB";
Review Comment:
@kerneltime The default size has been updated to 1GB, thanks for your comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime closed pull request #7117: HDDS-11230. Make RocksDB cache configurable in Ozone Manager.
URL: https://github.com/apache/ozone/pull/7117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on code in PR #7117:
URL: https://github.com/apache/ozone/pull/7117#discussion_r1737695377
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -159,6 +159,16 @@ public final class OzoneConfigKeys {
public static final String
OZONE_METADATA_STORE_ROCKSDB_CF_WRITE_BUFFER_SIZE_DEFAULT = "128MB";
+ public static final String OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE =
+ "ozone.metastore.rocksdb.block.cache.size";
+ public static final String
+ OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE_DEFAULT = "256MB";
Review Comment:
Indeed, this parameter should be larger. For larger tables, 1 to 2 GB is a suitable default value. I will update this parameter. Hi, @errose28 What do you think about it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2326723957
I'm ok with adding this field in the Ozone configuration to start if there are no immediate plans to support the full RocksDB INI file. We can file that as a follow up Jira. In the future I do think we should support the INI file instead of propagating every RocksDB config we might need into the Ozone configuration. If we start getting into serious RocksDB tuning then there will be too many options in RocksDB for that approach to be sustainable long term.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on code in PR #7117:
URL: https://github.com/apache/ozone/pull/7117#discussion_r1742196007
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -159,6 +159,16 @@ public final class OzoneConfigKeys {
public static final String
OZONE_METADATA_STORE_ROCKSDB_CF_WRITE_BUFFER_SIZE_DEFAULT = "128MB";
+ public static final String OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE =
+ "ozone.metastore.rocksdb.block.cache.size";
+ public static final String
+ OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE_DEFAULT = "256MB";
Review Comment:
That seems fine to me. I don't have data on the size of serialized proto values but which could help approximate the number of keys the cache would hold.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit closed pull request #7117: HDDS-11230. Make RocksDB cache configurable in Ozone Manager.
URL: https://github.com/apache/ozone/pull/7117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "kerneltime (via GitHub)" <gi...@apache.org>.
kerneltime commented on code in PR #7117:
URL: https://github.com/apache/ozone/pull/7117#discussion_r1735351972
##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java:
##########
@@ -159,6 +159,16 @@ public final class OzoneConfigKeys {
public static final String
OZONE_METADATA_STORE_ROCKSDB_CF_WRITE_BUFFER_SIZE_DEFAULT = "128MB";
+ public static final String OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE =
+ "ozone.metastore.rocksdb.block.cache.size";
+ public static final String
+ OZONE_METADATA_STORE_ROCKSDB_BLOCK_CACHE_SIZE_DEFAULT = "256MB";
Review Comment:
The default needs to be larger, 1 or 2 GB perhaps?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "errose28 (via GitHub)" <gi...@apache.org>.
errose28 commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2310925940
Hi @weimingdiit thanks for looking in to this. Maybe it would be better to support RocksDB config files directly instead of needing to expose each config through the Ozone configuration? See the [RocksDB docs](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) on this for reference. We could first load the options set by the user in this file, and then override any in Ozone that must be set a certain way, for example creating column families that don't exist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2330516102
> I'm ok with adding this field in the Ozone configuration to start if there are no immediate plans to support the full RocksDB INI file. We can file that as a follow up Jira. In the future I do think we should support the INI file instead of propagating every RocksDB config we might need into the Ozone configuration. If we start getting into serious RocksDB tuning then there will be too many options in RocksDB for that approach to be sustainable long term.
@errose28 I will close this PR, and then make all RocksDB parameters configurable in https://github.com/apache/ozone/pull/7150
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2330504352
> I'm ok with adding this field in the Ozone configuration to start if there are no immediate plans to support the full RocksDB INI file. We can file that as a follow up Jira. In the future I do think we should support the INI file instead of propagating every RocksDB config we might need into the Ozone configuration. If we start getting into serious RocksDB tuning then there will be too many options in RocksDB for that approach to be sustainable long term.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2325800357
> > Hi @weimingdiit thanks for looking in to this. Maybe it would be better to support RocksDB config files directly instead of needing to expose each config through the Ozone configuration? See the [RocksDB docs](https://github.com/facebook/rocksdb/wiki/RocksDB-Options-File) on this for reference. We could first load the options set by the user in this file, and then override any in Ozone that must be set a certain way, for example creating column families that don't exist.
>
> @errose28 Thank you very much for your comments，this is a good idea. However, for the current version, could we first make some of the more performance-critical parameters configurable, such as the ones I mentioned?
I thought about it, maybe it would be more convenient for us to manage it through ozone configuration, so that our configuration will have only one file, which is easier to manage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2309225950
There are some test errors, I will update the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-11230. Make RocksDB cache configurable in Ozone Manager. [ozone]
Posted by "weimingdiit (via GitHub)" <gi...@apache.org>.
weimingdiit commented on PR #7117:
URL: https://github.com/apache/ozone/pull/7117#issuecomment-2325517492
@jojochuang Could you help review this PR? This PR is related to some caching in RocksDB. My current thought is to keep the status quo unchanged in this PR and then create a new issue to make all RocksDB parameters configurable. I'd like to hear your thoughts on this—what do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org