You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/10/24 11:56:03 UTC
[GitHub] [ozone] myskov commented on a diff in pull request #3875: HDDS-7183. Expose RocksDB critical properties.
myskov commented on code in PR #3875:
URL: https://github.com/apache/ozone/pull/3875#discussion_r1003225424
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreMBean.java:
##########
@@ -62,21 +71,70 @@ public class RocksDBStoreMBean implements DynamicMBean, MetricsSource {
public static final String ROCKSDB_CONTEXT_PREFIX = "Rocksdb_";
- public RocksDBStoreMBean(Statistics statistics, String dbName) {
+ // RocksDB properties
+ String [][] cfPros = {
+ // 1 if a memtable flush is pending; otherwise, returns 0
+ {"mem-table-flush-pending", "false"},
+ // estimated total number of bytes compaction needs to rewrite to get
+ // all levels down to under target size.
+ {"estimate-pending-compaction-bytes", "true"},
+ // 1 if at least one compaction is pending; otherwise, returns 0.
+ {"compaction-pending", "false"},
+ // block cache capacity
+ {"block-cache-capacity", "true"},
+ // the memory size for the entries residing in block cache
+ {"block-cache-usage", "true"},
+ // the memory size for the entries being pinned
+ {"block-cache-pinned-usage", "true"},
+ // number of level to which L0 data will be compacted.
+ {"base-level", "false"},
+ // approximate active mem table size (bytes)
+ {"cur-size-active-mem-table", "true"},
+ // approximate size of active and unflushed immutable (bytes)
+ {"cur-size-all-mem-tables", "true"},
+ // approximate size of active, unflushed immutable, and pinned immutable
+ // memtables (bytes)
+ {"size-all-mem-tables", "true"},
+ // number of immutable memtables that have not yet been flushed
+ {"num-immutable-mem-table", "true"},
+ // total size (bytes) of all SST files belong to the latest LSM tree
+ {"live-sst-files-size", "true"},
+ // estimated number of total keys in memtables and storage(Can be very
+ // wrong according to RocksDB document)
+ {"estimate-num-keys", "true"},
+ // estimated memory used for reading SST tables, excluding memory used
+ // in block cache (e.g., filter and index blocks)
+ {"estimate-table-readers-mem", "true"}
+ };
+
+ // level-x sst file info (Global)
+ String numFilesPerLevel = "num_files_at_level";
Review Comment:
look like a constant. Should be something linke private final static ROCKSDB_PROP_NUM_FILES_PER_LEVEL
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreMBean.java:
##########
@@ -62,21 +71,70 @@ public class RocksDBStoreMBean implements DynamicMBean, MetricsSource {
public static final String ROCKSDB_CONTEXT_PREFIX = "Rocksdb_";
- public RocksDBStoreMBean(Statistics statistics, String dbName) {
+ // RocksDB properties
+ String [][] cfPros = {
+ // 1 if a memtable flush is pending; otherwise, returns 0
+ {"mem-table-flush-pending", "false"},
+ // estimated total number of bytes compaction needs to rewrite to get
+ // all levels down to under target size.
+ {"estimate-pending-compaction-bytes", "true"},
+ // 1 if at least one compaction is pending; otherwise, returns 0.
+ {"compaction-pending", "false"},
+ // block cache capacity
+ {"block-cache-capacity", "true"},
+ // the memory size for the entries residing in block cache
+ {"block-cache-usage", "true"},
+ // the memory size for the entries being pinned
+ {"block-cache-pinned-usage", "true"},
+ // number of level to which L0 data will be compacted.
+ {"base-level", "false"},
+ // approximate active mem table size (bytes)
+ {"cur-size-active-mem-table", "true"},
+ // approximate size of active and unflushed immutable (bytes)
+ {"cur-size-all-mem-tables", "true"},
+ // approximate size of active, unflushed immutable, and pinned immutable
+ // memtables (bytes)
+ {"size-all-mem-tables", "true"},
+ // number of immutable memtables that have not yet been flushed
+ {"num-immutable-mem-table", "true"},
+ // total size (bytes) of all SST files belong to the latest LSM tree
+ {"live-sst-files-size", "true"},
+ // estimated number of total keys in memtables and storage(Can be very
+ // wrong according to RocksDB document)
+ {"estimate-num-keys", "true"},
+ // estimated memory used for reading SST tables, excluding memory used
+ // in block cache (e.g., filter and index blocks)
+ {"estimate-table-readers-mem", "true"}
+ };
+
+ // level-x sst file info (Global)
+ String numFilesPerLevel = "num_files_at_level";
+ String sizePerLevel = "size_at_level";
+
+ // To be Prometheus metric name format
+ String [] formattedCFProps;
+
+ public RocksDBStoreMBean(Statistics statistics, RocksDatabase db,
+ String dbName) {
this.contextName = ROCKSDB_CONTEXT_PREFIX + dbName;
this.statistics = statistics;
+ this.rocksDB = db;
histogramAttributes.add("Average");
histogramAttributes.add("Median");
histogramAttributes.add("Percentile95");
histogramAttributes.add("Percentile99");
histogramAttributes.add("StandardDeviation");
+ histogramAttributes.add("Max");
+
+ List<String> tempList = new ArrayList<>();
+ Arrays.stream(cfPros).forEach(p -> tempList.add(p[0].replace("-", "_")));
+ formattedCFProps = tempList.toArray(new String[0]);
Review Comment:
No need for additional tempList(). Can be done with
formattedCFProps = Arrays.stream(cfPros).map(x -> x[0].replace("-", "_")).toArray(new String[0])
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreMBean.java:
##########
@@ -53,6 +60,8 @@ public class RocksDBStoreMBean implements DynamicMBean, MetricsSource {
private Statistics statistics;
+ private RocksDatabase rocksDB;
Review Comment:
can be final
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org