You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Jasen Minton (Jira)" <ji...@apache.org> on 2021/07/23 16:49:00 UTC
[jira] [Commented] (HDDS-4839) RocksDB statistics not exposed on
Ozone DN
[ https://issues.apache.org/jira/browse/HDDS-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386371#comment-17386371 ]
Jasen Minton commented on HDDS-4839:
------------------------------------
I wrote a junit, minicluster integration-test that covers this.
Right now there's one junit test that goes through the following steps:
1 Create a key in a container
2 close the container
3 create another key in another container
4 delete the original first container.
In between each step, I query both the OM jmx url and the DN jmx urls and filter for the rocksdb entries:
[JMX. ] PHASE: START
[JMX.OM] PHASE: url:[http://0.0.0.0:45951/jmx]
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.OM] PHASE: ----
[JMX.DN] PHASE: url:[http://0.0.0.0:34735/jmx]
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.DN] PHASE: ----
[JMX. ] PHASE: FIRST KEY CREATED
[JMX.OM] PHASE: url:[http://0.0.0.0:45951/jmx]
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=1-dn-container.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.OM] PHASE: ----
[JMX.DN] PHASE: url:[http://0.0.0.0:34735/jmx]
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=1-dn-container.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.DN] PHASE: ----
[JMX. ] PHASE: FIRST CONTAINER CLOSED
[JMX.OM] PHASE: url:[http://0.0.0.0:45951/jmx]
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=1-dn-container.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.OM] PHASE: ----
[JMX.DN] PHASE: url:[http://0.0.0.0:34735/jmx]
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=1-dn-container.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.DN] PHASE: ----
[JMX. ] PHASE: SECOND KEY CREATED
[JMX.OM] PHASE: url:[http://0.0.0.0:45951/jmx]
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=1-dn-container.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=2-dn-container.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.OM] PHASE: ----
[JMX.DN] PHASE: url:[http://0.0.0.0:34735/jmx]
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=1-dn-container.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=2-dn-container.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.DN] PHASE: ----
[JMX. ] PHASE: FIRST CONTAINER DELETED - END
[JMX.OM] PHASE: url:[http://0.0.0.0:45951/jmx]
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=2-dn-container.db",
[JMX.OM] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.OM] PHASE: ----
[JMX.DN] PHASE: url:[http://0.0.0.0:34735/jmx]
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=dn-crl.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=2-dn-container.db",
[JMX.DN] PHASE: "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=scm.db",
[JMX.DN] PHASE: ----
So far everything looks as expected. I have some other scenarios to test with this still.
> RocksDB statistics not exposed on Ozone DN
> ------------------------------------------
>
> Key: HDDS-4839
> URL: https://issues.apache.org/jira/browse/HDDS-4839
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Datanode
> Affects Versions: 1.0.0
> Reporter: Wei-Chiu Chuang
> Assignee: Jasen Minton
> Priority: Major
>
> Setting ozone.metastore.rocksdb.statistics = ALL, I can find the rocksdb stats exposed in OM JMX
> {noformat}
> "name" : "Hadoop:service=Ozone,name=RocksDbStore,dbName=om.db",
> {noformat}
> but the entry is not found on DN if I try to enable this property on a DN. Initial hunch is that the tocksdb stats is created per DN, and DN has many DBs. When one DB is closed, the corresponding JMX entry is removed.
>
> This bug is similar to HDDS-4722.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org