You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@geode.apache.org by Jacob Barrett <ja...@vmware.com> on 2020/07/08 16:24:36 UTC
Re: API (Recommanded way) to get heap and disk usage for cluster
nodes
Steve,
Geode is in a transition from its on disk proprietary stats format to utilizing Micrometer.io<http://Micrometer.io>. Some of what you are looking for may already be exposed via Micrometer. If so you can just use whatever registry of your choice to publish those stats. If the metric you need is not converted to Micrometer its pretty easy in most cases to refactor it and we would welcome a JIRA or even better a PR.
-Jake
On Jul 7, 2020, at 9:58 PM, steve mathew <st...@gmail.com>> wrote:
Hello Geode Dev and users
We have a requirement to constantly monitor the resource utilization (Disk
and Heap usage) for the cluster nodes from external processes.
Seeking help to understand the recommended way (or APIs available ) to get
this in a separate process...We need to trigger some actions/custom logic
if it goes above some threshold..
Thanks
Steve.
Re: API (Recommanded way) to get heap and disk usage for cluster
nodes
Posted by Barrett Oglesby <bo...@vmware.com>.
Steve,
Here are some ways to access these statistics using JMX.
To access JVM metrics for a member including heap usage using JMX, invoke the MemberMXBean showJVMMetrics operation.
These values are gotten from ManagementFactory.getMemoryMXBean(), ManagementFactory.getThreadMXBean() and ManagementFactory.getGarbageCollectorMXBeans().
The DiskStoreMXBean attributes contain the disk metrics you're looking for (including TotalBytesOnDisk). These are per DiskStore per member.
I attached a couple jconsole charts showing these.
jconsole_jvm_metrics.png - shows a member's JVMMetrics
jconsole_diskstore_attributes.png - shows a DiskStore's attributes
I also attached a java client that dumps the JVM metrics, but it can easily be changed to periodically get them and take action depending on the state.
java DumpJvmMetrics localhost 1091
...
=======================================================
GemFire:type=Member,member=server-1
=======================================================
JVM Metrics:
committedMemory->518979584
gcCount->0
gcTimeMillis->31
initMemory->536870912
maxMemory->8502706176
totalThreads->60
usedMemory->145895584
...
I also attached a java client that dumps the DiskStore attributes, but it can easily be changed to periodically get them and take action depending on the state.
java DumpDiskStores localhost 1091
...
=======================================================
GemFire:service=DiskStore,name=disk_store_1,type=Member,member=server-1
=======================================================
Disk Store:
attribute=Name; value=disk_store_1
attribute=DiskReadsRate; value=0.0
attribute=DiskWritesRate; value=0.0
attribute=TotalBackupInProgress; value=0
attribute=TotalBackupCompleted; value=0
attribute=ForceCompactionAllowed; value=false
attribute=MaxOpLogSize; value=10
attribute=DiskDirectories; value=[/path/to/server-1/.]
attribute=DiskReadsAvgLatency; value=0
attribute=DiskWritesAvgLatency; value=0
attribute=FlushTimeAvgLatency; value=0
attribute=TotalRecoveriesInProgress; value=0
attribute=TimeInterval; value=1000
attribute=AutoCompact; value=true
attribute=CompactionThreshold; value=50
attribute=WriteBufferSize; value=32768
attribute=DiskUsageWarningPercentage; value=90.0
attribute=DiskUsageCriticalPercentage; value=99.0
attribute=QueueSize; value=0
attribute=TotalQueueSize; value=0
attribute=TotalBytesOnDisk; value=2216241
attribute=DiskUsagePercentage; value=-1.0
attribute=DiskFreePercentage; value=-1.0
...
Thanks,
Barry
________________________________
From: steve mathew <st...@gmail.com>
Sent: Wednesday, July 8, 2020 11:20 AM
To: dev@geode.apache.org <de...@geode.apache.org>
Cc: user@geode.apache.org <us...@geode.apache.org>
Subject: Re: API (Recommanded way) to get heap and disk usage for cluster nodes
Thanks Jacob and Anthony for sharing the details.
I have tried to understand list_of_mbeans supported but finding it tough to
understand completely. I can see "DiskStoreMXBean" and document says it can
provide region(s) specific disk usage.
For my experiment, looking for *mbeans that provide data node's (member)
specific heap and disk usage*.. It would be great help if someone can guide
me about these MBeans and how to use it to get the required stats (or point
me some reference outlining this details.)
Thanks
-Steve
On Wed, Jul 8, 2020 at 10:27 PM Anthony Baker <ba...@vmware.com> wrote:
> Another option is JMX, see
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgeode.apache.org%2Fdocs%2Fguide%2F19%2Fmanaging%2Fmanagement%2Flist_of_mbeans.html&data=02%7C01%7Cboglesby%40vmware.com%7C2bb9bdb99af64d9f6cfb08d8236ba39c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637298292514607524&sdata=ArIBcLD2FLtBkup6bI7tNlwTWN0GBkc1Z1kOIdzypnU%3D&reserved=0
> .
>
> Anthony
>
>
> On Jul 8, 2020, at 9:24 AM, Jacob Barrett <jabarrett@vmware.com<mailto:
> jabarrett@vmware.com>> wrote:
>
> Steve,
>
> Geode is in a transition from its on disk proprietary stats format to
> utilizing Micrometer.io<
> https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmicrometer.io%2F&data=02%7C01%7Cboglesby%40vmware.com%7C2bb9bdb99af64d9f6cfb08d8236ba39c%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637298292514607524&sdata=bcAudAbLBnF1nySFevDnykilL3IOtRFwSuVy5C0jdkc%3D&reserved=0>.
> Some of what you are looking for may already be exposed via Micrometer. If
> so you can just use whatever registry of your choice to publish those
> stats. If the metric you need is not converted to Micrometer its pretty
> easy in most cases to refactor it and we would welcome a JIRA or even
> better a PR.
>
> -Jake
>
>
> On Jul 7, 2020, at 9:58 PM, steve mathew <steve.mathew47@gmail.com<mailto:
> steve.mathew47@gmail.com>> wrote:
>
> Hello Geode Dev and users
>
> We have a requirement to constantly monitor the resource utilization (Disk
> and Heap usage) for the cluster nodes from external processes.
> Seeking help to understand the recommended way (or APIs available ) to get
> this in a separate process...We need to trigger some actions/custom logic
> if it goes above some threshold..
>
> Thanks
> Steve.
>
>
>
Re: API (Recommanded way) to get heap and disk usage for cluster nodes
Posted by steve mathew <st...@gmail.com>.
Thanks Jacob and Anthony for sharing the details.
I have tried to understand list_of_mbeans supported but finding it tough to
understand completely. I can see "DiskStoreMXBean" and document says it can
provide region(s) specific disk usage.
For my experiment, looking for *mbeans that provide data node's (member)
specific heap and disk usage*.. It would be great help if someone can guide
me about these MBeans and how to use it to get the required stats (or point
me some reference outlining this details.)
Thanks
-Steve
On Wed, Jul 8, 2020 at 10:27 PM Anthony Baker <ba...@vmware.com> wrote:
> Another option is JMX, see
> https://geode.apache.org/docs/guide/19/managing/management/list_of_mbeans.html
> .
>
> Anthony
>
>
> On Jul 8, 2020, at 9:24 AM, Jacob Barrett <jabarrett@vmware.com<mailto:
> jabarrett@vmware.com>> wrote:
>
> Steve,
>
> Geode is in a transition from its on disk proprietary stats format to
> utilizing Micrometer.io<
> https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmicrometer.io%2F&data=02%7C01%7Cbakera%40vmware.com%7C839050aa773449021fec08d8235b6e94%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637298222885245763&sdata=akhmzrYaO3ARYP5mV6vhSAjcoudT1kp1QLeJVCjTdFM%3D&reserved=0>.
> Some of what you are looking for may already be exposed via Micrometer. If
> so you can just use whatever registry of your choice to publish those
> stats. If the metric you need is not converted to Micrometer its pretty
> easy in most cases to refactor it and we would welcome a JIRA or even
> better a PR.
>
> -Jake
>
>
> On Jul 7, 2020, at 9:58 PM, steve mathew <steve.mathew47@gmail.com<mailto:
> steve.mathew47@gmail.com>> wrote:
>
> Hello Geode Dev and users
>
> We have a requirement to constantly monitor the resource utilization (Disk
> and Heap usage) for the cluster nodes from external processes.
> Seeking help to understand the recommended way (or APIs available ) to get
> this in a separate process...We need to trigger some actions/custom logic
> if it goes above some threshold..
>
> Thanks
> Steve.
>
>
>
Re: API (Recommanded way) to get heap and disk usage for cluster
nodes
Posted by Anthony Baker <ba...@vmware.com>.
Another option is JMX, see https://geode.apache.org/docs/guide/19/managing/management/list_of_mbeans.html.
Anthony
On Jul 8, 2020, at 9:24 AM, Jacob Barrett <ja...@vmware.com>> wrote:
Steve,
Geode is in a transition from its on disk proprietary stats format to utilizing Micrometer.io<https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmicrometer.io%2F&data=02%7C01%7Cbakera%40vmware.com%7C839050aa773449021fec08d8235b6e94%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637298222885245763&sdata=akhmzrYaO3ARYP5mV6vhSAjcoudT1kp1QLeJVCjTdFM%3D&reserved=0>. Some of what you are looking for may already be exposed via Micrometer. If so you can just use whatever registry of your choice to publish those stats. If the metric you need is not converted to Micrometer its pretty easy in most cases to refactor it and we would welcome a JIRA or even better a PR.
-Jake
On Jul 7, 2020, at 9:58 PM, steve mathew <st...@gmail.com>> wrote:
Hello Geode Dev and users
We have a requirement to constantly monitor the resource utilization (Disk
and Heap usage) for the cluster nodes from external processes.
Seeking help to understand the recommended way (or APIs available ) to get
this in a separate process...We need to trigger some actions/custom logic
if it goes above some threshold..
Thanks
Steve.
Re: API (Recommanded way) to get heap and disk usage for cluster
nodes
Posted by Anthony Baker <ba...@vmware.com>.
Another option is JMX, see https://geode.apache.org/docs/guide/19/managing/management/list_of_mbeans.html.
Anthony
On Jul 8, 2020, at 9:24 AM, Jacob Barrett <ja...@vmware.com>> wrote:
Steve,
Geode is in a transition from its on disk proprietary stats format to utilizing Micrometer.io<https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmicrometer.io%2F&data=02%7C01%7Cbakera%40vmware.com%7C839050aa773449021fec08d8235b6e94%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637298222885245763&sdata=akhmzrYaO3ARYP5mV6vhSAjcoudT1kp1QLeJVCjTdFM%3D&reserved=0>. Some of what you are looking for may already be exposed via Micrometer. If so you can just use whatever registry of your choice to publish those stats. If the metric you need is not converted to Micrometer its pretty easy in most cases to refactor it and we would welcome a JIRA or even better a PR.
-Jake
On Jul 7, 2020, at 9:58 PM, steve mathew <st...@gmail.com>> wrote:
Hello Geode Dev and users
We have a requirement to constantly monitor the resource utilization (Disk
and Heap usage) for the cluster nodes from external processes.
Seeking help to understand the recommended way (or APIs available ) to get
this in a separate process...We need to trigger some actions/custom logic
if it goes above some threshold..
Thanks
Steve.