You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by greghogan <gi...@git.apache.org> on 2016/11/29 22:05:35 UTC
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/2902
[FLINK-5209] [webfrontend] Fix TaskManager metrics
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink 5209_fix_taskmanager_metrics
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2902.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2902
----
commit 57efb3d6d7cbdf70cbc1ee6d6c1831aeb2210b36
Author: Greg Hogan <co...@greghogan.com>
Date: 2016-11-29T21:25:21Z
[FLINK-5209] [webfrontend] Fix TaskManager metrics
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90174505
--- Diff: flink-runtime-web/web-dashboard/app/partials/taskmanager/taskmanager.metrics.jade ---
@@ -75,18 +75,18 @@ div(ng-if="metrics.id")
tbody
tr
td Direct
- td {{ metrics.metrics.directCount }}
- td {{ metrics.metrics.directUsed }}
- td {{ metrics.metrics.directTotal }}
+ td {{ metrics.metrics.directCount | humanizeBytes }}
+ td {{ metrics.metrics.directUsed | humanizeBytes }}
+ td {{ metrics.metrics.directMax | humanizeBytes }}
tr
td Mapped
- td {{ metrics.metrics.mappedCount }}
- td {{ metrics.metrics.mappedUsed }}
- td {{ metrics.metrics.mappedMax }}
+ td {{ metrics.metrics.mappedCount | humanizeBytes }}
--- End diff --
The unit for none of these metrics is bytes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90272935
--- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/TaskManagersHandler.java ---
@@ -124,13 +124,21 @@ public String handleJsonRequest(Map<String, String> pathParams, Map<String, Stri
gen.writeNumberField("totalUsed", heapUsed + nonHeapUsed);
gen.writeNumberField("totalMax", heapTotal + nonHeapTotal);
- gen.writeStringField("directCount", metrics.getMetric("Status.JVM.Memory.Direct.Count", "0"));
- gen.writeStringField("directUsed", metrics.getMetric("Status.JVM.Memory.Direct.MemoryUsed", "0"));
- gen.writeStringField("directMax", metrics.getMetric("Status.JVM.Memory.Direct.TotalCapacity", "0"));
+ long directCount = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.Count", "0"));
--- End diff --
Some of the numeric metrics were passed as long and some as string. I would expect someone consuming the JSON to prefer numeric values.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90273043
--- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/TaskManagersHandler.java ---
@@ -124,13 +124,21 @@ public String handleJsonRequest(Map<String, String> pathParams, Map<String, Stri
gen.writeNumberField("totalUsed", heapUsed + nonHeapUsed);
gen.writeNumberField("totalMax", heapTotal + nonHeapTotal);
- gen.writeStringField("directCount", metrics.getMetric("Status.JVM.Memory.Direct.Count", "0"));
- gen.writeStringField("directUsed", metrics.getMetric("Status.JVM.Memory.Direct.MemoryUsed", "0"));
- gen.writeStringField("directMax", metrics.getMetric("Status.JVM.Memory.Direct.TotalCapacity", "0"));
+ long directCount = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.Count", "0"));
+ long directUsed = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.MemoryUsed", "0"));
+ long directMax = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.TotalCapacity", "0"));
- gen.writeStringField("mappedCount", metrics.getMetric("Status.JVM.Memory.Mapped.Count", "0"));
- gen.writeStringField("mappedUsed", metrics.getMetric("Status.JVM.Memory.Mapped.MemoryUsed", "0"));
- gen.writeStringField("mappedMax", metrics.getMetric("Status.JVM.Memory.Mapped.TotalCapacity", "0"));
+ gen.writeNumberField("directCount", directCount);
+ gen.writeNumberField("directUsed", directUsed);
+ gen.writeNumberField("directMax", directMax);
+
+ long mappedCount = Long.valueOf(metrics.getMetric("Status.JVM.Memory.mapped.Count", "0"));
+ long mappedUsed = Long.valueOf(metrics.getMetric("Status.JVM.Memory.mapped.MemoryUsed", "0"));
+ long mappedMax = Long.valueOf(metrics.getMetric("Status.JVM.Memory.mapped.TotalCapacity", "0"));
+
+ gen.writeNumberField("mappedCount", mappedCount);
+ gen.writeNumberField("mappedUsed", mappedUsed);
+ gen.writeNumberField("mappedMax", mappedMax);
gen.writeStringField("memorySegmentsAvailable", metrics.getMetric("Status.Network.AvailableMemorySegments", "0"));
--- End diff --
Will do ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90174966
--- Diff: flink-runtime-web/web-dashboard/app/partials/taskmanager/taskmanager.metrics.jade ---
@@ -75,18 +75,18 @@ div(ng-if="metrics.id")
tbody
tr
td Direct
- td {{ metrics.metrics.directCount }}
- td {{ metrics.metrics.directUsed }}
- td {{ metrics.metrics.directTotal }}
+ td {{ metrics.metrics.directCount | humanizeBytes }}
+ td {{ metrics.metrics.directUsed | humanizeBytes }}
+ td {{ metrics.metrics.directMax | humanizeBytes }}
tr
td Mapped
- td {{ metrics.metrics.mappedCount }}
- td {{ metrics.metrics.mappedUsed }}
- td {{ metrics.metrics.mappedMax }}
+ td {{ metrics.metrics.mappedCount | humanizeBytes }}
--- End diff --
wait; only mapped/direct-count are not measured in bytes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/2902
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90174556
--- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/TaskManagersHandler.java ---
@@ -124,13 +124,21 @@ public String handleJsonRequest(Map<String, String> pathParams, Map<String, Stri
gen.writeNumberField("totalUsed", heapUsed + nonHeapUsed);
gen.writeNumberField("totalMax", heapTotal + nonHeapTotal);
- gen.writeStringField("directCount", metrics.getMetric("Status.JVM.Memory.Direct.Count", "0"));
- gen.writeStringField("directUsed", metrics.getMetric("Status.JVM.Memory.Direct.MemoryUsed", "0"));
- gen.writeStringField("directMax", metrics.getMetric("Status.JVM.Memory.Direct.TotalCapacity", "0"));
+ long directCount = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.Count", "0"));
--- End diff --
is there a benefit to passing them as longs?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90274186
--- Diff: flink-runtime-web/web-dashboard/app/partials/taskmanager/taskmanager.metrics.jade ---
@@ -75,18 +75,18 @@ div(ng-if="metrics.id")
tbody
tr
td Direct
- td {{ metrics.metrics.directCount }}
- td {{ metrics.metrics.directUsed }}
- td {{ metrics.metrics.directTotal }}
+ td {{ metrics.metrics.directCount | humanizeBytes }}
+ td {{ metrics.metrics.directUsed | humanizeBytes }}
+ td {{ metrics.metrics.directMax | humanizeBytes }}
tr
td Mapped
- td {{ metrics.metrics.mappedCount }}
- td {{ metrics.metrics.mappedUsed }}
- td {{ metrics.metrics.mappedMax }}
+ td {{ metrics.metrics.mappedCount | humanizeBytes }}
--- End diff --
I'll add a Coffee filter to localize the numbers with commas / periods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/2902#discussion_r90175096
--- Diff: flink-runtime-web/src/main/java/org/apache/flink/runtime/webmonitor/handlers/TaskManagersHandler.java ---
@@ -124,13 +124,21 @@ public String handleJsonRequest(Map<String, String> pathParams, Map<String, Stri
gen.writeNumberField("totalUsed", heapUsed + nonHeapUsed);
gen.writeNumberField("totalMax", heapTotal + nonHeapTotal);
- gen.writeStringField("directCount", metrics.getMetric("Status.JVM.Memory.Direct.Count", "0"));
- gen.writeStringField("directUsed", metrics.getMetric("Status.JVM.Memory.Direct.MemoryUsed", "0"));
- gen.writeStringField("directMax", metrics.getMetric("Status.JVM.Memory.Direct.TotalCapacity", "0"));
+ long directCount = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.Count", "0"));
+ long directUsed = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.MemoryUsed", "0"));
+ long directMax = Long.valueOf(metrics.getMetric("Status.JVM.Memory.direct.TotalCapacity", "0"));
- gen.writeStringField("mappedCount", metrics.getMetric("Status.JVM.Memory.Mapped.Count", "0"));
- gen.writeStringField("mappedUsed", metrics.getMetric("Status.JVM.Memory.Mapped.MemoryUsed", "0"));
- gen.writeStringField("mappedMax", metrics.getMetric("Status.JVM.Memory.Mapped.TotalCapacity", "0"));
+ gen.writeNumberField("directCount", directCount);
+ gen.writeNumberField("directUsed", directUsed);
+ gen.writeNumberField("directMax", directMax);
+
+ long mappedCount = Long.valueOf(metrics.getMetric("Status.JVM.Memory.mapped.Count", "0"));
+ long mappedUsed = Long.valueOf(metrics.getMetric("Status.JVM.Memory.mapped.MemoryUsed", "0"));
+ long mappedMax = Long.valueOf(metrics.getMetric("Status.JVM.Memory.mapped.TotalCapacity", "0"));
+
+ gen.writeNumberField("mappedCount", mappedCount);
+ gen.writeNumberField("mappedUsed", mappedUsed);
+ gen.writeNumberField("mappedMax", mappedMax);
gen.writeStringField("memorySegmentsAvailable", metrics.getMetric("Status.Network.AvailableMemorySegments", "0"));
--- End diff --
If you change the above to write longs these can be changed as well, along with with the GC metrics.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2902: [FLINK-5209] [webfrontend] Fix TaskManager metrics
Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:
https://github.com/apache/flink/pull/2902
Looks good to merge, +1 :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---