You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by kishorvpatil <gi...@git.apache.org> on 2017/10/04 20:19:24 UTC
[GitHub] storm pull request #2358: [STORM-2770] Add fragmentation metrics for CPU and...
GitHub user kishorvpatil opened a pull request:
https://github.com/apache/storm/pull/2358
[STORM-2770] Add fragmentation metrics for CPU and Memory
The patch addresses following:
- Calculate fragmentation of CPU/memory
- Display it on UI under Cluster summary
- Add following guages:
-- fragmented_memory
-- fragmented_cpu
-- available_memory
-- available_cpu
-- total_memory
-- total_cpu
- Fix num_supervisors calculations.
<img width="1431" alt="storm2770-screenshot" src="https://user-images.githubusercontent.com/6090397/31197647-bd7f147a-a91f-11e7-9566-355eff7f1349.png">
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kishorvpatil/incubator-storm storm2770
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2358.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2358
----
commit 22427d0e793dd67c16498e59f4da70ed174e6cc9
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2017-10-04T15:25:51Z
Adding cluster level fragmented CPU/Memory metrics to nimbus
commit e0e9bcb30c5d6dff8f2fe08322312374fb7d63f2
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2017-10-04T17:21:16Z
Add fragmented cpu/memory to SupervisorSummary
commit 2269e2028db6999c19c887116d0fcc95448a9a6d
Author: Kishor Patil <kp...@yahoo-inc.com>
Date: 2017-10-04T17:59:51Z
Show fragmented cpu/memory on cluster summary
Fix Configuration porting
----
---
[GitHub] storm pull request #2358: [STORM-2770] Add fragmentation metrics for CPU and...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/2358#discussion_r143374510
--- Diff: storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java ---
@@ -1825,6 +1844,11 @@ private void mkAssignments(String scratchTopoId) throws Exception {
if (!newAssignments.equals(existingAssignments)) {
LOG.debug("RESETTING id->resources and id->worker-resources cache!");
+ LOG.info("Fragmentation after scheduling is: {} MB, {} PCore CPUs", fragmentedMemory(), fragmentedCpu());
+ //map.forEach((x, y) -> System.out.println(x + ": " + y));
--- End diff --
Let's remove the comment.
---
[GitHub] storm pull request #2358: [STORM-2770] Add fragmentation metrics for CPU and...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/2358#discussion_r143374458
--- Diff: storm-server/src/main/java/org/apache/storm/scheduler/Cluster.java ---
@@ -934,7 +942,7 @@ public void setStatusMap(Map<String, String> statusMap) {
public Map<String, SupervisorResources> getSupervisorsResourcesMap() {
Map<String, SupervisorResources> ret = new HashMap<>();
for (SupervisorDetails sd : supervisors.values()) {
- ret.put(sd.getId(), new SupervisorResources(sd.getTotalMemory(), sd.getTotalMemory(), 0, 0));
+ ret.put(sd.getId(), new SupervisorResources(sd.getTotalMemory(), sd.getTotalCPU(), 0, 0));
--- End diff --
Nice catch.
---
[GitHub] storm pull request #2358: [STORM-2770] Add fragmentation metrics for CPU and...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/2358
---