[GitHub] [storm] Ethanlm commented on a change in pull request #3217: STORM-3590: Adds test to validate that GRAS's node sort is stable and…

[GitBox <gi...@apache.org>]

Ethanlm commented on a change in pull request #3217: STORM-3590: Adds test to validate that GRAS's node sort is stable and…
URL: https://github.com/apache/storm/pull/3217#discussion_r402631134
 
 

 ##########
 File path: storm-server/src/test/java/org/apache/storm/scheduler/resource/strategies/scheduling/TestGenericResourceAwareStrategy.java
 ##########
 @@ -300,7 +302,64 @@ public void testGrasRequiringEviction() {
         assertTopologiesFullyScheduled(cluster, noGpu);
         assertTopologiesFullyScheduled(cluster, gpu2);
     }
-    
+
+    /*
+     * test GRAS should not evict other topologies for the sake of satisfying generic resource requirements.
+     * GRAS should also not resort nodes and 'pack' a node instead of spreading it out causing
+     * fragmentation.
+     */
+    @Test
+    public void testGrasStarvation() {
+        double cpuPercent = 200;
+        double memoryOnHeap = 100;
+        double memoryOffHeap = 100;
+
+        TopologyBuilder builder = new TopologyBuilder();
+
+        // non-gpu topology
+        builder.setSpout("spout", new TestSpout(), 2);
+        StormTopology stormTopologyNoGpu = builder.createTopology();
+        String noGpu = "hasNoGpu";
+        List<TopologyDetails> topologyDetails = new ArrayList();
+        Config nonGpuconf = createClusterConfig(cpuPercent, memoryOnHeap, memoryOffHeap, null);
+        nonGpuconf.put(Config.TOPOLOGY_SCHEDULER_STRATEGY, DefaultResourceAwareStrategy.class.getName());
+
+        cpuPercent = 100;
+
+        // gpu topology (requires 4 gpu's in total)
+        builder = new TopologyBuilder();
+        builder.setSpout("spout", new TestSpout(), 4).addResource("gpu.count", 1.0);
+        StormTopology stormTopologyWithGpu = builder.createTopology();
+
+
+        Config conf = createGrasClusterConfig(cpuPercent, memoryOnHeap, memoryOffHeap, null, Collections.emptyMap());
+        conf.put(Config.TOPOLOGY_SCHEDULER_STRATEGY, GenericResourceAwareStrategy.class.getName());
+
+        String gpu = "hasGpu";
+        Topologies topologies = new Topologies(createTestStormTopology(stormTopologyWithGpu, 9, gpu, conf));
+
+        Map<String, Double> genericResourcesMap = new HashMap<>();
+        genericResourcesMap.put("gpu.count", 2.0);
+        Map<String, SupervisorDetails> supMap = genSupervisors(4, 4, 200, 2000, genericResourcesMap);
+        Cluster cluster = new Cluster(new INimbusTest(), new ResourceMetrics(new StormMetricsRegistry()), supMap, new HashMap<>(), topologies, conf);
+
+        // should schedule gpu and noGpu successfully
+        scheduler = new ResourceAwareScheduler();
+        nonGpuconf.put(DaemonConfig.RESOURCE_AWARE_SCHEDULER_MAX_TOPOLOGY_SCHEDULING_ATTEMPTS, 1); // allows no evictions
 
 Review comment:
   This is not able to indicate if eviction happens or not.
   Waiting on https://github.com/apache/storm/pull/3213 so we can have a right way to detect eviction. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services