[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #284: HDDS-2653. Improve executor memory usage in new Freon tests

[GitBox <gi...@apache.org>]

xiaoyuyao commented on a change in pull request #284: HDDS-2653. Improve executor memory usage in new Freon tests
URL: https://github.com/apache/hadoop-ozone/pull/284#discussion_r353939273
 
 

 ##########
 File path: hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/BaseFreonGenerator.java
 ##########
 @@ -101,71 +101,102 @@
   private MetricRegistry metrics = new MetricRegistry();
 
   private AtomicLong successCounter;
-
   private AtomicLong failureCounter;
+  private AtomicLong attemptCounter;
 
   private long startTime;
 
   private PathSchema pathSchema;
+  private String spanName;
+  private ExecutorService executor;
+  private ProgressBar progressBar;
 
   /**
    * The main logic to execute a test generator.
    *
    * @param provider creates the new steps to execute.
    */
   public void runTests(TaskProvider provider) {
+    setup(provider);
+    startTaskRunners(provider);
+    waitForCompletion();
+    shutdown();
+    reportAnyFailure();
+  }
 
-    ExecutorService executor = Executors.newFixedThreadPool(threadNo);
-
-    ProgressBar progressBar =
-        new ProgressBar(System.out, testNo, successCounter::get);
-    progressBar.start();
-
-    startTime = System.currentTimeMillis();
-    //schedule the execution of all the tasks.
-
-    for (long i = 0; i < testNo; i++) {
-
-      final long counter = i;
+  /**
+   * Performs {@code provider}-specific initialization.
+   */
+  private void setup(TaskProvider provider) {
+    //provider is usually a lambda, print out only the owner class name:
+    spanName = provider.getClass().getSimpleName().split("\\$")[0];
+  }
 
-      //provider is usually a lambda, print out only the owner class name:
-      String spanName = provider.getClass().getSimpleName().split("\\$")[0];
+  /**
+   * Launches {@code threadNo} task runners in executor.  Each one executes test
+   * tasks in a loop until completion or failure.
+   */
+  private void startTaskRunners(TaskProvider provider) {
+    for (int i = 0; i < threadNo; i++) {
 
 Review comment:
   NIT: threadNo is confusing here as we are submit task with no guaranteed # of threads?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org