[PR] Parallelize CreateTableIT.testCreateLotsOfTables() to speed it up [accumulo]

["DomGarguilo (via GitHub)" <gi...@apache.org>]

DomGarguilo opened a new pull request, #5534:
URL: https://github.com/apache/accumulo/pull/5534

   This test times out pretty consistently when resource constrained. The following improvement was made to speed the test up:
   * use `Arrays.stream(tableNames).parallel().forEach()` instead of for loop to parallelize the test
   
   Other improvements that don't necessarily improve runtime:
   * use `Timer` objects instead of System.millis for timekeeping
   * use `Logger` instead of System.out
   
   I assume this test was not created explicitly to run serially. When run locally this speeds runtime up from ~1 min to ~15 seconds.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Parallelize CreateTableIT.testCreateLotsOfTables() to speed it up [accumulo]

["DomGarguilo (via GitHub)" <gi...@apache.org>]

DomGarguilo merged PR #5534:
URL: https://github.com/apache/accumulo/pull/5534


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Parallelize CreateTableIT.testCreateLotsOfTables() to speed it up [accumulo]

["dlmarion (via GitHub)" <gi...@apache.org>]

dlmarion commented on code in PR #5534:
URL: https://github.com/apache/accumulo/pull/5534#discussion_r2073794879


##########
test/src/main/java/org/apache/accumulo/test/CreateTableIT.java:
##########
@@ -47,20 +57,22 @@ public static void teardown() {
   }
 
   @Test
-  public void testCreateLotsOfTables() throws Exception {
+  public void testCreateLotsOfTables() {
     try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+      String[] tableNames = getUniqueNames(NUM_TABLES);
 
-      String[] tableNames = getUniqueNames(500);
+      Arrays.stream(tableNames).parallel().forEach(name -> {
+        Timer timer = Timer.startNew();
+        try {
+          client.tableOperations().create(name);
+        } catch (AccumuloException | AccumuloSecurityException | TableExistsException e) {
+          throw new RuntimeException(e);
+        }
+        log.info("Table {} creation took: {} ms", name, timer.elapsed(TimeUnit.MILLISECONDS));
+      });
 
-      for (int i = 0; i < tableNames.length; i++) {
-        // Create waits for the Fate operation to complete
-        long start = System.currentTimeMillis();
-        client.tableOperations().create(tableNames[i]);
-        System.out.println("Table creation took: " + (System.currentTimeMillis() - start) + "ms");
-      }
-      // Confirm all 500 user tables exist in addition to Root, Metadata,
-      // and ScanRef tables
-      assertEquals(503, client.tableOperations().list().size());
+      final int systemTables = 3;

Review Comment:
   On merge to main will need to change this to use `SystemTables`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org