[GitHub] [incubator-ratis] szetszwo commented on a change in pull request #320: RATIS-1201. Enhance FileStore steaming tests

[GitBox <gi...@apache.org>]

szetszwo commented on a change in pull request #320:
URL: https://github.com/apache/incubator-ratis/pull/320#discussion_r535888965



##########
File path: ratis-examples/src/test/java/org/apache/ratis/examples/filestore/FileStoreStreamingBaseTest.java
##########
@@ -84,36 +110,62 @@ private void testSingleFile(
       IOException> newClient)
       throws Exception {
     LOG.info("runTestSingleFile with path={}, fileLength={}", path, fileLength);
-    final int size = fileLength.getSizeInt();
-    try (FileStoreClient client = newClient.get()) {
-      final DataStreamOutput dataStreamOutput = client.getStreamOutput(path, size);
-      final List<CompletableFuture<DataStreamReply>> futures = new ArrayList<>();
-      final List<Integer> sizes = new ArrayList<>();
-
-      for(int offset = 0; offset < size; ) {
-        final int remaining = size - offset;
-        final int length = Math.min(remaining, bufferSize);
-        final boolean close = length == remaining;
-
-        LOG.trace("write {}, offset={}, length={}, close? {}",
-            path, offset, length, close);
-        final ByteBuffer bf = DataStreamTestUtils.initBuffer(0, length);
-        futures.add(dataStreamOutput.writeAsync(bf, close));
-        sizes.add(length);
-        offset += length;
-      }
+    new StreamWriter().write(path, fileLength, bufferSize, newClient);
+  }
+
+  private void testMultipleFiles(String pathBase, int numFile, SizeInBytes fileLength,
+      int bufferSize, CheckedSupplier<FileStoreClient, IOException> newClient) throws Exception {
+    final ExecutorService executor = Executors.newFixedThreadPool(numFile);
 
-      DataStreamReply reply = dataStreamOutput.closeAsync().join();
-      Assert.assertTrue(reply.isSuccess());
+    final List<Future<StreamWriter>> writerFutures = new ArrayList<>();
+    for (int i = 0; i < numFile; i++) {
+      String path =  pathBase + "-" + i;
+      final Callable<StreamWriter> callable = LogUtils.newCallable(LOG,
+          () -> new StreamWriter().write(path, fileLength, bufferSize, newClient),
+          () -> path);
+      writerFutures.add(executor.submit(callable));
+    }
+    for (Future<StreamWriter> future : writerFutures) {
+      future.get();
+    }
+  }
+
+  static class StreamWriter {

Review comment:
       Let's combine the StreamWriter to FileStoreBaseTest.Writer?  Then we can easily compare the performance in the unit tests.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org