[jira] [Commented] (RATIS-470) write(List) not correctly handling all records

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/RATIS-470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737554#comment-16737554 ] 

Josh Elser commented on RATIS-470:
----------------------------------

{code:java}
diff --git a/ratis-logservice/src/test/java/org/apache/ratis/logservice/LogServiceReadWriteBase.java b/ratis-logservice/src/test/java/org/apache/ratis/logservice/LogServiceReadWriteBase.java
index fd71311d..0478392e 100644
--- a/ratis-logservice/src/test/java/org/apache/ratis/logservice/LogServiceReadWriteBase.java
+++ b/ratis-logservice/src/test/java/org/apache/ratis/logservice/LogServiceReadWriteBase.java
@@ -18,9 +18,12 @@
 package org.apache.ratis.logservice;

 import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;

 import java.io.IOException;
 import java.nio.ByteBuffer;
+import java.nio.charset.StandardCharsets;
+import java.util.Iterator;
 import java.util.List;

 import org.apache.ratis.BaseTest;
@@ -101,11 +104,35 @@ public abstract class LogServiceReadWriteBase<CLUSTER extends MiniRaftCluster>
       long startId = logStream.getStartRecordId();
       LOG.info("start id {}", startId);
       //
-      reader.seek(lastId + 1);
+      reader.seek(startId);
       // Read records back
-      List<ByteBuffer> data = reader.readBulk(1);
-      assertEquals(1, data.size());
+      List<ByteBuffer> data = reader.readBulk(records.size());
+      LOG.info("Read records: " + toString(data));
+      assertEquals(records.size(), data.size());
+
+      Iterator<ByteBuffer> expectedIter = records.iterator();
+      Iterator<ByteBuffer> actualIter = data.iterator();
+      while (expectedIter.hasNext() && actualIter.hasNext()) {
+        ByteBuffer expected = expectedIter.next();
+        ByteBuffer actual = actualIter.next();
+        assertEquals(expected, actual);
+      }
+    }
+  }
+
+  private String toString(List<ByteBuffer> records) {
+    StringBuilder sb = new StringBuilder();
+    for (ByteBuffer bb : records) {
+      assertTrue("Cannot use a ByteBuffer that doesn't have an array: " + bb, bb.hasArray());
+      int size = bb.limit() - bb.position();
+      byte[] data = new byte[size];
+      System.arraycopy(bb.array(), bb.position(), data, 0, size);
+      if (sb.length() > 0) {
+        sb.append(", ");
+      }
+      sb.append(new String(data, StandardCharsets.UTF_8));
     }
+    return sb.insert(0, "[").append("]").toString();
   }{code}
This is the test modification that shows the problem. This will fail saying that we have {{2}} records, one which is "our" batched protobuf message and one which is from the quorum (and should have been filtered out), not {{10}} as expected.

> write(List<ByteBuffer>) not correctly handling all records
> ----------------------------------------------------------
>
>                 Key: RATIS-470
>                 URL: https://issues.apache.org/jira/browse/RATIS-470
>             Project: Ratis
>          Issue Type: Bug
>          Components: LogService
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Major
>             Fix For: 0.4.0
>
>
> Digging into LogServiceReadWriteBase, I'm noticing that the validation clause for the data we wrote to the state machine is lacking.
> The test writes 10 "records" into the log, but because we write these in a single message to the state machine, we only get one "record" in the Raft log.
> Need to figure out how to take the one client-protobuf message containing N "logservice records" but make sure we have N "raft messages" in the log (or something equivalent).
> fyi [~chrajeshbabu32@gmail.com], [~vrodionov].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)