[GitHub] [incubator-iotdb] jt2594838 commented on a change in pull request #455: [IOTDB-251]improve TSQueryDataSet structure in RPC

[GitBox <gi...@apache.org>]

jt2594838 commented on a change in pull request #455: [IOTDB-251]improve TSQueryDataSet structure in RPC
URL: https://github.com/apache/incubator-iotdb/pull/455#discussion_r336965978
 
 

 ##########
 File path: server/src/main/java/org/apache/iotdb/db/utils/QueryDataSetUtils.java
 ##########
 @@ -55,66 +55,86 @@ public static TSQueryDataSet convertQueryDataSetByFetchSize(QueryDataSet queryDa
 
   public static TSQueryDataSet convertQueryDataSetByFetchSize(QueryDataSet queryDataSet,
       int fetchSize, WatermarkEncoder watermarkEncoder) throws IOException {
+    List<TSDataType> dataTypes = queryDataSet.getDataTypes();
+    int columnNum = dataTypes.size();
     TSQueryDataSet tsQueryDataSet = new TSQueryDataSet();
-    tsQueryDataSet.setRecords(new ArrayList<>());
+    int columnNumWithTime = columnNum + 1;
+    DataOutputStream[] dataOutputStreams = new DataOutputStream[columnNumWithTime];
+    ByteArrayOutputStream[] byteArrayOutputStreams = new ByteArrayOutputStream[columnNumWithTime];
+    for (int i = 0; i < columnNumWithTime; i++) {
+      byteArrayOutputStreams[i] = new ByteArrayOutputStream();
+      dataOutputStreams[i] = new DataOutputStream(byteArrayOutputStreams[i]);
+    }
+
+    int rowCount = 0;
+    int valueOccupation = 0;
     for (int i = 0; i < fetchSize; i++) {
       if (queryDataSet.hasNext()) {
+        rowCount++;
         RowRecord rowRecord = queryDataSet.next();
         if (watermarkEncoder != null) {
           rowRecord = watermarkEncoder.encodeRecord(rowRecord);
         }
-        tsQueryDataSet.getRecords().add(convertToTSRecord(rowRecord));
+        // use columnOutput to write byte array
+        dataOutputStreams[0].writeLong(rowRecord.getTimestamp());
+        List<Field> fields = rowRecord.getFields();
+        for (int k = 0; k < fields.size(); k++) {
+          Field field = fields.get(k);
+          DataOutputStream dataOutputStream = dataOutputStreams[k + 1]; // DO NOT FORGET +1
+          if (field.getDataType() == null) {
+            dataOutputStream.writeBoolean(true); // is_empty true
+          } else {
+            dataOutputStream.writeBoolean(false); // is_empty false
 
 Review comment:
   Optional: replace the booleans with a bit map which induce less overhead.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services