[GitHub] [incubator-livy] mgaido91 commented on a change in pull request #155: [LIVY-565] Ensure column buffer for string and binary contains all data.

[GitBox <gi...@apache.org>]

mgaido91 commented on a change in pull request #155: [LIVY-565] Ensure column buffer for string and binary contains all data.
URL: https://github.com/apache/incubator-livy/pull/155#discussion_r266039623
 
 

 ##########
 File path: thriftserver/session/src/main/java/org/apache/livy/thriftserver/session/ColumnBuffer.java
 ##########
 @@ -241,6 +232,25 @@ private boolean isNull(int index) {
     return (nulls[byteIdx] & (1 << bitIdx)) != 0;
   }
 
+  /**
+   * Transforms and internal buffer into a list that meets Hive expectations. Used for
+   * string and binary fields.
+   *
+   * org.apache.hadoop.hive.serde2.thrift.ColumnBuffer expects a List<String> or List<ByteBuffer>,
+   * depending on the column type. The Hive/Thrift stack also dislikes nulls, and returning a list
+   * with a different number of elements than expected.
+   */
+  private <T> List<T> toList(Stream<T> data, T defaultValue) {
+    final List<T> ret = new ArrayList<>(currentSize);
+    data.limit(currentSize).forEach(e -> {
+      ret.add(e != null ? e : defaultValue);
+    });
+    while (ret.size() < currentSize) {
 
 Review comment:
   Maybe we can rather do `ensureCapacity();` and `currentSize++;` before the null check in add?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services