[GitHub] [incubator-pinot] amrishlal commented on a change in pull request #6918: Convert collections not unnested to json

[GitBox <gi...@apache.org>]

amrishlal commented on a change in pull request #6918:
URL: https://github.com/apache/incubator-pinot/pull/6918#discussion_r633788767



##########
File path: pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ComplexTypeTransformer.java
##########
@@ -256,23 +318,73 @@ private void flattenMap(String path, Map<String, Object> map, Collection<String>
           flattenMap(concatName, map, innerMapFields);
         }
       } else if (value instanceof Collection && _unnestFields.contains(concatName)) {
-        for (Object inner : (Collection) value) {
-          if (inner instanceof Map) {
-            Map<String, Object> innerMap = (Map<String, Object>) inner;
-            flattenMap(concatName, innerMap, new ArrayList<>(innerMap.keySet()));
+        Collection collection = (Collection) value;
+        if (_unnestFields.contains(concatName)) {
+          for (Object inner : (Collection) value) {
+            if (inner instanceof Map) {
+              Map<String, Object> innerMap = (Map<String, Object>) inner;
+              flattenMap(concatName, innerMap, new ArrayList<>(innerMap.keySet()));
+            }
+          }
+        } else if (shallConvertToJson(collection)) {
+          try {
+            // convert the collection to JSON string
+            String jsonString = JsonFunctions.jsonFormat(collection);
+            map.put(field, jsonString);
+          } catch (JsonProcessingException e) {
+            throw new RuntimeException(
+                String.format("Caught exception while converting value to JSON string %s", value), e);
           }
         }
-      } else if (isArray(value) && _unnestFields.contains(concatName)) {
-        for (Object inner : (Object[]) value) {
-          if (inner instanceof Map) {
-            Map<String, Object> innerMap = (Map<String, Object>) inner;
-            flattenMap(concatName, innerMap, new ArrayList<>(innerMap.keySet()));
+      } else if (isArray(value)) {
+        Object[] array = (Object[]) value;
+        if (_unnestFields.contains(concatName)) {
+          for (Object inner : (Object[]) value) {
+            if (inner instanceof Map) {
+              Map<String, Object> innerMap = (Map<String, Object>) inner;
+              flattenMap(concatName, innerMap, new ArrayList<>(innerMap.keySet()));
+            }
+          }
+        } else if (shallConvertToJson(array)) {
+          try {
+            // convert the array to JSON string
+            String jsonString = JsonFunctions.jsonFormat(array);
+            map.put(field, jsonString);
+          } catch (JsonProcessingException e) {
+            throw new RuntimeException(
+                String.format("Caught exception while converting value to JSON string %s", value), e);
           }
         }
       }
     }
   }
 
+  private boolean shallConvertToJson(Object[] value) {
+    switch (_collectionToJsonMode) {
+      case ALL:
+        return true;
+      case NONE:
+        return false;
+      case NON_PRIMITIVE:
+        return !containPrimitives(value);
+      default:
+        throw new IllegalArgumentException(String.format("Unsupported collectionToJsonMode %s", _collectionToJsonMode));
+    }
+  }
+
+  private boolean shallConvertToJson(Collection value) {

Review comment:
       Instead of duplicating code, just do:
     `return shallConvertToJson(value.toArray());`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org