[GitHub] [druid] clintropolis commented on a diff in pull request #13406: fix KafkaInputFormat with nested columns

[GitBox <gi...@apache.org>]

clintropolis commented on code in PR #13406:
URL: https://github.com/apache/druid/pull/13406#discussion_r1033976095


##########
extensions-core/kafka-indexing-service/src/main/java/org/apache/druid/data/input/kafkainput/KafkaInputReader.java:
##########
@@ -113,14 +169,9 @@ private CloseableIterator<InputRow> buildBlendedRows(InputEntityReader valuePars
                 "Unsupported input format in valueFormat. KafkaInputFormat only supports input format that return MapBasedInputRow rows"
             );
           }
-          Map<String, Object> event = new HashMap<>(headerKeyList);
-          /* Currently we prefer payload attributes if there is a collision in names.
-              We can change this beahvior in later changes with a config knob. This default
-              behavior lets easy porting of existing inputFormats to the new one without any changes.
-            */
-          event.putAll(valueRow.getEvent());
-
-          HashSet<String> newDimensions = new HashSet<String>(valueRow.getDimensions());
+
+          final Map<String, Object> event = buildBlendedEventMap(valueRow.getEvent(), headerKeyList);

Review Comment:
   Yeah, it is left like that intentionally for now because auto discovered dimensions are created with a string dimension indexer pointing at them today which is just going to end up calling java .toString on these values.
   
   I am actually working on making it possible to use the nested column indexer for all auto discovered dimensions instead of the string dimension indexer we use today, which will be the appropriate time to add these columns to the list of discovered columns.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org