[GitHub] [incubator-doris] morningman commented on a diff in pull request #9358: fix #9351 can't load parquet file with column name case sensitive with Doris column

[GitBox <gi...@apache.org>]

morningman commented on code in PR #9358:
URL: https://github.com/apache/incubator-doris/pull/9358#discussion_r867316239


##########
fe/fe-core/src/main/java/org/apache/doris/planner/StreamLoadScanNode.java:
##########
@@ -66,8 +66,8 @@ public class StreamLoadScanNode extends LoadScanNode {
     private TupleDescriptor srcTupleDesc;
     private TBrokerScanRange brokerScanRange;
 
-    private Map<String, SlotDescriptor> slotDescByName = Maps.newHashMap();
-    private Map<String, Expr> exprsByName = Maps.newHashMap();
+    private Map<String, SlotDescriptor> slotDescByName = Maps.newTreeMap(String.CASE_INSENSITIVE_ORDER);

Review Comment:
   Better add comment to explain why we need CASE_INSENSITIVE_ORDER



##########
fe/fe-core/src/main/java/org/apache/doris/load/Load.java:
##########
@@ -1045,12 +1045,23 @@ private static void initColumns(Table tbl, List<ImportColumnDesc> columnExprs,
             return;
         }
 
+        Set<String> tmpSet = Sets.newHashSet();
+        for (ImportColumnDesc importColumnDesc : copiedColumnExprs) {
+            if (importColumnDesc.getExpr() == null) {
+                tmpSet.add(importColumnDesc.getColumnName());
+            }
+        }
+
         // init slot desc add expr map, also transform hadoop functions
         for (ImportColumnDesc importColumnDesc : copiedColumnExprs) {
             // make column name case match with real column name
             String columnName = importColumnDesc.getColumnName();
-            String realColName = tbl.getColumn(columnName) == null ? columnName
-                    : tbl.getColumn(columnName).getName();
+            String realColName;
+            if (tbl.getColumn(columnName) == null || tmpSet.contains(columnName) ){
+                realColName = columnName;
+            } else {
+                    realColName = tbl.getColumn(columnName).getName();

Review Comment:
   `tbl.getColumn(columnName)` may be null here.



##########
fe/fe-core/src/main/java/org/apache/doris/load/Load.java:
##########
@@ -1045,12 +1045,23 @@ private static void initColumns(Table tbl, List<ImportColumnDesc> columnExprs,
             return;
         }
 
+        Set<String> tmpSet = Sets.newHashSet();

Review Comment:
   Better add comment to explain this logic.
   Give an example is better.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org