[GitHub] [doris] stalary commented on a diff in pull request #9583: [Feature] Support es external table not assign schema

[GitBox <gi...@apache.org>]

stalary commented on code in PR #9583:
URL: https://github.com/apache/doris/pull/9583#discussion_r910794922


##########
fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java:
##########
@@ -87,12 +93,129 @@ public static JSONObject getJsonObject(JSONObject jsonObject, String key, int fr
         }
     }
 
+    /**
+     * Get boolean field, if parse error throw DdlException.
+     **/
     public static boolean getBoolean(Map<String, String> properties, String name) throws DdlException {
         String property = properties.get(name).trim();
         try {
             return Boolean.parseBoolean(property);
         } catch (Exception e) {
-            throw new DdlException(String.format("fail to parse %s, %s = %s, `%s` should be like 'true' or 'false'， value should be double quotation marks", name, name, property, name));
+            throw new DdlException(String.format(
+                    "fail to parse %s, %s = %s, `%s` should be like 'true' or 'false',"
+                            + " value should be double quotation marks",
+                    name, name, property, name));
+        }
+    }
+
+    /**
+     * Parse the required field information from the json.
+     *
+     * @param searchContext the current associated column searchContext
+     * @param indexMapping  the return value of _mapping
+     */
+    public static void resolveFields(SearchContext searchContext, String indexMapping) throws DorisEsException {
+        JSONObject properties =
+                getMappingProps(searchContext.sourceIndex(), searchContext.esTable().getName(), indexMapping,
+                        searchContext.type());
+        for (Column col : searchContext.columns()) {
+            String colName = col.getName();
+            // if column exists in Doris Table but no found in ES's mapping, we choose to ignore this situation?
+            if (!properties.containsKey(colName)) {
+                continue;
+            }
+            JSONObject fieldObject = (JSONObject) properties.get(colName);
+            resolveKeywordFields(searchContext, fieldObject, colName);
+            resolveDocValuesFields(searchContext, fieldObject, colName);
+        }
+    }
+
+    /**
+     * Get mapping properties JSONObject.
+     **/
+    public static JSONObject getMappingProps(String sourceIndex, String tableName, String indexMapping,
+            String mappingType) {
+        JSONObject jsonObject = (JSONObject) JSONValue.parse(indexMapping);
+        // the indexName use alias takes the first mapping
+        Iterator<String> keys = jsonObject.keySet().iterator();
+        String docKey = keys.next();
+        JSONObject docData = (JSONObject) jsonObject.get(docKey);
+        JSONObject mappings = (JSONObject) docData.get("mappings");
+        JSONObject rootSchema = (JSONObject) mappings.get(mappingType);
+        JSONObject properties;
+        // After (include) 7.x, type was removed from ES mapping, default type is `_doc`
+        // https://www.elastic.co/guide/en/elasticsearch/reference/7.0/removal-of-types.html
+        if (rootSchema == null) {
+            if (!"_doc".equals(mappingType)) {

Review Comment:
   I will rebase later. This has been modified.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org