[GitHub] [hive] zhangbutao commented on a diff in pull request #3611: HIVE-26551: Support CREATE TABLE LIKE FILE for ORC

[GitBox <gi...@apache.org>]

zhangbutao commented on code in PR #3611:
URL: https://github.com/apache/hive/pull/3611#discussion_r977276239


##########
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java:
##########
@@ -117,4 +130,87 @@ public ObjectInspector getObjectInspector() throws SerDeException {
     return inspector;
   }
 
+  @Override
+  public List<FieldSchema> readSchema(Configuration conf, String file) throws SerDeException {
+    List<String> fieldNames;
+    List<TypeDescription> fieldTypes;
+    try (Reader reader = OrcFile.createReader(new Path(file), OrcFile.readerOptions(conf))) {
+      fieldNames = reader.getSchema().getFieldNames();
+      fieldTypes =  reader.getSchema().getChildren();
+    } catch (Exception e) {
+      throw new SerDeException(ErrorMsg.ORC_FOOTER_ERROR.getErrorCodedMsg(), e);
+    }
+
+    List<FieldSchema> schema = new ArrayList<>();
+    for (int i = 0; i < fieldNames.size(); i++) {
+      FieldSchema fieldSchema = convertOrcTypeToFieldSchema(fieldNames.get(i), fieldTypes.get(i));
+      schema.add(fieldSchema);
+      LOG.debug("Inferred field schema {}", fieldSchema);
+    }
+    return schema;
+  }
+
+  private FieldSchema convertOrcTypeToFieldSchema(String fieldName, TypeDescription fieldType) {
+    String typeName = convertOrcTypeToFieldType(fieldType);
+    return new FieldSchema(fieldName, typeName, "Inferred from Orc file.");
+  }
+
+  private String convertOrcTypeToFieldType(TypeDescription fieldType) {
+    if (fieldType.getCategory().isPrimitive()) {
+        return convertPrimitiveType(fieldType);
+      }
+    return convertComplexType(fieldType);
+  }
+
+  private String convertPrimitiveType(TypeDescription fieldType) {
+    if (fieldType.getCategory().getName().equals("timestamp with local time zone")) {
+      throw new IllegalArgumentException("Unhandled ORC type " + fieldType.getCategory().getName());
+    }
+    return fieldType.toString();
+  }
+
+  private String convertComplexType(TypeDescription fieldType) {
+    StringBuilder buffer = new StringBuilder();
+    buffer.append(fieldType.getCategory().getName());
+    switch (fieldType.getCategory()) {
+    case LIST:
+    case MAP:
+    case UNION:
+      buffer.append('<');
+      for (int i = 0; i < fieldType.getChildren().size(); i++) {
+        if (i != 0) {
+          buffer.append(',');
+        }
+        buffer.append(convertOrcTypeToFieldType(fieldType.getChildren().get(i)));
+      }
+      buffer.append('>');
+      break;
+    case STRUCT:
+      buffer.append('<');
+      for(int i=0; i < fieldType.getChildren().size(); ++i) {
+        if (i != 0) {
+          buffer.append(',');
+        }
+        getStructFieldName(buffer, fieldType.getFieldNames().get(i));
+        buffer.append(':');
+        buffer.append(convertOrcTypeToFieldType(fieldType.getChildren().get(i)));
+      }
+      buffer.append('>');
+      break;
+    default:
+      throw new IllegalArgumentException("ORC doesn't handle " +
+          fieldType.getCategory());
+    }
+    return buffer.toString();
+  }
+
+  static void getStructFieldName(StringBuilder buffer, String name) {
+    if (UNQUOTED_NAMES.matcher(name).matches()) {

Review Comment:
   Actually, this code snippet was borrowed from ORC repo.
   https://github.com/apache/orc/blob/6a74eef74b8101c7396f196f1a27f5e124a005f6/java/core/src/java/org/apache/orc/TypeDescription.java#L667-L673
   
   And related ORC jira is https://issues.apache.org/jira/browse/ORC-104.  I didn't do much research here, but i think it is better to be sync with ORC code. wdyt?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org