[GitHub] [carbondata] ShreelekhyaG commented on a change in pull request #4180: [CARBONDATA-4164][CARBONDATA-4198][CARBONDATA-4199][CARBONDATA-4234] Support alter add map, multilevel complex columns and rename/change datatype.

[GitBox <gi...@apache.org>]

ShreelekhyaG commented on a change in pull request #4180:
URL: https://github.com/apache/carbondata/pull/4180#discussion_r694642393



##########
File path: core/src/main/java/org/apache/carbondata/core/scan/executor/util/RestructureUtil.java
##########
@@ -165,6 +171,42 @@
     return presentDimension;
   }
 
+  public static void compareDatatypes(CarbonDimension currentDimension) {
+    if (currentDimension == null) {
+      return;
+    }
+    if (currentDimension.isComplex()) {
+      for (CarbonDimension childDimension : currentDimension.getListOfChildDimensions()) {
+        compareDatatypes(childDimension);
+      }
+    } else {
+      DataType newComplexChildDataType =
+          newComplexChildrenDatatypes.get(currentDimension.getColumnId());
+      // insert datatypes only in case of primitive types, as only they are allowed to be altered.
+      if (newComplexChildDataType != null && !(newComplexChildDataType

Review comment:
       Done

##########
File path: docs/ddl-of-carbondata.md
##########
@@ -775,10 +775,10 @@ CarbonData DDL statements are documented here,which includes:
      ```
      ALTER TABLE carbon ADD COLUMNS (a1 INT, b1 STRING) TBLPROPERTIES('DEFAULT.VALUE.a1'='10')
      ```
-      **NOTE:** Adding of only single-level Complex datatype columns(only array and struct) is supported.
-      Example - 
+      Adding of single-level and nested level of Complex datatype columns is supported.

Review comment:
       Done

##########
File path: docs/ddl-of-carbondata.md
##########
@@ -866,6 +866,12 @@ Users can specify which columns to include and exclude for local dictionary gene
      ALTER TABLE test_db.carbon CHANGE oldArray newArray array<int>
      ```
 
+     Example 7: Change column name in column: mapField map\<int, int> from mapField to mapField1.

Review comment:
       Done

##########
File path: integration/spark/src/main/java/org/apache/spark/sql/secondaryindex/query/SecondaryIndexQueryResultProcessor.java
##########
@@ -362,6 +367,9 @@ private void processResult(List<CarbonIterator<RowBatch>> detailQueryResultItera
    */
   private Object getData(Object[] data, int index, DataType dataType) {
     if (data == null || data.length == 0) {
+      if (DataTypeUtil.isPrimitiveColumn(dataType)) {

Review comment:
       There is no impact now, added testcase.

##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/catalyst/CarbonParserUtil.scala
##########
@@ -1124,24 +1125,17 @@ object CarbonParserUtil {
     val dataTypeName = DataTypeConverterUtil.convertToCarbonType(complexField).getName
     val dataTypeInfo = CarbonParserUtil.parseDataType(columnName, dataTypeName.toLowerCase, values)
     complexField.dataType match {
-      case Some(CarbonCommonConstants.ARRAY) =>
-        val childField = complexField.children.get(0)
-        val childType = childField.dataType
-        val childName = columnName + CarbonCommonConstants.POINT + childField.name
-        val childValues = childType match {
-          case d: DecimalType => Some(List((d.precision, d.scale)))
-          case _ => None
-        }
-        val childDatatypeInfo = parseDataType(childName, childField, childValues)
-        dataTypeInfo.setChildren(List(childDatatypeInfo))
-      case Some(CarbonCommonConstants.STRUCT) =>
+      case Some(CarbonCommonConstants.ARRAY) | Some(CarbonCommonConstants.STRUCT) |
+           Some(CarbonCommonConstants.MAP) =>
         var childTypeInfoList: List[DataTypeInfo] = null
         for (childField <- complexField.children.get) {
           val childType = childField.dataType
-          val childName = columnName + CarbonCommonConstants.POINT + childField.name.get
-          val childValues = childType match {
-            case d: DecimalType => Some(List((d.precision, d.scale)))
-            case _ => None
+          val childName = columnName + CarbonCommonConstants.POINT + childField.column
+          val childValues = if (childType.get.contains(CarbonCommonConstants.DECIMAL)) {
+            val decimalInfo = ("""\d+""".r findAllIn childType.get).toList

Review comment:
       Done

##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/catalyst/CarbonParserUtil.scala
##########
@@ -1233,9 +1226,27 @@ object CarbonParserUtil {
           }
         }
         dataTypeInfo.setChildren(childTypeInfoList)
+      case mapType: MapType =>
+        val keyType: DataType = mapType.keyType
+        val valType: DataType = mapType.valueType
+        var childTypeInfoList: List[DataTypeInfo] = List()
+        val childName1 = columnName + ".key"
+        val childName2 = columnName + ".value"
+        val keyTypeValues = keyType match {
+          case d: DecimalType => Some(List((d.precision, d.scale)))
+          case _ => None
+        }
+        val valTypeValues = valType match {

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org