[GitHub] [carbondata] ajantha-bhat commented on a change in pull request #3778: [CARBONDATA-3916] Support array complex type with SI

[GitBox <gi...@apache.org>]

ajantha-bhat commented on a change in pull request #3778:
URL: https://github.com/apache/carbondata/pull/3778#discussion_r473599181



##########
File path: integration/spark/src/main/scala/org/apache/spark/sql/secondaryindex/command/SICreationCommand.scala
##########
@@ -443,10 +443,34 @@ private[sql] case class CarbonCreateSecondaryIndexCommand(
       databaseName: String, tableName: String, indexTableName: String,
       absoluteTableIdentifier: AbsoluteTableIdentifier): TableInfo = {
     var schemaOrdinal = -1
-    var allColumns = indexModel.columnNames.map { indexCol =>
-      val colSchema = carbonTable.getDimensionByName(indexCol).getColumnSchema
+    val complexDimensions = carbonTable.getAllDimensions.asScala
+      .filter(dim => dim.getDataType.isComplexType &&
+                     indexModel.columnNames.asJava.contains(dim.getColName))
+    if (complexDimensions.size > 1) {
+      throw new ErrorMessage("SI creation with more than one complex type is not supported yet");
+    }
+    var allColumns = List[ColumnSchema]()

Review comment:
       Consider the scenario where one SI table contains (complex, primitive1, primitive2), we need to maintain the same order. But now it will become primitive1,primitive2,complex1. which is wrong.
   
   So, suggest to keep the user-specified order.
   Can refer below code
   
   var allColumns = List[ColumnSchema]()
   indexModel.columnNames.foreach { indexCol =>
     val dimension = carbonTable.getDimensionByName(tableName, indexCol)
     val colSchema = dimension.getColumnSchema
     schemaOrdinal += 1
     allColumns = allColumns :+ cloneColumnSchema(colSchema, schemaOrdinal)
   }
   complexDimensions.foreach { complexDim =>
     if (complexDim.getNumberOfChild > 0) {
       if (complexDim.getListOfChildDimensions.asScala
         .exists(col => DataTypes.isArrayType(col.getDataType))) {
         throw new ErrorMessage("SI creation with nested array complex type is not supported yet");
       }
     }
   }




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org