You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by GitBox <gi...@apache.org> on 2022/06/09 15:25:29 UTC
[GitHub] [carbondata] Indhumathi27 opened a new pull request, #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Indhumathi27 opened a new pull request, #4277:
URL: https://github.com/apache/carbondata/pull/4277
### Why is this PR needed?
### What changes were proposed in this PR?
### Does this PR introduce any user interface change?
- No
- Yes. (please explain the change and update document)
### Is any new testcase added?
- No
- Yes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1152219493
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6340/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1151846268
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1165593049
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4625/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r904630776
##########
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala:
##########
@@ -177,25 +175,8 @@ private[sql] case class CarbonAlterTableDropColumnCommand(
carbonTable,
schemaEvolutionEntry,
tableInfo)(sparkSession)
- // get the columns in schema order and filter the dropped column in the column set
- val cols = carbonTable.getCreateOrderColumn.asScala
- .collect { case carbonColumn if !carbonColumn.isInvisible => carbonColumn.getColumnSchema }
- .filterNot(column => delCols.contains(column))
- // When we call
- // alterExternalCatalogForTableWithUpdatedSchema to update the new schema to external catalog
- // in case of drop column, spark gets the catalog table and then it itself adds the partition
- // columns if the table is partition table for all the new data schema sent by carbon,
- // so there will be duplicate partition columns, so send the columns without partition columns
- val columns = if (carbonTable.isHivePartitionTable) {
- val partitionColumns = partitionInfo.getColumnSchemaList.asScala
- val carbonColumnsWithoutPartition = cols.filterNot(col => partitionColumns.contains(col))
- Some(carbonColumnsWithoutPartition)
- } else {
- Some(cols)
- }
- CarbonSessionCatalogUtil.alterDropColumns(
- tableIdentifier, columns, sparkSession)
- sparkSession.catalog.refreshTable(tableIdentifier.quotedString)
+ AlterTableUtil.revertSchemaInCaseOfFailure(carbonTable,
Review Comment:
please rename the method to a more meaningful one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r905965848
##########
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala:
##########
@@ -177,25 +175,8 @@ private[sql] case class CarbonAlterTableDropColumnCommand(
carbonTable,
schemaEvolutionEntry,
tableInfo)(sparkSession)
- // get the columns in schema order and filter the dropped column in the column set
- val cols = carbonTable.getCreateOrderColumn.asScala
- .collect { case carbonColumn if !carbonColumn.isInvisible => carbonColumn.getColumnSchema }
- .filterNot(column => delCols.contains(column))
- // When we call
- // alterExternalCatalogForTableWithUpdatedSchema to update the new schema to external catalog
- // in case of drop column, spark gets the catalog table and then it itself adds the partition
- // columns if the table is partition table for all the new data schema sent by carbon,
- // so there will be duplicate partition columns, so send the columns without partition columns
- val columns = if (carbonTable.isHivePartitionTable) {
- val partitionColumns = partitionInfo.getColumnSchemaList.asScala
- val carbonColumnsWithoutPartition = cols.filterNot(col => partitionColumns.contains(col))
- Some(carbonColumnsWithoutPartition)
- } else {
- Some(cols)
- }
- CarbonSessionCatalogUtil.alterDropColumns(
- tableIdentifier, columns, sparkSession)
- sparkSession.catalog.refreshTable(tableIdentifier.quotedString)
+ AlterTableUtil.revertSchemaInCaseOfFailure(carbonTable,
Review Comment:
changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1151375683
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1151699373
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4595/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1165578356
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/756/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1163260999
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/749/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1166857315
@akashrn5 Handled all comments. Please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1163238494
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4618/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1164252027
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.3/3/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r904632456
##########
integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala:
##########
@@ -1191,4 +1191,28 @@ object AlterTableUtil {
}
}
+ def revertSchemaInCaseOfFailure(carbonTable: CarbonTable, deleteCols: Seq[ColumnSchema],
Review Comment:
change method name and add the method level comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1165592395
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6369/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r905964526
##########
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala:
##########
@@ -177,25 +175,8 @@ private[sql] case class CarbonAlterTableDropColumnCommand(
carbonTable,
schemaEvolutionEntry,
tableInfo)(sparkSession)
- // get the columns in schema order and filter the dropped column in the column set
- val cols = carbonTable.getCreateOrderColumn.asScala
- .collect { case carbonColumn if !carbonColumn.isInvisible => carbonColumn.getColumnSchema }
- .filterNot(column => delCols.contains(column))
- // When we call
- // alterExternalCatalogForTableWithUpdatedSchema to update the new schema to external catalog
- // in case of drop column, spark gets the catalog table and then it itself adds the partition
- // columns if the table is partition table for all the new data schema sent by carbon,
- // so there will be duplicate partition columns, so send the columns without partition columns
- val columns = if (carbonTable.isHivePartitionTable) {
- val partitionColumns = partitionInfo.getColumnSchemaList.asScala
- val carbonColumnsWithoutPartition = cols.filterNot(col => partitionColumns.contains(col))
- Some(carbonColumnsWithoutPartition)
- } else {
- Some(cols)
- }
- CarbonSessionCatalogUtil.alterDropColumns(
Review Comment:
Handled Drop and Rename scenarios also, since it also has the same issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1166855586
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1152174555
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/728/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1177262885
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4644/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] asfgit closed pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
URL: https://github.com/apache/carbondata/pull/4277
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r905965648
##########
integration/spark/src/main/scala/org/apache/spark/util/AlterTableUtil.scala:
##########
@@ -1191,4 +1191,28 @@ object AlterTableUtil {
}
}
+ def revertSchemaInCaseOfFailure(carbonTable: CarbonTable, deleteCols: Seq[ColumnSchema],
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1152019650
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4597/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1163238487
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6362/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1166989094
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4629/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1166989095
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6373/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1161597705
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_2.4.5/4611/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1167050963
Build Success with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/760/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1177104074
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1180477564
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1177235017
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/775/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r904630776
##########
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala:
##########
@@ -177,25 +175,8 @@ private[sql] case class CarbonAlterTableDropColumnCommand(
carbonTable,
schemaEvolutionEntry,
tableInfo)(sparkSession)
- // get the columns in schema order and filter the dropped column in the column set
- val cols = carbonTable.getCreateOrderColumn.asScala
- .collect { case carbonColumn if !carbonColumn.isInvisible => carbonColumn.getColumnSchema }
- .filterNot(column => delCols.contains(column))
- // When we call
- // alterExternalCatalogForTableWithUpdatedSchema to update the new schema to external catalog
- // in case of drop column, spark gets the catalog table and then it itself adds the partition
- // columns if the table is partition table for all the new data schema sent by carbon,
- // so there will be duplicate partition columns, so send the columns without partition columns
- val columns = if (carbonTable.isHivePartitionTable) {
- val partitionColumns = partitionInfo.getColumnSchemaList.asScala
- val carbonColumnsWithoutPartition = cols.filterNot(col => partitionColumns.contains(col))
- Some(carbonColumnsWithoutPartition)
- } else {
- Some(cols)
- }
- CarbonSessionCatalogUtil.alterDropColumns(
- tableIdentifier, columns, sparkSession)
- sparkSession.catalog.refreshTable(tableIdentifier.quotedString)
+ AlterTableUtil.revertSchemaInCaseOfFailure(carbonTable,
Review Comment:
Method name looks confusing, please rename the method to a more meaningful one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r904631943
##########
integration/spark/src/main/scala/org/apache/spark/sql/execution/command/schema/CarbonAlterTableDropColumnCommand.scala:
##########
@@ -177,25 +175,8 @@ private[sql] case class CarbonAlterTableDropColumnCommand(
carbonTable,
schemaEvolutionEntry,
tableInfo)(sparkSession)
- // get the columns in schema order and filter the dropped column in the column set
- val cols = carbonTable.getCreateOrderColumn.asScala
- .collect { case carbonColumn if !carbonColumn.isInvisible => carbonColumn.getColumnSchema }
- .filterNot(column => delCols.contains(column))
- // When we call
- // alterExternalCatalogForTableWithUpdatedSchema to update the new schema to external catalog
- // in case of drop column, spark gets the catalog table and then it itself adds the partition
- // columns if the table is partition table for all the new data schema sent by carbon,
- // so there will be duplicate partition columns, so send the columns without partition columns
- val columns = if (carbonTable.isHivePartitionTable) {
- val partitionColumns = partitionInfo.getColumnSchemaList.asScala
- val carbonColumnsWithoutPartition = cols.filterNot(col => partitionColumns.contains(col))
- Some(carbonColumnsWithoutPartition)
- } else {
- Some(cols)
- }
- CarbonSessionCatalogUtil.alterDropColumns(
Review Comment:
please check if the same issue exists for alter drop column, column rename also and handle the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a diff in pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on code in PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#discussion_r904630497
##########
integration/spark/src/main/scala/org/apache/spark/sql/hive/CarbonSessionCatalogUtil.scala:
##########
@@ -153,10 +153,17 @@ object CarbonSessionCatalogUtil {
dbName: String): CatalogStorageFormat = {
storage.copy(locationUri = Some(path.toUri))
}
+
+ def mockAlterAddCol(): Unit = {
Review Comment:
try reusing `updateCatalogTableForAlter` by small refactoring for the testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1161597703
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/742/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [CARBONDATA-4342] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1177235018
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6389/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1151637567
Build Failed with Spark 3.1, Please check CI http://121.244.95.60:12602/job/ApacheCarbon_PR_Builder_3.1/726/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4277: [WIP] Fix Desc Columns shows New Column added, even though ALter ADD column query failed
Posted by GitBox <gi...@apache.org>.
CarbonDataQA2 commented on PR #4277:
URL: https://github.com/apache/carbondata/pull/4277#issuecomment-1151587985
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12602/job/ApacheCarbonPRBuilder2.3/6338/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@carbondata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org