You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/04/27 06:29:13 UTC

[GitHub] [iceberg] jccsjtu opened a new issue #2524: use removeField, and then cannot get desc

jccsjtu opened a new issue #2524:
URL: https://github.com/apache/iceberg/issues/2524


   if use removeField for a table, and then we cannot get desc command result
   table.updateSpec()
       .removeField("category")
       .commit();
   
   spark-sql> desc iceberg_catalog.dlf_test.iceberg_part_test;
   21/04/27 10:39:06 ERROR SparkSQLDriver: Failed in [desc iceberg_catalog.dlf_test.iceberg_part_test]
   java.lang.UnsupportedOperationException: Void transform is not supported
           at org.apache.iceberg.transforms.PartitionSpecVisitor.alwaysNull(PartitionSpecVisitor.java:86)
           at org.apache.iceberg.transforms.PartitionSpecVisitor.visit(PartitionSpecVisitor.java:148)
           at org.apache.iceberg.transforms.PartitionSpecVisitor.visit(PartitionSpecVisitor.java:120)
           at org.apache.iceberg.transforms.PartitionSpecVisitor.visit(PartitionSpecVisitor.java:102)
           at org.apache.iceberg.spark.Spark3Util.toTransforms(Spark3Util.java:246)
           at org.apache.iceberg.spark.source.SparkTable.partitioning(SparkTable.java:127)
           at org.apache.spark.sql.execution.datasources.v2.DescribeTableExec.addPartitioning(DescribeTableExec.scala:78)
           at org.apache.spark.sql.execution.datasources.v2.DescribeTableExec.run(DescribeTableExec.scala:41)
           at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result$lzycompute(V2CommandExec.scala:39)
           at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.result(V2CommandExec.scala:39)
           at org.apache.spark.sql.execution.datasources.v2.V2CommandExec.executeCollect(V2CommandExec.scala:45
   
   I read the manifest file *.matadata.json, find the partitionSpec is as following, the transform is void
    "partition-spec" : [ {
       "name" : "a",
       "transform" : "**void**",
       "source-id" : 1,
       "field-id" : 1000
     }, {
       "name" : "b",
       "transform" : **"void"**,
       "source-id" : 2,
       "field-id" : 1001
     } ],
     "default-spec-id" : 2,
     "partition-specs" : [ {
       "spec-id" : 0,
       "fields" : [ {
         "name" : "a",
         "transform" : "identity",
         "source-id" : 1,
         "field-id" : 1000
       } ]
     }, {
       "spec-id" : 1,
       "fields" : [ {
         "name" : "a",
         "transform" : "identity",
         "source-id" : 1,
         "field-id" : 1000
       }, {
         "name" : "b",
         "transform" : "identity",
         "source-id" : 2,
         "field-id" : 1001
       } ]
     }, {
       "spec-id" : 2,
       "fields" : [ {
         "name" : "a",
         "transform" : "void",
         "source-id" : 1,
         "field-id" : 1000
       }, {
         "name" : "b",
         "transform" : "void",
         "source-id" : 2,
         "field-id" : 1001
       } ]
     } ],


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] RussellSpitzer commented on issue #2524: use removeField, and then cannot get desc

Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #2524:
URL: https://github.com/apache/iceberg/issues/2524#issuecomment-827552075


   #2453 I think


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] jccsjtu commented on issue #2524: use removeField, and then cannot get desc

Posted by GitBox <gi...@apache.org>.
jccsjtu commented on issue #2524:
URL: https://github.com/apache/iceberg/issues/2524#issuecomment-834118497


   same issue, thank you! I will close this one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

[GitHub] [iceberg] jccsjtu closed issue #2524: use removeField, and then cannot get desc

Posted by GitBox <gi...@apache.org>.
jccsjtu closed issue #2524:
URL: https://github.com/apache/iceberg/issues/2524


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org