You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/03/10 02:45:15 UTC
[GitHub] [iceberg] aokolnychyi opened a new issue #2311: Investigate new DS V2 interfaces in Spark 3.1
aokolnychyi opened a new issue #2311:
URL: https://github.com/apache/iceberg/issues/2311
We need to take a look at new interfaces in DS V2 in Spark 3.1 and see if we can support them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] aokolnychyi commented on issue #2311: Investigate new DS V2 interfaces in Spark 3.1
Posted by GitBox <gi...@apache.org>.
aokolnychyi commented on issue #2311:
URL: https://github.com/apache/iceberg/issues/2311#issuecomment-794773590
@flyrain, could you take a look at this one?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] flyrain commented on issue #2311: Investigate new DS V2 interfaces in Spark 3.1
Posted by GitBox <gi...@apache.org>.
flyrain commented on issue #2311:
URL: https://github.com/apache/iceberg/issues/2311#issuecomment-831616239
Currently, partition related ops like `Alter table add partition` and `Show partitions` fail due to Hive doesn't think an Iceberg table is partitioned even though it is. This probably can be resolved in Spark 3.1.
Spark 3.1 introduces interface SupportsPartitionManagement. We can let SparkTable extends it so that partition related ops can be supported for Iceberg table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] flyrain edited a comment on issue #2311: Investigate new DS V2 interfaces in Spark 3.1
Posted by GitBox <gi...@apache.org>.
flyrain edited a comment on issue #2311:
URL: https://github.com/apache/iceberg/issues/2311#issuecomment-831616239
Currently, partition related ops like `Alter table add partition` and `Show partitions` fail due to Hive doesn't think an Iceberg table is partitioned even though it is, since the table partition metadata is in file sytem, NOT in HMS. We may resolved the issue in Spark 3.1.
Spark 3.1 introduces interface SupportsPartitionManagement. We can let SparkTable extends it so that partition related ops can be supported for Iceberg table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org