You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2019/12/18 17:29:30 UTC
[GitHub] [incubator-iceberg] rdblue opened a new pull request #705: Add
session catalog for Spark 3.0
rdblue opened a new pull request #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705
For compatibility with DSv1, DSv2 has a special catalog, named `spark_catalog`, that can load both v1 and v2 tables. This PR adds `SparkSessionCatalog` that is a `CatalogExtension` and replaces the built-in `spark_catalog` to support both Iceberg and Hive tables in the same namespace.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] chenjunjiedada commented on issue #705: Add
session catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
chenjunjiedada commented on issue #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705#issuecomment-568157416
LGTM +1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue commented on a change in pull request
#705: Add session catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705#discussion_r360949137
##########
File path: hive/src/main/java/org/apache/iceberg/hive/HiveCatalog.java
##########
@@ -99,7 +99,7 @@ protected String name() {
@Override
public boolean dropTable(TableIdentifier identifier, boolean purge) {
Review comment:
No, it is called by `TableOperations` when the table is refreshed to check whether it exists at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue commented on issue #705: Add session
catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
rdblue commented on issue #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705#issuecomment-568528694
Thanks for looking at this, @jerryshao and @chenjunjiedada!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue merged pull request #705: Add session
catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
rdblue merged pull request #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue commented on issue #705: Add session
catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
rdblue commented on issue #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705#issuecomment-568005081
@chenjunjiedada, Catalogs throw NotIcebergException (extends NoSuchTableException) if a table is not Iceberg, so Iceberg won't try to delete a non-Iceberg table. In the case of drop, when the table is not Iceberg or is missing, the method returns false. That's why we don't use try/catch: the exception is not thrown because throwing it would violate the method contract.
@jerryshao, that's why I updated the methods. This now checks that the table is Iceberg, or relies on the method contract as I explained above for drop.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] rdblue commented on a change in pull request
#705: Add session catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
rdblue commented on a change in pull request #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705#discussion_r360949137
##########
File path: hive/src/main/java/org/apache/iceberg/hive/HiveCatalog.java
##########
@@ -99,7 +99,7 @@ protected String name() {
@Override
public boolean dropTable(TableIdentifier identifier, boolean purge) {
Review comment:
No, it is thrown by `TableOperations` when the table is refreshed to check whether it exists at all.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [incubator-iceberg] chenjunjiedada commented on a change in pull
request #705: Add session catalog for Spark 3.0
Posted by GitBox <gi...@apache.org>.
chenjunjiedada commented on a change in pull request #705: Add session catalog for Spark 3.0
URL: https://github.com/apache/incubator-iceberg/pull/705#discussion_r360633551
##########
File path: hive/src/main/java/org/apache/iceberg/hive/HiveCatalog.java
##########
@@ -99,7 +99,7 @@ protected String name() {
@Override
public boolean dropTable(TableIdentifier identifier, boolean purge) {
Review comment:
Do we need to add `validateTableIsIceberg` in dropTable then catch it and return false?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org