You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/10/26 08:36:00 UTC
[GitHub] [iceberg] liukun4515 opened a new issue #1662: Action: support spark3 and customer catalog
liukun4515 opened a new issue #1662:
URL: https://github.com/apache/iceberg/issues/1662
In the spark2.4, the action uses the interface `spark.read().format("iceberg").load(table)` to load the table as the dataset.
But It just can load the default spark catalog(`spark_catalog`) table, and the `table` is `db.tablename`.
If I want to use the action to handle some tables which are in the other iceberg customer catalog, there will throw an exception referred to the #1652.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-716576458
I thought you could still load it directly from the path (ignoring the other catalogs?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] liukun4515 commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-719439906
Is this #1525 pr is about supporting spark3 action?
@RussellSpitzer
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-716833359
I think I understand the point now,
We want to invoke an action on a table whose identifier cannot be read using the current logic in BaseAction to generate metadata tables.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] HeartSaVioR commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-716533196
When you deal with DataFrameWriterV1 you may want to "forget about" catalog, and also want to "double-check about" how the "path" is interpreted for the format (data source).
Btw what's the use case we would like to touch some tables in other catalog?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-718793691
Not yet, but obviously we will need to make some :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] liukun4515 commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-718469437
> Yep we'll have to make some modifications. I think probably a first step is just only making the extensions work for the default catalog. But otherwise we need to start extending some of the methods in the base class so they properly handle multi-part identifiers and we determine how to load up metadata tables properly for tables in other catalogs.
There are pull requests about the modifications?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #1662: Action: support spark3 and customer catalog
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #1662:
URL: https://github.com/apache/iceberg/issues/1662#issuecomment-717272981
Yep we'll have to make some modifications. I think probably a first step is just only making the extensions work for the default catalog. But otherwise we need to start extending some of the methods in the base class so they properly handle multi-part identifiers and we determine how to load up metadata tables properly for tables in other catalogs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org