You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ajantha-bhat (via GitHub)" <gi...@apache.org> on 2023/02/09 08:44:10 UTC
[GitHub] [iceberg] ajantha-bhat opened a new pull request, #6787: Spark: Update register table procedure test case for all the catalogs
ajantha-bhat opened a new pull request, #6787:
URL: https://github.com/apache/iceberg/pull/6787
PR #[5037](https://github.com/apache/iceberg/pull/5037) supports register tables for all (most of) the catalogs.
There is no restriction in the register table procedure about catalog type. But the testcase has a restriction.
Hence, update the testcase to test for all the catalogs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#discussion_r1101460917
##########
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRegisterTableProcedure.java:
##########
@@ -55,10 +53,6 @@ public void dropTables() {
@Test
public void testRegisterTable() throws NoSuchTableException, ParseException {
Review Comment:
So we aren't actually adding any more tests here we are just removing this Assume clause?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on a diff in pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#discussion_r1101521454
##########
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRegisterTableProcedure.java:
##########
@@ -55,10 +53,6 @@ public void dropTables() {
@Test
public void testRegisterTable() throws NoSuchTableException, ParseException {
Review Comment:
Existing testcase for hadoop catalog
https://github.com/apache/iceberg/blob/master/core/src/test/java/org/apache/iceberg/hadoop/TestHadoopCatalog.java#L637
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#issuecomment-1423827362
@RussellSpitzer, @aokolnychyi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#discussion_r1101484896
##########
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRegisterTableProcedure.java:
##########
@@ -55,10 +53,6 @@ public void dropTables() {
@Test
public void testRegisterTable() throws NoSuchTableException, ParseException {
Review Comment:
How do we Register a table in Hadoop? That doesn't sound possible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on code in PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#discussion_r1101536372
##########
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRegisterTableProcedure.java:
##########
@@ -55,10 +53,6 @@ public void dropTables() {
@Test
public void testRegisterTable() throws NoSuchTableException, ParseException {
Review Comment:
OK Got it, Still not sure I think that was a good idea but it's already in.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer merged pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer merged PR #6787:
URL: https://github.com/apache/iceberg/pull/6787
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on a diff in pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#discussion_r1101483409
##########
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRegisterTableProcedure.java:
##########
@@ -55,10 +53,6 @@ public void dropTables() {
@Test
public void testRegisterTable() throws NoSuchTableException, ParseException {
Review Comment:
Yes.
Already API level test is there for each catalog for register table.
Configuring other catalogs in this test case is hard (like Glue, REST). It is better to keep them as integration tests (may in in a follow up PR)
Now removing this check will make sure that this testcase runs for hadoop, hive, spark catalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on a diff in pull request #6787: Spark: Update register table procedure test case for all the catalogs
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on code in PR #6787:
URL: https://github.com/apache/iceberg/pull/6787#discussion_r1101494404
##########
spark/v3.2/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestRegisterTableProcedure.java:
##########
@@ -55,10 +53,6 @@ public void dropTables() {
@Test
public void testRegisterTable() throws NoSuchTableException, ParseException {
Review Comment:
`HadoopCatalog` also extends `BaseMetastoreCatalog`
So, when a register table is called. It makes a fresh commit to the Hadoop catalog too (file name will be as per hadoop catalog)
https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java#L84
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org