You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "nk1506 (via GitHub)" <gi...@apache.org> on 2024/04/05 07:23:49 UTC
[PR] Spark: Hive-View Integration Tests [iceberg]
nk1506 opened a new pull request, #10088:
URL: https://github.com/apache/iceberg/pull/10088
This one should be ready for review after Hive-View [PR](https://github.com/apache/iceberg/pull/9852)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Hive-View Integration Tests [iceberg]
Posted by "nk1506 (via GitHub)" <gi...@apache.org>.
nk1506 commented on code in PR #10088:
URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553073213
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java:
##########
@@ -1318,8 +1324,16 @@ public void createViewWithSubqueryExpressionInFilterThatIsRewritten()
sql("USE spark_catalog");
assertThatThrownBy(() -> sql(sql))
- .isInstanceOf(AnalysisException.class)
- .hasMessageContaining(String.format("The table or view `%s` cannot be found", tableName));
+ .satisfiesAnyOf(
+ throwable ->
+ assertThat(throwable)
Review Comment:
Hive throws Exception of type `SparkException` with error message as `Exception thrown in awaitResult`.
Should we override these tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Hive-View Integration Tests [iceberg]
Posted by "nk1506 (via GitHub)" <gi...@apache.org>.
nk1506 commented on code in PR #10088:
URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553071451
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java:
##########
@@ -900,7 +906,7 @@ public void dropV1View() {
String v1View = "v1ViewToBeDropped";
sql("USE spark_catalog");
sql("CREATE NAMESPACE IF NOT EXISTS %s", NAMESPACE);
- sql("CREATE TABLE %s (id INT, data STRING)", tableName);
+ sql("CREATE TABLE IF NOT EXISTS %s (id INT, data STRING)", tableName);
Review Comment:
Hive is throwing table already found error. Even though we are switching catalog. we can also override this test for Hive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Hive-View Integration Tests [iceberg]
Posted by "nk1506 (via GitHub)" <gi...@apache.org>.
nk1506 commented on code in PR #10088:
URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553075946
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java:
##########
@@ -1421,38 +1442,38 @@ public void showViews() throws NoSuchTableException {
insertRows(6);
String sql = String.format("SELECT * from %s", tableName);
sql("CREATE VIEW v1 AS %s", sql);
- sql("CREATE VIEW prefixV2 AS %s", sql);
- sql("CREATE VIEW prefixV3 AS %s", sql);
+ sql("CREATE VIEW prefixv2 AS %s", sql);
Review Comment:
Hive Catalog returns table/view name always in lower case. To be safe changed the name to lowercase only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Hive-View Integration Tests [iceberg]
Posted by "nk1506 (via GitHub)" <gi...@apache.org>.
nk1506 commented on code in PR #10088:
URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553067799
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java:
##########
@@ -62,6 +64,9 @@ public class TestViews extends ExtensionsTestBase {
private static final Namespace NAMESPACE = Namespace.of("default");
private final String tableName = "table";
+ @Parameter(index = 3)
+ protected String location;
Review Comment:
Different catalog has different scheme+path. Current tests were only for InMemoryCatalog.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Hive-View Integration Tests [iceberg]
Posted by "nk1506 (via GitHub)" <gi...@apache.org>.
nk1506 commented on code in PR #10088:
URL: https://github.com/apache/iceberg/pull/10088#discussion_r1553073692
##########
spark/v3.5/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestViews.java:
##########
@@ -1341,10 +1355,17 @@ public void createViewWithSubqueryExpressionInQueryThatIsRewritten() throws NoSu
.containsExactly(row(3), row(3), row(3));
sql("USE spark_catalog");
-
assertThatThrownBy(() -> sql(sql))
- .isInstanceOf(AnalysisException.class)
- .hasMessageContaining(String.format("The table or view `%s` cannot be found", tableName));
+ .satisfiesAnyOf(
+ throwable ->
+ assertThat(throwable)
+ .isInstanceOfAny(AnalysisException.class)
+ .hasMessageContaining(
+ String.format("The table or view `%s` cannot be found", tableName)),
+ throwable ->
Review Comment:
same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org