You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/02/20 09:40:48 UTC
[GitHub] [spark] stczwd opened a new pull request #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
stczwd opened a new pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644
### Why are the changes needed?
At present, HiveClientImpl.runHive will not throw an exception when it runs incorrectly, which will cause it to fail to feedback error information normally.
Example
```scala
spark.sql("add jar file:///tmp/not_exists.jar")
spark.sql("show databases").show()
```
/tmp/not_exists.jar doesn't exist, thus add jar is failed. However this code will run completely without causing application failure.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
add new suite tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589541612
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118756/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r381950657
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala
##########
@@ -847,6 +848,11 @@ class HiveQuerySuite extends HiveComparisonTest with SQLTestUtils with BeforeAnd
sql("DROP TEMPORARY FUNCTION udtf_count2")
}
+ test("SPARK-30868 ADD JAR command failed") {
+ val jarPath = "file:///tmp/not_exists.jar"
Review comment:
can we use some common commands like `CREATE TABLE`, `SELECT`, `INSERT`, etc.?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590186077
**[Test build #118841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118841/testReport)** for PR 27644 at commit [`dfabd08`](https://github.com/apache/spark/commit/dfabd083e2142983a6b625bed1a01ea4ad47b0b5).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589510776
**[Test build #118756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118756/testReport)** for PR 27644 at commit [`6d7435e`](https://github.com/apache/spark/commit/6d7435ed11e502c8ae517f33eaf6fb2ea12aae29).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590177386
**[Test build #118839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118839/testReport)** for PR 27644 at commit [`d74378d`](https://github.com/apache/spark/commit/d74378d92c225603ca252621083717dc66ea82b6).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan closed pull request #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan closed pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589566587
**[Test build #118775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118775/testReport)** for PR 27644 at commit [`390c07e`](https://github.com/apache/spark/commit/390c07e39f71fff81806368ef1f7294d3d2c5f64).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589511011
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23507/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382186706
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,31 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use db_not_exists"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n into)"))
+
+ // test describe table failed
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"DESC FORMATED $tblName"))
Review comment:
This is an abnormal DESC statement, the `FORMATED` is a wrong word.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590161994
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23589/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382056832
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala
##########
@@ -847,6 +848,11 @@ class HiveQuerySuite extends HiveComparisonTest with SQLTestUtils with BeforeAnd
sql("DROP TEMPORARY FUNCTION udtf_count2")
}
+ test("SPARK-30868 ADD JAR command failed") {
+ val jarPath = "file:///tmp/not_exists.jar"
Review comment:
sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590161991
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589566587
**[Test build #118775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118775/testReport)** for PR 27644 at commit [`390c07e`](https://github.com/apache/spark/commit/390c07e39f71fff81806368ef1f7294d3d2c5f64).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589541605
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382409518
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,31 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use db_not_exists"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n into)"))
+
+ // test describe table failed
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"DESC FORMATED $tblName"))
Review comment:
ah i see, then the table name doesn't matter. how about `DESC FORMATED t`? and add comment to say that `FORMATED` is a wrong word.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382085974
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ val dbName = "db_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use $dbName"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n int)"))
Review comment:
this should not fail
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589541503
**[Test build #118756 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118756/testReport)** for PR 27644 at commit [`6d7435e`](https://github.com/apache/spark/commit/6d7435ed11e502c8ae517f33eaf6fb2ea12aae29).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r381952920
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala
##########
@@ -847,6 +848,11 @@ class HiveQuerySuite extends HiveComparisonTest with SQLTestUtils with BeforeAnd
sql("DROP TEMPORARY FUNCTION udtf_count2")
}
+ test("SPARK-30868 ADD JAR command failed") {
+ val jarPath = "file:///tmp/not_exists.jar"
Review comment:
sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382801656
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -839,7 +839,12 @@ private[hive] class HiveClientImpl(
state.out.println(tokens(0) + " " + cmd_1)
// scalastyle:on println
}
- Seq(proc.run(cmd_1).getResponseCode.toString)
+ val response: CommandProcessorResponse = proc.run(cmd)
Review comment:
This should be `cmd_1` instead of `cmd`. Is there a reason to use `cmd`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590156320
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589511006
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590161836
**[Test build #118841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118841/testReport)** for PR 27644 at commit [`dfabd08`](https://github.com/apache/spark/commit/dfabd083e2142983a6b625bed1a01ea4ad47b0b5).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589567117
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590156177
**[Test build #118839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118839/testReport)** for PR 27644 at commit [`d74378d`](https://github.com/apache/spark/commit/d74378d92c225603ca252621083717dc66ea82b6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382085768
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
Review comment:
`run hive query failed` is misleading, how about
```
SPARK-30868: throw an exception if HiveClient#runSqlHive fails
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589581558
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590156177
**[Test build #118839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118839/testReport)** for PR 27644 at commit [`d74378d`](https://github.com/apache/spark/commit/d74378d92c225603ca252621083717dc66ea82b6).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590177734
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590177741
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118839/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589541605
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382056832
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveQuerySuite.scala
##########
@@ -847,6 +848,11 @@ class HiveQuerySuite extends HiveComparisonTest with SQLTestUtils with BeforeAnd
sql("DROP TEMPORARY FUNCTION udtf_count2")
}
+ test("SPARK-30868 ADD JAR command failed") {
+ val jarPath = "file:///tmp/not_exists.jar"
Review comment:
sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590186455
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118841/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382087427
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
Review comment:
good
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589581570
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118775/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589581375
**[Test build #118775 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118775/testReport)** for PR 27644 at commit [`390c07e`](https://github.com/apache/spark/commit/390c07e39f71fff81806368ef1f7294d3d2c5f64).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382086240
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ val dbName = "db_not_exists"
Review comment:
we can just inline it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-588970753
cc @xuanyuanking @cloud-fan
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw
Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590161836
**[Test build #118841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118841/testReport)** for PR 27644 at commit [`dfabd08`](https://github.com/apache/spark/commit/dfabd083e2142983a6b625bed1a01ea4ad47b0b5).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589567117
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589511006
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590156324
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23588/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589581558
Merged build finished. Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r383074400
##########
File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala
##########
@@ -839,7 +839,12 @@ private[hive] class HiveClientImpl(
state.out.println(tokens(0) + " " + cmd_1)
// scalastyle:on println
}
- Seq(proc.run(cmd_1).getResponseCode.toString)
+ val response: CommandProcessorResponse = proc.run(cmd)
Review comment:
done. we should use cmd_1 here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590177734
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590161994
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23589/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590186455
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118841/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589567125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23528/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-588860265
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-588861984
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382086447
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ val dbName = "db_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use $dbName"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n int)"))
Review comment:
maybe use a non existing data type?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382477219
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,26 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which doesn't exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use db_not_exists"))
+
+ // test create hive table failed
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE table_not_exists(n into)"))
Review comment:
make the comment clear that this is to test the wrong `into` type. The table name probably doesn't matter and we can use `t`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590156320
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589511011
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23507/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589510776
**[Test build #118756 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/118756/testReport)** for PR 27644 at commit [`6d7435e`](https://github.com/apache/spark/commit/6d7435ed11e502c8ae517f33eaf6fb2ea12aae29).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382087388
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ val dbName = "db_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use $dbName"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n int)"))
+
+ // test describe hive table which not exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"DESC FORMATED $tblName"))
+
+ // test insert hive table which not exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"INSERT into table $tblName values(1)"))
Review comment:
we don't need to test non-existing table all the time. How about unsupported syntax? `INSERT INTO t SELECT ...`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590161991
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589581570
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118775/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382476835
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,26 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
Review comment:
nit: to shorten the test we can add `val client = externalCatalog.client` at the beginning.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590186453
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590156324
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23588/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-588861984
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590186453
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589541612
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118756/
Test FAILed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382135580
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,31 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use db_not_exists"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n into)"))
Review comment:
I'd simply write `CREATE TABLE t(n not_a_type)`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] stczwd commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
stczwd commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382087329
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,32 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw QueryExectionException if run hive query failed") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ val dbName = "db_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use $dbName"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n int)"))
Review comment:
This is going to run in Hive, not in Spark. It is invalid in hive.
Or maybe I changed to a better query.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382135827
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,31 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
+ val jarPath = "file:///tmp/not_exists.jar"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(s"ADD JAR $jarPath"))
+
+ // test change to the database which not exists
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"use db_not_exists"))
+
+ // test create hive table failed
+ val tblName = "table_not_exists"
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"CREATE TABLE $tblName(n into)"))
+
+ // test describe table failed
+ assertThrows[QueryExecutionException](externalCatalog.client.runSqlHive(
+ s"DESC FORMATED $tblName"))
Review comment:
I'd keep either `DESC` or `DROP`, to test non-existing table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-588860265
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589567125
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23528/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on a change in pull request #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on a change in pull request #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#discussion_r382135362
##########
File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogSuite.scala
##########
@@ -154,4 +155,31 @@ class HiveExternalCatalogSuite extends ExternalCatalogSuite {
catalog.createTable(hiveTable, ignoreIfExists = false)
assert(catalog.getTable("db1", "spark_29498").owner === owner)
}
+
+ test("SPARK-30868 throw an exception if HiveClient#runSqlHive fails") {
+ // test add jars which not exists
Review comment:
`which not exists` -> `which doesn't exists`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27644:
[SPARK-30868][SQL] Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590177741
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118839/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-590199025
thanks, merging to master/3.0!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cloud-fan commented on issue #27644: [SPARK-30868][SQL]
Throw Exception if runHive(sql) failed
Posted by GitBox <gi...@apache.org>.
cloud-fan commented on issue #27644: [SPARK-30868][SQL] Throw Exception if runHive(sql) failed
URL: https://github.com/apache/spark/pull/27644#issuecomment-589510531
ok to test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org