You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/11/05 10:17:16 UTC

[GitHub] [spark] artiship opened a new pull request #30263: [SPARK-33358][SQL] Return code when command process failed

artiship opened a new pull request #30263:
URL: https://github.com/apache/spark/pull/30263


   ### What changes were proposed in this pull request?
   Exit Spark SQL CLI processing loop if one of the commands (sub sql statement) process failed
   
   ### Why are the changes needed?
   ### Does this PR introduce _any_ user-facing change?
   No
   ### How was this patch tested?
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-727675112


   I found your id from the commit log.
   SPARK-33358 is assigned to you. Thanks.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-725540255


   Hi, @artiship . Does this bug happen only in YARN environment? It seems to work correctly in the local environment.
   ```
   spark-3.0.1-bin-hadoop3.2:$ cat 1.sql
   select * from n;
   select 2;
   
   spark-3.0.1-bin-hadoop3.2:$ bin/spark-sql -f 1.sql
   Error in query: Table or view not found: n; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [n]
   
   2
   Time taken: 2.244 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722284407


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722845926






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722715792


   ok to test


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722304753


   @artiship how was this patch tested?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722744026


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35287/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722744051






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722805735


   Looks like something went wrong during rebase. You can either resolve it here or just open another PR.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722735706






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722961639


   **[Test build #130710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130710/testReport)** for PR 30263 at commit [`db231c3`](https://github.com/apache/spark/commit/db231c30ddb49da115832b7fbbc47df97194f0dc).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-723016533






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722284407


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-727660834


   Ah, it seems that there is misunderstanding due to the PR description, @artiship .
   
   In the PR description, you mentioned that the output `2` is repeated twice. Is that true?
   
   ![Screen Shot 2020-11-15 at 3 57 40 PM](https://user-images.githubusercontent.com/9700541/99200646-61b9b600-275b-11eb-84a7-c9e63e7c6c54.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-723016533






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722285154


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722735404


   **[Test build #130676 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130676/testReport)** for PR 30263 at commit [`96c753c`](https://github.com/apache/spark/commit/96c753c2c0537b5cdfed87da40a566102eee8c50).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722961639


   **[Test build #130710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130710/testReport)** for PR 30263 at commit [`db231c3`](https://github.com/apache/spark/commit/db231c30ddb49da115832b7fbbc47df97194f0dc).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-728650584


   @dongjoon-hyun  Thanks for your carefully review. The output 2 repeated twice is a testing result I got from my production environment. It seems that it might have a configuration makes it print both command and result.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722981022






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-725540255


   Hi, @artiship . Does this happen only YARN? It seems to work correctly in the local.
   ```
   spark-3.0.1-bin-hadoop3.2:$ cat 1.sql
   select * from n;
   select 2;
   
   spark-3.0.1-bin-hadoop3.2:$ bin/spark-sql -f 1.sql
   Error in query: Table or view not found: n; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [n]
   
   2
   Time taken: 2.244 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722980496


   **[Test build #130710 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130710/testReport)** for PR 30263 at commit [`db231c3`](https://github.com/apache/spark/commit/db231c30ddb49da115832b7fbbc47df97194f0dc).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722819933


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35298/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722845926






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-727674718


   Thank you so much for your contribution, @artiship . 
   This landed at `master` branch for Apache Spark 3.1.0 and `branch-3.0` for Apache Spark 3.0.2.
   SPARK-33358 will be assigned to you. What is your JIRA id?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722744051






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722361783


   @artiship can you elaborate the reproducible steps and how you tested manually in your env?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship edited a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship edited a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722477673


   @HyukjinKwon 
   
   env
   
   ```
   spark version: 3.0.1
   os: centos 7
   ```
   
   /tmp/tmp.sql
   
   ```sql
   select * from nonexistent_table;
   select 2;
   ```
   
   submit command:
   
   ```sh
   export HADOOP_USER_NAME=my-hadoop-user
   bin/spark-sql  \
   --master yarn \
   --deploy-mode client \
   --queue my.queue.name \
   --conf spark.driver.host=$(hostname -i) \
   --conf spark.app.name=spark-test  \
   --name "spark-test" \
   -f /tmp/tmp.sql 
   ```
   
   execution log:
   
   ```sh
   # bin/spark-sql  \
   > --master yarn \
   > --deploy-mode client \
   > --queue my.queue.name \
   > --conf spark.driver.host=$(hostname -i) \
   > --conf spark.app.name=spark-test  \
   > --name "spark-test" \
   > -f /tmp/tmp.sql
   20/11/06 00:06:19 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.spark.client.rpc.server.address.use.ip does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.spark.client.submit.timeout.interval does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.enforce.bucketing does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.server2.enable.impersonation does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.run.timeout.seconds does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.support.sql11.reserved.keywords does not exist
   20/11/06 00:06:20 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
   20/11/06 00:06:20 WARN SparkConf: Note that spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone/kubernetes and LOCAL_DIRS in YARN).
   20/11/06 00:06:22 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
   
   Error in query: Table or view not found: nonexistent_table; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [nonexistent_table]
   
   2
   2
   Time taken: 4.437 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722825571


   add a new test case


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-727660834


   Ah, it seems that there is misunderstanding due to the PR description, @artiship .
   
   In the PR description and your comment (https://github.com/apache/spark/pull/30263#issuecomment-722477673), you mentioned that the output `2` is repeated twice. Is that true?
   
   ![Screen Shot 2020-11-15 at 3 57 40 PM](https://user-images.githubusercontent.com/9700541/99200646-61b9b600-275b-11eb-84a7-c9e63e7c6c54.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722735706






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship edited a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship edited a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-725810612


   @dongjoon-hyun 
   
   I've verified it again using a distribution dowloaded from the apache spark website:
   
   https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz
   
   This bug still can reproduce in local mode. The spark-sql should break after the first statement failed.
   
   ```
   ➜  spark-3.0.1-bin-hadoop2.7 cat 1.sql
   select * from n;
   select 2+2;
   ➜  spark-3.0.1-bin-hadoop2.7 bin/spark-sql -f 1.sql
   20/11/12 11:35:28 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   20/11/12 11:35:38 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout does not exist
   20/11/12 11:35:38 WARN HiveConf: HiveConf of name hive.stats.retries.wait does not exist
   20/11/12 11:35:46 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
   20/11/12 11:35:46 WARN ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore lichuanliang@10.104.16.36
   Error in query: Table or view not found: n; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [n]
   
   4
   Time taken: 4.39 seconds, Fetched 1 row(s)
   ```
   
   With no statement fail:
   
   ```
   ➜  spark-3.0.1-bin-hadoop2.7 cat 2.sql
   select 1+1;
   select 2+2;
   
   ➜  spark-3.0.1-bin-hadoop2.7 bin/spark-sql -f 2.sql
   20/11/12 11:45:56 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   20/11/12 11:46:06 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout does not exist
   20/11/12 11:46:06 WARN HiveConf: HiveConf of name hive.stats.retries.wait does not exist
   20/11/12 11:46:14 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
   20/11/12 11:46:14 WARN ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore lichuanliang@10.104.16.36
   2
   Time taken: 5.167 seconds, Fetched 1 row(s)
   4
   Time taken: 0.151 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722805513


   **[Test build #130687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130687/testReport)** for PR 30263 at commit [`3d71aca`](https://github.com/apache/spark/commit/3d71acadcb1c9040bd3de4d585dfc5db3683e42c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722818804


   **[Test build #130687 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130687/testReport)** for PR 30263 at commit [`3d71aca`](https://github.com/apache/spark/commit/3d71acadcb1c9040bd3de4d585dfc5db3683e42c).
    * This patch passes all tests.
    * This patch merges cleanly.
    * This patch adds no public classes.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722981022






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722819176






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-725810612


   @dongjoon-hyun 
   
   I've verified it again using a distribution dowloaded from the apache spark website:
   
   https://www.apache.org/dyn/closer.lua/spark/spark-3.0.1/spark-3.0.1-bin-hadoop2.7.tgz
   
   This bug still can reproduce in local mode. The spark-sql should break after the first statement failed.
   
   ```
   ➜  spark-3.0.1-bin-hadoop2.7 cat 1.sql
   select * from n;
   select 2+2;
   ➜  spark-3.0.1-bin-hadoop2.7 bin/spark-sql -f 1.sql
   20/11/12 11:35:28 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   20/11/12 11:35:38 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout does not exist
   20/11/12 11:35:38 WARN HiveConf: HiveConf of name hive.stats.retries.wait does not exist
   20/11/12 11:35:46 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
   20/11/12 11:35:46 WARN ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment = Set by MetaStore lichuanliang@10.104.16.36
   Error in query: Table or view not found: n; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [n]
   
   4
   Time taken: 4.39 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722961686


   > Looks like something went wrong during rebase. You can either resolve it here or just open another PR.
   
   Sorry for the incorrect rebase. It was resolved and now fix and test case have been merged into one commit.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722819176






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722717831


   **[Test build #130676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130676/testReport)** for PR 30263 at commit [`96c753c`](https://github.com/apache/spark/commit/96c753c2c0537b5cdfed87da40a566102eee8c50).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #30263:
URL: https://github.com/apache/spark/pull/30263#discussion_r523839759



##########
File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##########
@@ -571,4 +571,13 @@ class CliSuite extends SparkFunSuite with BeforeAndAfterAll with Logging {
     // the date formatter for `java.sql.LocalDate` must output negative years with sign.
     runCliWithin(1.minute)("SELECT MAKE_DATE(-44, 3, 15);" -> "-0044-03-15")
   }
+
+  test("SPARK-33358 CLI should break when have command failed") {

Review comment:
       This test case succeeded without your patch.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722735097


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35287/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-723016511


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35320/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722285154


   Can one of the admins verify this patch?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722805513


   **[Test build #130687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130687/testReport)** for PR 30263 at commit [`3d71aca`](https://github.com/apache/spark/commit/3d71acadcb1c9040bd3de4d585dfc5db3683e42c).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #30263:
URL: https://github.com/apache/spark/pull/30263#discussion_r523839759



##########
File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala
##########
@@ -571,4 +571,13 @@ class CliSuite extends SparkFunSuite with BeforeAndAfterAll with Logging {
     // the date formatter for `java.sql.LocalDate` must output negative years with sign.
     runCliWithin(1.minute)("SELECT MAKE_DATE(-44, 3, 15);" -> "-0044-03-15")
   }
+
+  test("SPARK-33358 CLI should break when have command failed") {

Review comment:
       This test case seems to succeed without your patch.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship edited a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship edited a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722477673


   @HyukjinKwon You can see after the first statement failed, following statement still got executed, and finally let the whole script succeed. 
   
   env
   
   ```
   spark version: 3.0.1
   os: centos 7
   ```
   
   /tmp/tmp.sql
   
   ```sql
   select * from nonexistent_table;
   select 2;
   ```
   
   submit command:
   
   ```sh
   export HADOOP_USER_NAME=my-hadoop-user
   bin/spark-sql  \
   --master yarn \
   --deploy-mode client \
   --queue my.queue.name \
   --conf spark.driver.host=$(hostname -i) \
   --conf spark.app.name=spark-test  \
   --name "spark-test" \
   -f /tmp/tmp.sql 
   ```
   
   execution log:
   
   ```sh
   # bin/spark-sql  \
   > --master yarn \
   > --deploy-mode client \
   > --queue my.queue.name \
   > --conf spark.driver.host=$(hostname -i) \
   > --conf spark.app.name=spark-test  \
   > --name "spark-test" \
   > -f /tmp/tmp.sql
   20/11/06 00:06:19 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.spark.client.rpc.server.address.use.ip does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.spark.client.submit.timeout.interval does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.enforce.bucketing does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.server2.enable.impersonation does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.run.timeout.seconds does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.support.sql11.reserved.keywords does not exist
   20/11/06 00:06:20 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
   20/11/06 00:06:20 WARN SparkConf: Note that spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone/kubernetes and LOCAL_DIRS in YARN).
   20/11/06 00:06:22 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
   
   Error in query: Table or view not found: nonexistent_table; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [nonexistent_table]
   
   2
   2
   Time taken: 4.437 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722845898


   Kubernetes integration test status success
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35298/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722477673


   @HyukjinKwon 
   
   env
   
   ```
   spark version: 3.0.1
   os: centos 7
   ```
   
   /tmp/tmp.sql
   
   ```sql
   select * from nonexistent_table;
   select 2;
   ```
   
   submit command:
   
   ```sh
   export HADOOP_USER_NAME=my-hadoop-user
   bin/spark-sql  \
   --master yarn \
   --deploy-mode client \
   --queue my.queue.name \
   --conf spark.driver.host=$(hostname -i) \
   --conf spark.app.name=spark-test  \
   --name "spark-test" \
   -f /tmp/tmp.sql 
   ```
   
   execution log:
   
   ```sh
   # bin/spark-sql  \
   > --master yarn \
   > --deploy-mode client \
   > --queue my.queue.name \
   > --conf spark.driver.host=$(hostname -i) \
   > --conf spark.app.name=spark-test  \
   > --name "spark-test" \
   > -f /tmp/tmp.sql
   20/11/06 00:06:19 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.spark.client.rpc.server.address.use.ip does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.spark.client.submit.timeout.interval does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.enforce.bucketing does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.server2.enable.impersonation does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.run.timeout.seconds does not exist
   20/11/06 00:06:20 WARN HiveConf: HiveConf of name hive.support.sql11.reserved.keywords does not exist
   20/11/06 00:06:20 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
   20/11/06 00:06:20 WARN SparkConf: Note that spark.local.dir will be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in mesos/standalone/kubernetes and LOCAL_DIRS in YARN).
   20/11/06 00:06:22 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
   20/11/06 00:06:46 WARN ShellBasedUnixGroupsMapping: got exception trying to get groups for user qttdw: id: qttdw: no such user
   
   Error in query: Table or view not found: nonexistent_table; line 1 pos 14;
   'Project [*]
   +- 'UnresolvedRelation [nonexistent_table]
   
   2
   2
   Time taken: 4.437 seconds, Fetched 1 row(s)
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-723006314


   Kubernetes integration test starting
   URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35320/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] artiship commented on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
artiship commented on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722322892


   @HyukjinKwon I reproduced this issue and then verified the fix manually from my dev env, I would like to add unit test to cover this.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] dongjoon-hyun closed pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #30263:
URL: https://github.com/apache/spark/pull/30263


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] SparkQA removed a comment on pull request #30263: [SPARK-33358][SQL] Return code when command process failed

Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #30263:
URL: https://github.com/apache/spark/pull/30263#issuecomment-722717831


   **[Test build #130676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130676/testReport)** for PR 30263 at commit [`96c753c`](https://github.com/apache/spark/commit/96c753c2c0537b5cdfed87da40a566102eee8c50).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org