You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@livy.apache.org by js...@apache.org on 2019/08/30 03:53:13 UTC
[incubator-livy] branch master updated: [LIVY-642] Fix a rare
status happened in yarn cause SparkApp change into error state
This is an automated email from the ASF dual-hosted git repository.
jshao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-livy.git
The following commit(s) were added to refs/heads/master by this push:
new a90f4fa [LIVY-642] Fix a rare status happened in yarn cause SparkApp change into error state
a90f4fa is described below
commit a90f4fac8be27a38cc961c24043a494a739ff188
Author: wangzhefeng <wa...@didichuxing.com>
AuthorDate: Fri Aug 30 11:53:03 2019 +0800
[LIVY-642] Fix a rare status happened in yarn cause SparkApp change into error state
some of batch session returned failed state while spark yarn application is succeeded. and in livy log，i find the reason：
```
2019-08-23 05:32:48,096 ERROR [Logging.scala:56] - Unknown YARN state RUNNING for app application_1559632632227_47808044 with final status SUCCEEDED.
```
it means yarn application state is RUNNING while final app status is SUCCEEDED, it's a correct status in yarn, means AM has not sync app state yet. and livy mapped this situatition as SparkApp.state.FAILED.
So, i think we should map this state into SparkApp.state.RUNNING，waiting for AM sync.
Author: wangzhefeng <wa...@didichuxing.com>
Closes #204 from Jassy1994/LIVY-642.
---
server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala b/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala
index 06d00a0..9762032 100644
--- a/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala
+++ b/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala
@@ -217,7 +217,8 @@ class SparkYarnApp private[utils] (
(YarnApplicationState.SUBMITTED, FinalApplicationStatus.UNDEFINED) |
(YarnApplicationState.ACCEPTED, FinalApplicationStatus.UNDEFINED) =>
SparkApp.State.STARTING
- case (YarnApplicationState.RUNNING, FinalApplicationStatus.UNDEFINED) =>
+ case (YarnApplicationState.RUNNING, FinalApplicationStatus.UNDEFINED) |
+ (YarnApplicationState.RUNNING, FinalApplicationStatus.SUCCEEDED) =>
SparkApp.State.RUNNING
case (YarnApplicationState.FINISHED, FinalApplicationStatus.SUCCEEDED) =>
SparkApp.State.FINISHED