You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by pm...@apache.org on 2019/05/31 21:03:21 UTC
[samza] branch master updated: SAMZA-2231: Exception thrown during
yarn staging directory cleanup masks original exception
This is an automated email from the ASF dual-hosted git repository.
pmaheshwari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/samza.git
The following commit(s) were added to refs/heads/master by this push:
new 8bb6e5f SAMZA-2231: Exception thrown during yarn staging directory cleanup masks original exception
8bb6e5f is described below
commit 8bb6e5fd2c7f756b790a4dca151ac506e208d52d
Author: thunderstumpges <ts...@ntent.com>
AuthorDate: Fri May 31 14:03:15 2019 -0700
SAMZA-2231: Exception thrown during yarn staging directory cleanup masks original exception
Failures cleaning up the staging directory on another exception were masking the original exception making troubleshooting difficult. Add some logging and an extra try/catch around the cleanup.
Author: thunderstumpges <ts...@ntent.com>
Reviewers: Daniel Nishimura <dn...@linkedin.com>
Closes #1059 from thunderstumpges/try-catch-on-staging-cleanup
---
.../src/main/scala/org/apache/samza/job/yarn/YarnJob.scala | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala b/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
index 1d72a88..c374507 100644
--- a/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
+++ b/samza-yarn/src/main/scala/org/apache/samza/job/yarn/YarnJob.scala
@@ -71,8 +71,16 @@ class YarnJob(config: Config, hadoopConfig: Configuration) extends StreamJob {
)
} catch {
case e: Throwable =>
- client.cleanupStagingDir
- throw e
+ logger.error("Exception submitting yarn job.", e )
+ try {
+ // try to clean up. this may throw an exception depending on how far into launching the job we got.
+ // we don't want to mask the original problem by throwing this.
+ client.cleanupStagingDir
+ } catch {
+ case ce: Throwable => logger.warn("Exception cleaning Staging Directory after failed launch attempt.", ce)
+ } finally {
+ throw e
+ }
}
this