You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by "Szehon Ho (JIRA)" <ji...@apache.org> on 2014/08/09 02:34:12 UTC

[jira] [Updated] (HIVE-7668) Improve error logging for spark test crashes

     [ https://issues.apache.org/jira/browse/HIVE-7668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Szehon Ho updated HIVE-7668:
----------------------------

    Description: 
While working on HIVE-7665, I noticed that spark local cluster will crash entire JVM on any dependency version mismatch, either due to native code linkage errors if you are unlucky, or gracefully (see spark's ExecutorUncaughtExceptionHandler.scala) most of the time.

To debug those issues, it would be nice to log a thread-dump upon normal JVM exits by spark by ExecutorUncaughtExceptionHandler, as its unexpected by Hive's test framework.  One way is to add a shutdownHook that does this.

  was:
While doing HIVE-7665, I noticed that spark local cluster will crash on any dependency version mismatch, either due to native code link errors (like in jetty) if unlucky, or gracefully (see spark's ExecutorUncaughtExceptionHandler.scala).

To debug those issues, it would be nice to log a thread-dump upon normal JVM exits by spark, as its unexpected by Hive's test framework.  One way is to add a shutdownHook that does this.


> Improve error logging for spark test crashes
> --------------------------------------------
>
>                 Key: HIVE-7668
>                 URL: https://issues.apache.org/jira/browse/HIVE-7668
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark, Testing Infrastructure
>            Reporter: Szehon Ho
>
> While working on HIVE-7665, I noticed that spark local cluster will crash entire JVM on any dependency version mismatch, either due to native code linkage errors if you are unlucky, or gracefully (see spark's ExecutorUncaughtExceptionHandler.scala) most of the time.
> To debug those issues, it would be nice to log a thread-dump upon normal JVM exits by spark by ExecutorUncaughtExceptionHandler, as its unexpected by Hive's test framework.  One way is to add a shutdownHook that does this.



--
This message was sent by Atlassian JIRA
(v6.2#6252)