You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@pig.apache.org by "Mohit Sabharwal (JIRA)" <ji...@apache.org> on 2015/01/26 06:25:34 UTC

[jira] [Commented] (PIG-4393) Add stats and error reporting for Spark

    [ https://issues.apache.org/jira/browse/PIG-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14291475#comment-14291475 ] 

Mohit Sabharwal commented on PIG-4393:
--------------------------------------

[~kellyzly], [~praveenr019] , added initial patch for review on RB. This patch adds spark specific stats. But mostly I'd like to get feedback around the assumption that a separate Spark job is launched for each POStore operator. 

> Add stats and error reporting for Spark
> ---------------------------------------
>
>                 Key: PIG-4393
>                 URL: https://issues.apache.org/jira/browse/PIG-4393
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>              Labels: spark
>
> After Pig submits a job to Spark cluster, we need to report to user the:
> -  job progress
> - spark specific stats for the jobs 
> - spark error log  (if there is an error) and/or stacktrace



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)