[jira] [Comment Edited] (PIG-5157) Upgrade to Spark 2.0

["Nandor Kollar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16028477#comment-16028477 ] 

Nandor Kollar edited comment on PIG-5157 at 5/29/17 3:52 PM:
-------------------------------------------------------------

[~kellyzly] could you please have a look at my patch? There are two questionable change:
- in JobMetricsListener.java there's a huge code section commented out (uncomment and remove the code onTaskEnd until we fix PIG-5157). Should we enable that?
- I removed JobLogger, do we need it? It seems that a property called 'spark.eventLog.enabled' is the proper replacement for this class, should we use it instead? It looks like JobLogger became deprecated and was removed from Spark 2.


was (Author: nkollar):
[~kellyzly] could you please have a look at my patch? There are two questionable change:
- in JobMetricsListener.java there's a huge code section commented out (uncomment and remove the code onTaskEnd until we fix PIG-5157). Should we enable that?
- I didn't find a proper replacement for JobLogger, hence it is removed. What was it used for? It looks like it became deprecated and was removed from Spark.

> Upgrade to Spark 2.0
> --------------------
>
>                 Key: PIG-5157
>                 URL: https://issues.apache.org/jira/browse/PIG-5157
>             Project: Pig
>          Issue Type: Improvement
>          Components: spark
>            Reporter: Nandor Kollar
>            Assignee: Nandor Kollar
>             Fix For: 0.17.0
>
>         Attachments: PIG-5157.patch
>
>
> Upgrade to Spark 2.0 (or latest)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)