[jira] [Updated] (PIG-4952) Calculate the value of parallism for spark mode

["liyunzhang_intel (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

liyunzhang_intel updated PIG-4952:
----------------------------------
    Attachment: PIG-4952.patch

[~kexianda]: help review, changes are:
when there are mutiple parent rdds of 1 rdd, we use the max value of parallellism value of  parent rdds as the value of current rdd. see document:
https://spark.apache.org/docs/latest/tuning.html#level-of-parallelism

> Calculate the value of parallism for spark mode
> -----------------------------------------------
>
>                 Key: PIG-4952
>                 URL: https://issues.apache.org/jira/browse/PIG-4952
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>             Fix For: spark-branch
>
>         Attachments: PIG-4952.patch, PIG-4952_1.patch
>
>
> Calculate the value of parallism for spark mode like what org.apache.pig.backend.hadoop.executionengine.tez.plan.optimizer.ParallelismSetter does.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)