[jira] [Updated] (PIG-4233) Package pig along with dependencies into a fat jar while job submission to Spark cluster

["Praveen Rachabattuni (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Praveen Rachabattuni updated PIG-4233:
--------------------------------------
    Fix Version/s: spark-branch

> Package pig along with dependencies into a fat jar while job submission to Spark cluster
> ----------------------------------------------------------------------------------------
>
>                 Key: PIG-4233
>                 URL: https://issues.apache.org/jira/browse/PIG-4233
>             Project: Pig
>          Issue Type: Sub-task
>          Components: spark
>            Reporter: Praveen Rachabattuni
>            Assignee: Praveen Rachabattuni
>             Fix For: spark-branch
>
>         Attachments: PIG-4233.patch
>
>
> Currently we have a fat jar created in legacy directory which contains pig along with dependencies. 
> Would need to modify build.xml to add spark dependency jars to include in legacy fat jar.
> Running job on Spark cluster:
> 1. export SPARK_HOME=/path/to/spark
> 2. export SPARK_PIG_JAR=$PIG_HOME/legacy/pig-0.14.0-SNAPSHOT-withouthadoop-h1.jar
> 3. export SPARK_MASTER=spark://localhost:7077
> 4 export HADOOP_HOME=/path/to/hadoop
> 5. Launch the job using ./bin/pig -x spark



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)