[jira] [Updated] (BIGTOP-715) Add Spark packaging

["Konstantin Boudnik (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BIGTOP-715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Boudnik updated BIGTOP-715:
--------------------------------------

    Attachment: BIGTOP-715.master-0.8.patch

Attaching the patch that creates correct RPM package with latest Spark project master. The package can really be executed without scala runtime being set on the target machine, so we might consider removing the Scala redistribution by this package.

A couple of TODOs:
- current the build would only works off my spark branch here https://github.com/c0s/spark/tree/assembly. The master branch will be usable once this PR https://github.com/mesos/spark/pull/675 is merged
- package requires hadoop-hdfs, hadoop-yarn, hadoop-mapreduce to be install in order to reuse Hadoop specific libraries
- a couple of improvements are needed:
-- start scripts to bring up Master and worker nodes need to be added
-- DEB specific files need to be fixes a little bit (most of the work is done in install script anyway)
-- man page needs to be created
-- components jars don't need to be unpacked into separate directories and can be put into the root dir of the package.

Comments are very welcome.
                
> Add Spark packaging
> -------------------
>
>                 Key: BIGTOP-715
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-715
>             Project: Bigtop
>          Issue Type: Task
>          Components: Debian, RPM
>    Affects Versions: 0.3.1, 0.5.0
>            Reporter: MTG dev
>            Assignee: Konstantin Boudnik
>         Attachments: BIGTOP-715.master-0.8.patch, BIGTOP-715.patch, BIGTOP-715.spark7.patch, BIGTOP-715.spark7.patch, BIGTOP-715.spark7.patch
>
>
> It would be great to add Spark native packages to the BigTop.
> Spark project is in-memory fast data analytic project from Berkeley (www.spark-project.org) 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira