You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Sean Mackrory <ma...@gmail.com> on 2014/01/21 20:20:56 UTC
Review Request 17151: BIGTOP-1181. Add pyspark to spark package
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17151/
-----------------------------------------------------------
Review request for bigtop.
Bugs: BIGTOP-1181
https://issues.apache.org/jira/browse/BIGTOP-1181
Repository: bigtop
Description
-------
Adding a separate package "spark-python" to add pyspark. It depends on Python, and there's a mechanism to override which python is used
Diffs
-----
bigtop-packages/src/common/spark/install_spark.sh 75dd337
bigtop-packages/src/deb/spark/control fc61489
bigtop-packages/src/deb/spark/rules bd0ad6b
bigtop-packages/src/deb/spark/spark-core.install PRE-CREATION
bigtop-packages/src/deb/spark/spark-python.install PRE-CREATION
bigtop-packages/src/rpm/spark/SPECS/spark.spec a8db290
bigtop-tests/test-artifacts/package/src/main/resources/package_data.xml 166323c
Diff: https://reviews.apache.org/r/17151/diff/
Testing
-------
I've made minor modifications based on feedback and other recent patches, but I've built these packages and run pyspark on multiple platforms. A good way to test the ability to run computation or access data is "sc.parallelize([1,2,3]).sum()" or "sc.textFile("/a-file-in-hdfs.txt").
Thanks,
Sean Mackrory
Re: Review Request 17151: BIGTOP-1181. Add pyspark to spark package
Posted by Mark Grover <gr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17151/#review32411
-----------------------------------------------------------
Ship it!
Ship It!
- Mark Grover
On Jan. 21, 2014, 7:22 p.m., Sean Mackrory wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17151/
> -----------------------------------------------------------
>
> (Updated Jan. 21, 2014, 7:22 p.m.)
>
>
> Review request for bigtop.
>
>
> Bugs: BIGTOP-1181
> https://issues.apache.org/jira/browse/BIGTOP-1181
>
>
> Repository: bigtop
>
>
> Description
> -------
>
> Adding a separate package "spark-python" to add pyspark. It depends on Python, and there's a mechanism to override which python is used (which I tested on RHEL 5), but the macros to activate that mechanism are not included in this patch, pending the OS support decisions we make for Bigtop 0.8.0.
>
>
> Diffs
> -----
>
> bigtop-packages/src/common/spark/install_spark.sh 75dd337
> bigtop-packages/src/deb/spark/control fc61489
> bigtop-packages/src/deb/spark/rules bd0ad6b
> bigtop-packages/src/deb/spark/spark-core.install PRE-CREATION
> bigtop-packages/src/deb/spark/spark-python.install PRE-CREATION
> bigtop-packages/src/rpm/spark/SPECS/spark.spec a8db290
> bigtop-tests/test-artifacts/package/src/main/resources/package_data.xml 166323c
>
> Diff: https://reviews.apache.org/r/17151/diff/
>
>
> Testing
> -------
>
> I've made minor modifications based on feedback and other recent patches, but I've built these packages and run pyspark on multiple platforms. A good way to test the ability to run computation or access data is "sc.parallelize([1,2,3]).sum()" or "sc.textFile("/a-file-in-hdfs.txt").
>
>
> Thanks,
>
> Sean Mackrory
>
>
Re: Review Request 17151: BIGTOP-1181. Add pyspark to spark package
Posted by Sean Mackrory <ma...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17151/
-----------------------------------------------------------
(Updated Jan. 21, 2014, 7:22 p.m.)
Review request for bigtop.
Bugs: BIGTOP-1181
https://issues.apache.org/jira/browse/BIGTOP-1181
Repository: bigtop
Description (updated)
-------
Adding a separate package "spark-python" to add pyspark. It depends on Python, and there's a mechanism to override which python is used (which I tested on RHEL 5), but the macros to activate that mechanism are not included in this patch, pending the OS support decisions we make for Bigtop 0.8.0.
Diffs
-----
bigtop-packages/src/common/spark/install_spark.sh 75dd337
bigtop-packages/src/deb/spark/control fc61489
bigtop-packages/src/deb/spark/rules bd0ad6b
bigtop-packages/src/deb/spark/spark-core.install PRE-CREATION
bigtop-packages/src/deb/spark/spark-python.install PRE-CREATION
bigtop-packages/src/rpm/spark/SPECS/spark.spec a8db290
bigtop-tests/test-artifacts/package/src/main/resources/package_data.xml 166323c
Diff: https://reviews.apache.org/r/17151/diff/
Testing
-------
I've made minor modifications based on feedback and other recent patches, but I've built these packages and run pyspark on multiple platforms. A good way to test the ability to run computation or access data is "sc.parallelize([1,2,3]).sum()" or "sc.textFile("/a-file-in-hdfs.txt").
Thanks,
Sean Mackrory