You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by jongyoul <gi...@git.apache.org> on 2015/06/24 08:13:47 UTC
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
GitHub user jongyoul opened a pull request:
https://github.com/apache/incubator-zeppelin/pull/118
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Spark supports pyspark on yarn cluster without deploying python libraries from Spark 1.4
- https://issues.apache.org/jira/browse/SPARK-6869
- apache/spark#5580, apache/spark#5478
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/incubator-zeppelin ZEPPELIN-18
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit c031cd4d6177bb177774f8b8f1aa3fca951393b3
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-21T03:24:34Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Added plugin to download and zip pyspark files
commit 7f86637d2969c870fb74059f4c77161c457bbc00
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-21T03:27:18Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Removed redundant dependency setting
commit 18986b52d7bbfcd7e81df698b499a85500abef8a
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:08:12Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Followed spark's way to support pyspark
- https://issues.apache.org/jira/browse/SPARK-6869
- https://github.com/apache/spark/pull/5580
- https://github.com/apache/spark/pull/5478/files
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-115448347
Thank you very much for contributing this!
It would be great to have a high level summary of the changes, so please correct me in case I miss-understand something:
This PR allows users of pyspark skip setting pythonpath env var, copy Python modules on every node of he cluster and have spark installed(in case of pyspark in local mode on 1 machine) by adding new artefact to the Zeppelin build, a python, hidden behind optional build profile, that brings py4j as well as Python code of pyspark by downloading (and caching) actual spark distribution and re-packing those to a zip file, available in Z class path at runtime.
Is that correct?
If it is - that sounds great to me, pspark not working in Z with local interpreter without having spark installed was very frustrating.
One question: a Python dir is not a maven submodule now but may be it should be, one day, what do you guys think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
GitHub user jongyoul reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/118
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Spark supports pyspark on yarn cluster without deploying python libraries from Spark 1.4
- https://issues.apache.org/jira/browse/SPARK-6869
- apache/spark#5580, apache/spark#5478
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/incubator-zeppelin ZEPPELIN-18
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit 4a24907063504171f38162a51e1fb7461eb9a305
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T05:33:15Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- rebasing
commit 3fa2642091ee6e447ed2d6449911883fc4eb68ff
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-21T03:27:18Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Removed redundant dependency setting
commit 2f1e3d8a04847584bdba416f4ebefe468f352941
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:08:12Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Followed spark's way to support pyspark
- https://issues.apache.org/jira/browse/SPARK-6869
- https://github.com/apache/spark/pull/5580
- https://github.com/apache/spark/pull/5478/files
commit 929a67cd3e8b23a07ebb98e77dfb09525eb7efb7
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:29:01Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Removed verbose setting
commit a474fc6eaf5e08054f90293e466e1cc1c1761935
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T10:32:29Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Excludes python/** from apache-rat
commit 987682b6c7c9ddf22f353e0be126f60090db4941
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:39:03Z
rebased
commit bfcd8a08e5d670639be100ceb1ab122b5ff26fee
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T05:35:04Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- rebasing
commit 282f8e59e36da2ef0d126cce0ccd3725e3aa541e
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:40:32Z
rebased
commit f5af0d66e5ec22392b0625faac7f6abd04c72377
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T16:33:14Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Fixed checkstyle
commit 4449413032f67de391dadec7400b37e0745dc91f
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:41:11Z
rebased
commit 4de7f3b90a68c836d5e8a55b797179ed2026d6a5
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T05:37:34Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- rebasing
commit 17f6132031645e5e3bae810d3f4489c79b030a2a
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:41:39Z
rebased
commit 6a82b042f2f07f6a478927c2a5b4e83bbc1b2137
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T03:04:37Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Dummy for trigger
commit f2b9355f1b1ce1b8e9c33d229bb3219d55d77b32
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-29T14:50:25Z
- Fixed typo
commit 2812a771bef59e5af95fb52054167f34728e39a0
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-02T09:01:58Z
- Fixed typo
commit 25c70b0bfcb30ef36feffd564dd73bbc97e6844f
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:28:54Z
- Removed setting SPARK_HOME
- Changed the location of pyspark's directory into interpreter/spark
commit 8224e1ae4134d43fba518d7a61a7202fb97c1964
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:29:45Z
- Removed meaningless codes comments
commit 203d866f0b376f1bb979b06c03c950246a9b9ae7
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:49:36Z
- Cleanup codes
commit 6a2930d17f0accf4700be034537393a6919cf166
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:50:41Z
- Cleanup codes
commit c477ba911156ec3d6b86c0942e00cc87af674177
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T07:25:44Z
- Cleanup codes
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/118
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
GitHub user jongyoul reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/118
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Spark supports pyspark on yarn cluster without deploying python libraries from Spark 1.4
- https://issues.apache.org/jira/browse/SPARK-6869
- apache/spark#5580, apache/spark#5478
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/incubator-zeppelin ZEPPELIN-18
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit c031cd4d6177bb177774f8b8f1aa3fca951393b3
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-21T03:24:34Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Added plugin to download and zip pyspark files
commit 7f86637d2969c870fb74059f4c77161c457bbc00
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-21T03:27:18Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Removed redundant dependency setting
commit 18986b52d7bbfcd7e81df698b499a85500abef8a
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:08:12Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Followed spark's way to support pyspark
- https://issues.apache.org/jira/browse/SPARK-6869
- https://github.com/apache/spark/pull/5580
- https://github.com/apache/spark/pull/5478/files
commit 7bae323a77e8c4a07b6af09c1ac89978ca465bd2
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:29:01Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Removed verbose setting
commit 2af513d0138f1ab0fcc90059473abb42b30109ab
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T10:32:29Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Excludes python/** from apache-rat
commit 8de7aa28d838a3adb8b869c0c31912afd3b87e63
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T10:35:22Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Ignored python/ into .gitignore
commit bd76cd65d9716f18158dfca2023c3dadfe78d429
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T15:47:56Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Added python libraries into distribution
- Supported pyspark with profile of yarn-pyspark
commit 58e671971d5a2aef90b686747a2de55649e0c779
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T16:22:19Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Checked python libraries existence
commit 1d0600ec67d0b806505b4b866b51b208332f8d79
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T16:33:14Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Fixed checkstyle
commit d811499494c2034bbf0c9c221020fb661fdb934b
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T02:10:24Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Reverted unwanted changes
commit 6c4a134309fd7bf3c45119018b2ff5632b351bf3
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T02:11:12Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Changed profile name
- Added that profile into .travis.yml
commit 94c583d9c8c3abd3bf25a08a841e7440e50c67f7
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T02:22:53Z
Reset unwanted changes
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-115112646
I've rebased
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-115557821
@bzz It's correct for your understanding. I'm worried about mesos cluster mode because I don't test my PR on these cluster yet, but standalone cluster will be ok because that cluster already has python libraries. I've only tested it from local mode and yarn cluster mode. Concerning a directory name of python, I think python interpreter will be added somedays, can you recommend the directory name?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-114974261
@Leemoonsoo thanks. I will check it tomorrow. And what do you think of making a profile for yarn-pyspark?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/118
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-114975739
Making profile for pyspark is not a bad idea. However, pyspark can work with not only yarn but also with mesos and standalone cluster. So i think it would be better if profile looks like `-Ppyspark` than `-Pyarn-pyspark`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/118
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-116913075
I'm trying to remove some unused settings. I hope not to set SPARK_HOME for using pyspark eventually.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-114973098
You can download the log file and search for "BUILD FAILURE"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
GitHub user jongyoul reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/118
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Spark supports pyspark on yarn cluster without deploying python libraries from Spark 1.4
- https://issues.apache.org/jira/browse/SPARK-6869
- apache/spark#5580, apache/spark#5478
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jongyoul/incubator-zeppelin ZEPPELIN-18
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit 982247a5d1912ef693941ad3443f505d627f467a
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T05:33:15Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- rebasing
commit cda51951337d4eae17023f8559c607ec9daafad6
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-21T03:27:18Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Removed redundant dependency setting
commit bff47ad6d51ea20db00f7aba47b7c381562e712c
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:08:12Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Followed spark's way to support pyspark
- https://issues.apache.org/jira/browse/SPARK-6869
- https://github.com/apache/spark/pull/5580
- https://github.com/apache/spark/pull/5478/files
commit d09b258d0c6d73b131317aad2622481922184a69
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T06:29:01Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Removed verbose setting
commit 5efced2a4e6e382b7d09b132499ca8b29f08530c
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T10:32:29Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Excludes python/** from apache-rat
commit b40c5cb98b37f8708585c3ff15ab1da643ae683e
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T10:35:22Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Ignored python/ into .gitignore
commit d164f4c5d273a5a4ae72a86737112a43a999ac04
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T05:35:04Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- rebasing
commit 3b2d3401bbb95ba363ac1a8fb50b90c995800abf
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T16:22:19Z
[ZEPPELIN-18] Remove setting SPARK_HOME for PySpark
- Checked python libraries existence
commit 2f5a707ecb2b2c635649f64e843a5f066e921305
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-24T16:33:14Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Fixed checkstyle
commit 5cfd65f626cba1d25ba8a387914e5c1441d60ad0
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T02:10:24Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Reverted unwanted changes
commit f50f2e65dba2994202dc52a14e219dca57fd7e10
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T05:37:34Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- rebasing
commit f030cca7fce20dd4ae28f71c97b4ae5279981809
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T02:22:53Z
Reset unwanted changes
commit 0ed0e8b25e83ff979d5d6c61b93d4cc77df901d1
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-25T03:04:37Z
[ZEPPELIN-18] Running pyspark without deploying python libraries to every yarn node
- Dummy for trigger
commit c1f39eec6a2a2d255d4adaecc4cb5c887627f0d4
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-06-29T14:50:25Z
- Fixed typo
commit f84731de6f447b127934e67c085b91ef8ac60185
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-02T09:01:58Z
- Fixed typo
commit 4403b025dc3890e0c3248e0b21bbdeaeccddbcb1
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:28:54Z
- Removed setting SPARK_HOME
- Changed the location of pyspark's directory into interpreter/spark
commit 914741bb421677f5c81955beb3a36b446a0a1748
Author: Jongyoul Lee <jo...@gmail.com>
Date: 2015-07-03T03:29:45Z
- Removed meaningless codes comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-114970107
Can anyone help me know the reason why last build fails?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-115413759
@jongyoul
Thanks for great contribution. It really helps not only yarn but also local mode and standalone cluster.
LGTM!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-118540529
Thanks @jongyoul. Great work!
Looks good to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/118
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-114797351
Do you think it's good to make a profile? There is no problem to make it. I'll handle it to make that joke as just joke.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-118515336
@Leemoonsoo I've rebased it again because all of travis tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-118222092
I've heard close and reopen issue is the easiest way to trigger a travis, but In my case, I have a merge conflict.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-115217834
@bzz @Leemoonsoo Review this again, please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: [ZEPPELIN-18] Running pyspark wit...
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/118#issuecomment-114794444
Rising a question on necessity of downloading spark on every dev machine because recently heard a joke from @anthonycorbacho (somebody on the Strata conference),
> "Want do download whole internet? Build Zeppelin!"
I.e `.m2` dir on my machine is about 4 Gb already.
Is there a reason we can't have it under some some kind of profile?
So users who need it will do like `-Pyarn-python`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---