You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Leemoonsoo <gi...@git.apache.org> on 2016/07/12 14:56:02 UTC
[GitHub] zeppelin pull request #1169: [WIP] [ZEPPELIN-759] Spark 2.0 support
GitHub user Leemoonsoo opened a pull request:
https://github.com/apache/zeppelin/pull/1169
[WIP] [ZEPPELIN-759] Spark 2.0 support
### What is this PR for?
This PR implement spark 2.0 support based on #747.
This PR has approach from #980 which is reimplementing code in scala.
You can try build this branch
```
mvn clean package -Dscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0-preview -Ppyspark -Psparkr -Pyarn -Phadoop-2.6 -DskipTests
```
### What type of PR is it?
Improvements
### Todos
* [x] - Spark 2.0 support
* [ ] - Rebase after #747 merge
* [ ] - .travis.yml to use spark version 2.0.0 instead of 2.0.0-preview after spark 2.0.0 release
* [ ] - Update LICENSE file
* [ ] - Update related document (build)
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-759
### How should this be tested?
Build and try
```
mvn clean package -Dscala-2.11 -Pspark-2.0 -Dspark.version=2.0.0-preview -Ppyspark -Psparkr -Pyarn -Phadoop-2.6 -DskipTests
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? yes
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Leemoonsoo/zeppelin spark2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1169.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1169
----
commit 44622072f633f1aa2329df107f691f4a293bfc31
Author: Luciano Resende <lr...@apache.org>
Date: 2016-02-25T04:46:46Z
[ZEPPELIN-605] Add support for Scala 2.11
Enable Zeppelin to be built with both Scala 2.10
and Scala 2.11, mostly to start supporting interpreters
that are moving to Scala 2.11 only such as Spark.
commit 864e52fc94997e16c2cb0922dd3af1423a4934a9
Author: Luciano Resende <lr...@apache.org>
Date: 2016-04-02T17:14:11Z
[ZEPPELIN-605] Add Scala 2.11 build profile
commit d8628c7158e29b076b661585ff5359f0e21a8438
Author: Luciano Resende <lr...@apache.org>
Date: 2016-04-07T15:16:10Z
[ZEPPELIN-605] Enable Scala 2.11 REPL support for Spark Interpreter
Scala 2.11 has changed a lot of it's REPL support and in order to enable
building with Scala 2.10 and 2.11 we need to enable the new Scala 2.11
REPL and also make the build to compile based on the scala version.
commit ac0088bc86fc25eecec7e737f3acbeb93c181a20
Author: Luciano Resende <lr...@apache.org>
Date: 2016-04-13T05:15:55Z
[ZEPPELIN-605] Rewrite Spark interpreter based on Scala 2.11 support
commit ceb1480be3d45f0c07f8dd01e42c0496863f2898
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-09T16:45:03Z
Initial scala-210, 211 support in the single binary
commit fefcb2b8ad8589c40cd3849f2b02a50486d89a3d
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-10T00:40:39Z
scala 2.11 support for spark interpreter
commit 32650c7b58257af5cf723c4ca55e736e02e6344d
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-10T02:29:36Z
Fix some reflections
commit ce4a2d828486b1200394ffe89192ece03c6ca750
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-10T02:57:57Z
Fix reflection on creating SparkCommandLine
commit fe3f8ea1fef6a4b5540cf3c70085156734659c8d
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-10T19:40:07Z
Fix reflection around HttpServer and createTempDir
commit 35fc8a08403d24713e3c70c0c0e981fc1c005808
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-11T05:55:49Z
fix style
commit 9542299a56a76a86403119b39b9e2a126b3312de
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-13T17:24:52Z
Fix reflection
commit 43d4a700c1d78b40009103b03cb5b6c0ac1d4ea3
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-13T17:31:39Z
style
commit 6da1364fe58df3188c1facdde2eae44f3dd77dfd
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-13T18:10:19Z
SparkContext sharing seems not working in scala 2.11, disable the test
commit d7bd3ee10596008e41b6dfe6021449d161d50454
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-13T19:10:20Z
Update FlinkInterpreter
commit c7df53d9007c8c3e78308c881928cacf97cfb468
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-13T19:34:19Z
Update ignite interpreter
commit 1102d00fea1da86fab9c73f8055db37dc9959e87
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-13T19:51:27Z
Remove unused methods
commit 9b2adf844f7dfe0986b975e2e9adda2efad30e30
Author: Luciano Resende <lr...@apache.org>
Date: 2016-06-16T06:47:09Z
Force close
Closes lresende/incubator-zeppelin/#1
commit 36e4eedaebcc8e9fac2b9cd0992f3c2b042e48b0
Author: Lee moon soo <mo...@apache.org>
Date: 2016-06-29T23:09:15Z
make binary built with scala 2.11 work with spark_2.10 binary
Closes lresende/incubator-zeppelin/#2
commit f62dc46bad559d1a8f6b29b0cf8bbb35c60cdcbe
Author: Luciano Resende <lr...@apache.org>
Date: 2016-07-07T16:55:14Z
Fix pom.xml merge conflict
commit 0d3bd464ae362e97aeb8d1fb5e48beac06dbb0d1
Author: Luciano Resende <lr...@apache.org>
Date: 2016-07-07T18:03:23Z
Fix new code to support both scala versions
commit 49cc8d6ea631c0b3bc122989307c4868eaf6f162
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-09T18:07:28Z
initialize imain
commit a7442049a78090b960cdd796673e924c07d6b72e
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-09T18:50:21Z
-Dscala-2.11 on mvn verify
commit bfd7340fe378c37894de0b1e957231c9f9bde1ce
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T02:36:06Z
Initial spark 2.0 support
commit 482864c61ea8ffd5ae6dedc2aeee9cd16fa250b1
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T15:04:07Z
comment out spark-streaming-twitter
commit b3d067087a3032b9669afda98475ddfce729cbce
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T15:04:58Z
generate spark2.0 compatible classname
commit 5243feaef814140a92560d88ebadce82b63f0a89
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T15:05:33Z
Remove unnecessary refernece to HiveContext
commit 1bd101afc1f46f5ee908dec4bd5f2a07ab063958
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T15:49:36Z
spark2 in PySparkInterpreter
commit bd4a5fe25eec2875dc483da88276986bb6031aec
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T16:45:43Z
get SQLContext correctly
commit 5bd564005e84ef76491d84dbbc007dc0193fe2c6
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T17:02:28Z
dd spark 2 test profile in travis
commit 55e9b7fbc5e3adde0d36927e7a8898833575532b
Author: Lee moon soo <mo...@apache.org>
Date: 2016-07-10T21:11:43Z
SparkSession.wrapped() fallbacks to SparkSession.sqlContext().
Use createDataFrame instead of applySchema
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1169: [WIP] [ZEPPELIN-759] Spark 2.0 support
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo closed the pull request at:
https://github.com/apache/zeppelin/pull/1169
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---