You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by Leemoonsoo <gi...@git.apache.org> on 2015/11/18 16:46:04 UTC
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
GitHub user Leemoonsoo opened a pull request:
https://github.com/apache/incubator-zeppelin/pull/446
ZEPPELIN-441 Make SparkInterpreter work even if Spark version is not listed
https://issues.apache.org/jira/browse/ZEPPELIN-441
Currently when spark version is not listed in SparkVersion.java, it throws
```
java.lang.IllegalArgumentException
at org.apache.zeppelin.spark.SparkVersion.fromVersionString(SparkVersion.java:58)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:477)
```
This PR changes SparkVersion from 'enum' to 'class' and make Spark Interpreter work with unlisted version.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Leemoonsoo/incubator-zeppelin ZEPPELIN-441
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/446.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #446
----
commit f6d3007b5cfa3bc4f730da2f24b53c39ae9d3a62
Author: Lee moon soo <mo...@apache.org>
Date: 2015-11-18T15:41:04Z
enum -> class to handle unlisted Spark version
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/446
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-158292821
looks good
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-158562540
looks good
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-157970684
If it's minor/patch version difference, maybe we could warn and continue?
I think it would be nice to have a message in the notebook output, if it's in the log file then it's not very discoverable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-158289416
@felixcheung I have updated code to allow minor/patch version difference, but print error message on the notebook when major version is unsupported.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-157842726
generally I think it's ok if it is minor/patch version [differences](http://semver.org/).
but for 1.5.x vs 1.6 or 1.x vs 2.0 - I am not sure we should automatically assume compatibility.
for instance there is a new version with py4j jar that I plan to address in https://issues.apache.org/jira/browse/ZEPPELIN-395
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-157906874
That make sense.
Which do you prefer? When Spark is future version, we can either print some warning log or raise exception to not working with it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: ZEPPELIN-441 Make SparkInterprete...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/446#issuecomment-158543989
Test added. and version check in SparkSqlInterperter, SparkInterpreter, PySparkInterpreter.
Ready to merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---