You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by lichenglin <gi...@git.apache.org> on 2017/01/06 07:02:07 UTC
[GitHub] zeppelin pull request #1856: remove org.apache.spark.sql.hive.HiveSharedStat...
GitHub user lichenglin opened a pull request:
https://github.com/apache/zeppelin/pull/1856
remove org.apache.spark.sql.hive.HiveSharedState class check
### What is this PR for?
remove org.apache.spark.sql.hive.HiveSharedState class check
because it has been removed since spark 2.1.x
### What type of PR is it?
[Bug Fix ]
### What is the Jira issue?
[ZEPPELIN-1909](https://issues.apache.org/jira/browse/ZEPPELIN-1909)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lichenglin/zeppelin master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1856.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1856
----
commit 3f90485d1c05d863c6957f6ae9879b7491fb2bfa
Author: lichenglin <li...@126.com>
Date: 2017-01-06T06:38:04Z
make hive enable under spark 2.1.0
remove org.apache.spark.sql.hive.HiveSharedState class check .
Because this class has been removed from spark since 2.1.0
https://issues.apache.org/jira/browse/ZEPPELIN-1909
commit 60c3c04cfd7b04da088c9a615b28df7c2962d27a
Author: lichenglin <li...@126.com>
Date: 2017-01-06T06:54:25Z
Merge pull request #1 from lichenglin/lichenglin-patch-1
make hive enable under spark 2.1.0
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1856
looks good, why doesn't CI tests for Spark 2.1 fails for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...
Posted by lichenglin <gi...@git.apache.org>.
Github user lichenglin commented on the issue:
https://github.com/apache/zeppelin/pull/1856
I don't know either\uff0cIt fails with spark 1.6.1.
May be the class check is still needed by 1.6,1?I'm not sure.
How can I trigger the build once more on travis-cli?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...
Posted by lichenglin <gi...@git.apache.org>.
Github user lichenglin commented on the issue:
https://github.com/apache/zeppelin/pull/1856
e...
It works on my local mac with spark 2.1.0.
And pass the build with 2.1.0.
I seems that the build fails with
{code}
"env": "PYTHON=\"2\" SCALA_VER=\"2.10\" SPARK_VER=\"1.6.1\" HADOOP_VER=\"2.6\" PROFILE=\"-Pspark-1.6 -Phadoop-2.6 -Ppyspark\" BUILD_FLAG=\"package -pl spark,python -am -DskipTests -DskipRat\" TEST_FLAG=\"verify -DskipRat\" TEST_PROJECTS=\"-pl zeppelin-interpreter,zeppelin-display,spark-dependencies,spark,python -Dtest=org.apache.zeppelin.spark.PySpark*Test,org.apache.zeppelin.python.* -Dpyspark.test.exclude='' -DfailIfNoTests=false\"",
{code}
I think we'd better reopen the PR and build once more.
Maybe the class check is still necessary for spark 1.6.1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1856
it's only the python2 + spark 1.6 profile. It ran fine with the other spark 1.6 profile so I think we are good.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1856
I don't think we need to check for all 3 classes, so I believe this fix is good.
what I don't understand is why it is failing for you but running just fine in travis CI where we have tests for Spark 2.1 - which is why I'm asking.
to trigger test, close this PR and reopen it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1856: remove org.apache.spark.sql.hive.HiveSharedStat...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1856
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1856
would be great to understand why it worked before, but might merge this fix first.
merging if no more comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---