You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hivemall.apache.org by maropu <gi...@git.apache.org> on 2017/07/20 02:28:19 UTC
[GitHub] incubator-hivemall pull request #103: [HIVEMALL-133][SPARK][WIP] Support spa...
GitHub user maropu opened a pull request:
https://github.com/apache/incubator-hivemall/pull/103
[HIVEMALL-133][SPARK][WIP] Support spark-v2.2 in the hivemalls-spark module
## What changes were proposed in this pull request?
This pr added supports for spark-2.2 in Hivemall.
This pr is currently WIP because:
1. Java7's been dropped in spark-v2.2 and Hivemall still supports it, so we need some entries to check a Java version when `spark-2.2` enabled.
2. We need to move common code into `spark/spark-common`.
## What type of PR is it?
Improvement
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-133
## How was this patch tested?
Existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/maropu/incubator-hivemall HIVEMALL-133
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-hivemall/pull/103.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #103
----
commit 391c769fac54b578e69c91c76ca1c15fd42c9273
Author: Takeshi Yamamuro <ya...@apache.org>
Date: 2017-07-20T02:18:05Z
Support spark-2.2
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
ok
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
@maropu check #106 as well.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
See #112
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by coveralls <gi...@git.apache.org>.
Github user coveralls commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
[](https://coveralls.io/builds/12470438)
Coverage remained the same at 40.747% when pulling **391c769fac54b578e69c91c76ca1c15fd42c9273 on maropu:HIVEMALL-133** into **11bd1f83e68a7fbd2e0cc7143303e35e32edf692 on apache:master**.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
I guess we can use different `javac` for each Maven module.
https://maven.apache.org/plugins/maven-compiler-plugin/examples/compile-using-different-jdk.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
Thanks, I'll check later
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by maropu <gi...@git.apache.org>.
Github user maropu commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
Probably, we need to set both `-target 1.8` and `-Pspark-2.2` (also, we need to enforce users to do so) when we compile them for Spark-v2.2. If users set `-Pspark-2.2` only, it prints errors, or something.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
@maropu see https://github.com/apache/incubator-hivemall/pull/105 for travis CI error.
Would be fixed when https://github.com/apache/incubator-hivemall/pull/105 is merged into master and rebase to master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
We cannot go Java 8 yet since the stable version of Hadoop still depends on Java 7.
Better to keep minimum dependencies to Java 7 for Hivemall core module.
Also, `-target 1.7` is not supported in JDK 8.
https://stackoverflow.com/questions/21014338/can-you-compile-a-class-using-java-8-and-then-use-it-in-java-7
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-hivemall issue #103: [HIVEMALL-133][SPARK][WIP] Support spark-v2.2...
Posted by myui <gi...@git.apache.org>.
Github user myui commented on the issue:
https://github.com/apache/incubator-hivemall/pull/103
@maropu I would like to merge this PR but it seems that you deleted the branch or repository.
---
[GitHub] incubator-hivemall pull request #103: [HIVEMALL-133][SPARK][WIP] Support spa...
Posted by maropu <gi...@git.apache.org>.
Github user maropu closed the pull request at:
https://github.com/apache/incubator-hivemall/pull/103
---