You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Will Berkeley (Code Review)" <ge...@cloudera.org> on 2016/10/30 23:18:41 UTC
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Will Berkeley has uploaded a new change for review.
http://gerrit.cloudera.org:8080/4886
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 and Scala 2.11
This patch adds build profiles to build artifacts for Spark 2.0
and Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one. It should be a temporary
solution; a permanent one in the future would either drop support
for 1.6 or maintain separate modules compatible with 1.6 and 2.0.
The original artifactId is preserved, so the four possible
artifactIds are
- kudu-spark_2.10
- kudu-spark_2.11
- kudu-spark2_2.10
- kudu-spark2_2.11
All spark/scala combinations compile and pass the tests.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
3 files changed, 84 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/1
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/6/java/README.md
File java/README.md:
PS6, Line 54: or Scala 2.11
> 'or Scala 2.11' should be removed now, right?
Done
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#6).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 92 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/6
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/1/java/kudu-spark/pom.xml
File java/kudu-spark/pom.xml:
Line 185: <profiles>
We'll need to run tests against more than just the default profile (2.10+spark1) on jenkins. Also, can you add something in the Java client's README about the different build profiles?
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/1//COMMIT_MSG
Commit Message:
PS1, Line 15: maintain separate modules compatible with 1.6 and 2.0.
So why not do this right? You think Spark 1.x doesn't have much longer to live?
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
Patch Set 6: Code-Review+1
(1 comment)
LGTM except for small nit. JD, do we need to do anything to get nightlies to build under both profiles?
http://gerrit.cloudera.org:8080/#/c/4886/6/java/README.md
File java/README.md:
PS6, Line 54: or Scala 2.11
'or Scala 2.11' should be removed now, right?
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#9).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 91 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/9
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 9
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
Patch Set 10:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/10/build-support/jenkins/build-and-test.sh
File build-support/jenkins/build-and-test.sh:
PS10, Line 356: mvn test
this should probably have $MVN_FLAGS like above, to be safe.
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/1//COMMIT_MSG
Commit Message:
PS1, Line 15: maintain separate modules compatible with 1.6 and 2.0.
> So why not do this right? You think Spark 1.x doesn't have much longer to l
I don't know how much longer Spark 1.x will be in common use. The issue is more that the change is so minor it's easy to keep one module. Happy to split it up though if you think that's best. The split will definitely happen if/when we want features that only 2.x provides. I guess that's also an argument for it: we can develop 2.x-dependent features without more hassle with a separate module. It's also an argument against since there's two possibly diverging modules to try and maintain feature parity between.
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has submitted this change and it was merged.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Reviewed-on: http://gerrit.cloudera.org:8080/4886
Tested-by: Kudu Jenkins
Reviewed-by: Jean-Daniel Cryans <jd...@apache.org>
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 91 insertions(+), 1 deletion(-)
Approvals:
Jean-Daniel Cryans: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 12
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/1/java/kudu-spark/pom.xml
File java/kudu-spark/pom.xml:
Line 204: <artifactId>scala-parser-combinators_2.11</artifactId>
> What is this used for?
In 2.11, some things, like the parser-combinator library used in SQLContext, were factored out of the scala std lib and into their own modules.
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#8).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 91 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/8
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#11).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 91 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/11
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Posted by "Dan Burkert (Code Review)" <ge...@cloudera.org>.
Dan Burkert has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4886/1/java/kudu-spark/pom.xml
File java/kudu-spark/pom.xml:
Line 204: <artifactId>scala-parser-combinators_2.11</artifactId>
What is this used for?
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#3).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 96 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/3
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#4).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 92 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/4
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#5).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 92 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/5
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
Patch Set 11:
Confirmed that kudu-spark (and only kudu-spark) was rebuilt and retested with the proper profile.
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#2).
Change subject: [kudu-spark] Allow build against Spark 2.0 and Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 and Scala 2.11
This patch adds build profiles to build artifacts for Spark 2.0
and Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one. It should be a temporary
solution; a permanent one in the future would either drop support
for 1.6 or maintain separate modules compatible with 1.6 and 2.0.
The original artifactId is preserved, so the four possible
artifactIds are
- kudu-spark_2.10
- kudu-spark_2.11
- kudu-spark2_2.10
- kudu-spark2_2.11
All spark/scala combinations compile and pass the tests.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
4 files changed, 104 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/2
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Will Berkeley has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
Patch Set 11:
> It looks like the Scala 2.11 build has new warnings that we should
> maybe fix in another patch, a bunch of
>
> 05:06:19 [WARNING] /home/jenkins-slave/workspace/kudu-0/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala:31:
> warning: no valid targets for annotation on value batchSize - it is
> discarded unused. You may specify targets with meta-annotations,
> e.g. @(transient @param)
> 05:06:19 [WARNING] @transient batchSize: Integer,
>
> and a bunch of
>
> 05:06:27 [WARNING] /home/jenkins-slave/workspace/kudu-0/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala:339:
> warning: This catches all Throwables. If this is really intended,
> use `case _ : Throwable` to clear this warning.
> 05:06:27 [WARNING] case _ => fail()
Most definitely. I've already started looking at them.
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has posted comments on this change.
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
Patch Set 11: Code-Review+2
It looks like the Scala 2.11 build has new warnings that we should maybe fix in another patch, a bunch of
05:06:19 [WARNING] /home/jenkins-slave/workspace/kudu-0/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduRDD.scala:31: warning: no valid targets for annotation on value batchSize - it is discarded unused. You may specify targets with meta-annotations, e.g. @(transient @param)
05:06:19 [WARNING] @transient batchSize: Integer,
and a bunch of
05:06:27 [WARNING] /home/jenkins-slave/workspace/kudu-0/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala:339: warning: This catches all Throwables. If this is really intended, use `case _ : Throwable` to clear this warning.
05:06:27 [WARNING] case _ => fail()
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#10).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 91 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/10
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
Posted by "Will Berkeley (Code Review)" <ge...@cloudera.org>.
Hello Dan Burkert, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4886
to look at the new patch set (#7).
Change subject: [kudu-spark] Allow build against Spark 2.0 with Scala 2.11
......................................................................
[kudu-spark] Allow build against Spark 2.0 with Scala 2.11
This patch adds build profiles to build an artifact for Spark 2.0
with Scala 2.11. There was one tiny code change to package.scala
required to support Spark 2.0. To handle this, the different
versions of the package.scala reside in different directories and
the build profile includes the proper one.
The two artifactIds are
- kudu-spark_2.10
- kudu-spark2_2.11
The Jenkins job builds and tests both the default and spark2_2.11
profiles.
Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
---
M build-support/jenkins/build-and-test.sh
M java/README.md
M java/kudu-spark/pom.xml
R java/kudu-spark/src/main/spark1/org/apache/kudu/spark/kudu/package.scala
A java/kudu-spark/src/main/spark2/org/apache/kudu/spark/kudu/package.scala
5 files changed, 91 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/86/4886/7
--
To view, visit http://gerrit.cloudera.org:8080/4886
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie055ac1e8f7be7cb992124afa0b601376db14f91
Gerrit-PatchSet: 7
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Will Berkeley <wd...@gmail.com>
Gerrit-Reviewer: Chris George <ch...@rms.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>