You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by pw...@apache.org on 2015/01/12 21:25:29 UTC
spark git commit: SPARK-5172 [BUILD] spark-examples-***.jar shades a
wrong Hadoop distribution
Repository: spark
Updated Branches:
refs/heads/master a3978f3e1 -> aff49a3ee
SPARK-5172 [BUILD] spark-examples-***.jar shades a wrong Hadoop distribution
In addition to the `hadoop-2.x` profiles in the parent POM, there is actually another set of profiles in `examples` that has to be activated differently to get the right Hadoop 1 vs 2 flavor of HBase. This wasn't actually used in making Hadoop 2 distributions, hence the problem.
To reduce complexity, I suggest merging them with the parent POM profiles, which is possible now.
You'll see this changes appears to update the HBase version, but actually, the default 0.94 version was not being used. HBase is only used in examples, and the examples POM always chose one profile or the other that updated the version to 0.98.x anyway.
Author: Sean Owen <so...@cloudera.com>
Closes #3992 from srowen/SPARK-5172 and squashes the following commits:
17830d9 [Sean Owen] Control hbase hadoop1/2 flavor in the parent POM with existing hadoop-2.x profiles
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/aff49a3e
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/aff49a3e
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/aff49a3e
Branch: refs/heads/master
Commit: aff49a3ee1a91d730590f4a0b9ac485fd52dc8bf
Parents: a3978f3
Author: Sean Owen <so...@cloudera.com>
Authored: Mon Jan 12 12:15:34 2015 -0800
Committer: Patrick Wendell <pw...@gmail.com>
Committed: Mon Jan 12 12:15:34 2015 -0800
----------------------------------------------------------------------
examples/pom.xml | 23 -----------------------
pom.xml | 5 ++++-
2 files changed, 4 insertions(+), 24 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/aff49a3e/examples/pom.xml
----------------------------------------------------------------------
diff --git a/examples/pom.xml b/examples/pom.xml
index 002d445..4b92147 100644
--- a/examples/pom.xml
+++ b/examples/pom.xml
@@ -393,29 +393,6 @@
</dependencies>
</profile>
<profile>
- <id>hbase-hadoop2</id>
- <activation>
- <property>
- <name>hbase.profile</name>
- <value>hadoop2</value>
- </property>
- </activation>
- <properties>
- <hbase.version>0.98.7-hadoop2</hbase.version>
- </properties>
- </profile>
- <profile>
- <id>hbase-hadoop1</id>
- <activation>
- <property>
- <name>!hbase.profile</name>
- </property>
- </activation>
- <properties>
- <hbase.version>0.98.7-hadoop1</hbase.version>
- </properties>
- </profile>
- <profile>
<!-- We add a source directory specific to Scala 2.10 since Kafka
only works with it -->
<id>scala-2.10</id>
http://git-wip-us.apache.org/repos/asf/spark/blob/aff49a3e/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 0403c00..f4466e5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -122,7 +122,7 @@
<hadoop.version>1.0.4</hadoop.version>
<protobuf.version>2.4.1</protobuf.version>
<yarn.version>${hadoop.version}</yarn.version>
- <hbase.version>0.94.6</hbase.version>
+ <hbase.version>0.98.7-hadoop1</hbase.version>
<hbase.artifact>hbase</hbase.artifact>
<flume.version>1.4.0</flume.version>
<zookeeper.version>3.4.5</zookeeper.version>
@@ -1466,6 +1466,7 @@
<properties>
<hadoop.version>2.2.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
+ <hbase.version>0.98.7-hadoop2</hbase.version>
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
</properties>
</profile>
@@ -1476,6 +1477,7 @@
<hadoop.version>2.3.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
+ <hbase.version>0.98.7-hadoop2</hbase.version>
<commons.math3.version>3.1.1</commons.math3.version>
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
</properties>
@@ -1487,6 +1489,7 @@
<hadoop.version>2.4.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
+ <hbase.version>0.98.7-hadoop2</hbase.version>
<commons.math3.version>3.1.1</commons.math3.version>
<avro.mapred.classifier>hadoop2</avro.mapred.classifier>
</properties>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org