You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by gu...@apache.org on 2019/03/11 02:16:32 UTC
[spark] branch master updated:
[SPARK-23807][FOLLOW-UP][BUILD][TEST-HADOOP3.1] Add test-hadoop3.1 phrase
This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8ab1306 [SPARK-23807][FOLLOW-UP][BUILD][TEST-HADOOP3.1] Add test-hadoop3.1 phrase
8ab1306 is described below
commit 8ab13065f626a12d18f6ac14a4dbf3d258919825
Author: Yuming Wang <yu...@ebay.com>
AuthorDate: Mon Mar 11 11:15:58 2019 +0900
[SPARK-23807][FOLLOW-UP][BUILD][TEST-HADOOP3.1] Add test-hadoop3.1 phrase
## What changes were proposed in this pull request?
Add `test-hadoop3.1` phrase to test Spark against Spark’s Hadoop 3.1 profile.
## How was this patch tested?
Tested on jenkins. This is output:
```
[info] Using build tool sbt with Hadoop profile hadoop3.1 under environment amplab_jenkins
...
[info] Building Spark (w/Hive 1.2.1) using SBT with these arguments: -Phadoop-3.1 -Pkubernetes -Phive-thriftserver -Pkinesis-asl -Pyarn -Pspark-ganglia-lgpl -Phive -Pmesos test:package streaming-kinesis-asl-assembly/assembly
```
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103282/console
Closes #24045 from wangyum/SPARK-23807.
Authored-by: Yuming Wang <yu...@ebay.com>
Signed-off-by: Hyukjin Kwon <gu...@apache.org>
---
dev/run-tests-jenkins.py | 2 ++
dev/run-tests.py | 1 +
2 files changed, 3 insertions(+)
diff --git a/dev/run-tests-jenkins.py b/dev/run-tests-jenkins.py
index 77d751f..e01bcbe 100755
--- a/dev/run-tests-jenkins.py
+++ b/dev/run-tests-jenkins.py
@@ -173,6 +173,8 @@ def main():
os.environ["AMPLAB_JENKINS_BUILD_PROFILE"] = "hadoop2.6"
if "test-hadoop2.7" in ghprb_pull_title:
os.environ["AMPLAB_JENKINS_BUILD_PROFILE"] = "hadoop2.7"
+ if "test-hadoop3.1" in ghprb_pull_title:
+ os.environ["AMPLAB_JENKINS_BUILD_PROFILE"] = "hadoop3.1"
build_display_name = os.environ["BUILD_DISPLAY_NAME"]
build_url = os.environ["BUILD_URL"]
diff --git a/dev/run-tests.py b/dev/run-tests.py
index 4e6f897..122c5c6 100755
--- a/dev/run-tests.py
+++ b/dev/run-tests.py
@@ -274,6 +274,7 @@ def get_hadoop_profiles(hadoop_version):
sbt_maven_hadoop_profiles = {
"hadoop2.7": ["-Phadoop-2.7"],
+ "hadoop3.1": ["-Phadoop-3.1"],
}
if hadoop_version in sbt_maven_hadoop_profiles:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org