You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by liancheng <gi...@git.apache.org> on 2014/10/19 09:51:49 UTC
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/2845
[SPARK-4000][BUILD] Sends archived unit tests logs to Jenkins master
This PR sends archived unit tests logs to the build history directory in Jenkins master, so that we can serve it via HTTP later to help debugging Jenkins build failures.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liancheng/spark log-archive
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2845.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2845
----
commit 4b912f78adbc9e6a1a3ca66bf32b5560d642ad5d
Author: Cheng Lian <li...@databricks.com>
Date: 2014-10-19T07:39:11Z
Sends archived unit tests logs to Jenkins master
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-60345312
Alright, I've modified the pull request builder job to inject the master hostname as `JENKINS_MASTER_HOSTNAME`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59649466
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21889/consoleFull) for PR 2845 at commit [`68c7010`](https://github.com/apache/spark/commit/68c7010748fd275cd4e10ac09d994dc0e61a4e24).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19064925
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
+ local scp_output=$(scp ${log_archive} amp-jenkins-master:${jenkins_build_dir}/${log_archive})
--- End diff --
This hostname is accessible from Jenkins slave nodes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19064927
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
--- End diff --
Yes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19064648
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
--- End diff --
Should we add BUILD_NUMBER in the message that we post? Something like this:
```
[Test build #XXX has started/finished] for PR 2845 at commit 4b912f7 (build $XXX).
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-60345019
This looks good to me. @pwendell, any more feedback? I'd like to get this merged soon in order to help debug a few Jenkins failures.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/2845
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59678062
[Test build #21898 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21898/consoleFull) for PR 2845 at commit [`ac8d9d4`](https://github.com/apache/spark/commit/ac8d9d4ac3912bc099585866b52e8b02507aa0f8).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59678066
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21898/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59643840
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21884/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59674934
[Test build #21898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21898/consoleFull) for PR 2845 at commit [`ac8d9d4`](https://github.com/apache/spark/commit/ac8d9d4ac3912bc099585866b52e8b02507aa0f8).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59671283
Overall this looks good, had a few minor questions. One thing we can do next is that we could scp the logs to a web server that we control (e.g. something under people.apache.org) and clean up the old ones every time we copy something over.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-60052480
@pwendell This should be ready to go.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19323244
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
+ local scp_output=$(scp ${log_archive} amp-jenkins-master:${jenkins_build_dir}/${log_archive})
--- End diff --
@liancheng I should be able to modify our pull request builders to inject this variable as `JENKINS_MASTER_HOSTNAME`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59649597
Verified that the log archive was uploaded to the correct location in Jenkins master node.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19064621
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
+ local scp_output=$(scp ${log_archive} amp-jenkins-master:${jenkins_build_dir}/${log_archive})
--- End diff --
I'm confused actually - is "amp-jenkins-master" the current hostname of the master machine?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59642454
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21884/consoleFull) for PR 2845 at commit [`4b912f7`](https://github.com/apache/spark/commit/4b912f78adbc9e6a1a3ca66bf32b5560d642ad5d).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19323760
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
+ local scp_output=$(scp ${log_archive} amp-jenkins-master:${jenkins_build_dir}/${log_archive})
--- End diff --
Cool, thanks :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19064633
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
--- End diff --
Just wondering, will these appear in the tarfile under the full path (e.g. streaming/target/unit-tests.log)? That's ideal.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-60345439
Merge away!
On Thu, Oct 23, 2014 at 10:10 PM, Josh Rosen <no...@github.com>
wrote:
> Alright, I've modified the pull request builder job to inject the master
> hostname as JENKINS_MASTER_HOSTNAME.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/spark/pull/2845#issuecomment-60345312>.
>
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59643839
[QA tests have finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21884/consoleFull) for PR 2845 at commit [`4b912f7`](https://github.com/apache/spark/commit/4b912f78adbc9e6a1a3ca66bf32b5560d642ad5d).
* This patch **fails MiMa tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59649470
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/21889/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-60424731
Shortcut / reminder for folks like me who don't know where JEKNINS_HOME is offhand: the paths to individual logs are like
```
/var/lib/jenkins/jobs/SparkPullRequestBuilder/builds/2014-10-24_09-52-18
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by JoshRosen <gi...@git.apache.org>.
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-60345496
Alright, I'm going to merge this now and I'll hotfix the `JENKINS_MASTER_HOSTNAME` myself later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19059750
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
+ local scp_output=$(scp ${log_archive} amp-jenkins-master:${jenkins_build_dir}/${log_archive})
--- End diff --
It's not good to hardcode Jenkins master hostname here. Should inject an extra environment variable `$MASTER_NODE_NAME` in Jenkins configurations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by liancheng <gi...@git.apache.org>.
Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19065075
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
--- End diff --
The build number is directly accessible from environment variables. [1] I think the PR number information can be useful when reading PR comments via mail.
[1] https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21889/injectedEnvVars/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/2845#issuecomment-59647627
[QA tests have started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/21889/consoleFull) for PR 2845 at commit [`68c7010`](https://github.com/apache/spark/commit/68c7010748fd275cd4e10ac09d994dc0e61a4e24).
* This patch merges cleanly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-4000][BUILD] Sends archived unit tests ...
Posted by nchammas <gi...@git.apache.org>.
Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2845#discussion_r19064710
--- Diff: dev/run-tests-jenkins ---
@@ -92,12 +92,39 @@ function post_message () {
echo " > api_response: ${api_response}" >&2
echo " > data: ${data}" >&2
fi
-
+
if [ "$curl_status" -eq 0 ] && [ "$http_code" -eq "201" ]; then
echo " > Post successful."
fi
}
+function send_archived_logs () {
+ echo "Archiving unit tests logs..."
+
+ local log_files=$(find . -name "unit-tests.log")
+
+ if [ -z "$log_files" ]; then
+ echo "> No log files found." >&2
+ else
+ local log_archive="unit-tests-logs.tar.gz"
+ echo "$log_files" | xargs tar czf ${log_archive}
+
+ local jenkins_build_dir=${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}
--- End diff --
That could be useful. We may also be able to do away with the "for PR XXXX" part since that's kinda redundant.
Note that you can currently get the build number from the build URL in the existing messages posted to GitHub.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org