You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by aarondav <gi...@git.apache.org> on 2014/04/03 05:34:17 UTC
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
GitHub user aarondav opened a pull request:
https://github.com/apache/spark/pull/310
SPARK-1404: Always upgrade spark-env.sh vars to environment vars
This was broken when spark-env.sh was made idempotent, as the idempotence check is an environment variable, but the spark-env.sh variables may not have been.
Tested in zsh, bash, and sh.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aarondav/spark SPARK-1404
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/310.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #310
----
commit 6a0e34063f97da38fdaf1cb72a033dabcc4c9b04
Author: Aaron Davidson <aa...@databricks.com>
Date: 2014-04-03T03:11:27Z
SPARK-1404: Always upgrade spark-env.sh vars to environment vars
This was broken when spark-env.sh was made idempotent, as the idempotence
check is an environment variable, but the spark-env.sh variables may
not have been.
Tested in zsh, bash, and sh...
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39529938
LGTM also
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39409526
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39476946
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by andrewor14 <gi...@git.apache.org>.
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39507068
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39409518
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39409966
Merged build finished.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/310
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39409967
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13715/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39476959
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by aarondav <gi...@git.apache.org>.
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/310#discussion_r11238082
--- Diff: bin/spark-shell ---
@@ -145,7 +145,7 @@ function resolve_spark_master(){
fi
if [ -z "$MASTER" ]; then
- MASTER="$DEFAULT_MASTER"
+ export MASTER="$DEFAULT_MASTER"
--- End diff --
This was an unrelated bug in spark-shell
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by ueshin <gi...@git.apache.org>.
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/310#discussion_r11246612
--- Diff: bin/spark-shell ---
@@ -145,7 +145,7 @@ function resolve_spark_master(){
fi
if [ -z "$MASTER" ]; then
- MASTER="$DEFAULT_MASTER"
+ export MASTER="$DEFAULT_MASTER"
--- End diff --
I've just found this bug.
I think you also need to add `export` at [line 130](https://github.com/aarondav/spark/blob/6a0e34063f97da38fdaf1cb72a033dabcc4c9b04/bin/spark-shell#L130) for `-m` option or at [the last part](https://github.com/aarondav/spark/blob/6a0e34063f97da38fdaf1cb72a033dabcc4c9b04/bin/spark-shell#L150) of `resolve_spark_master` function.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by aarondav <gi...@git.apache.org>.
Github user aarondav commented on a diff in the pull request:
https://github.com/apache/spark/pull/310#discussion_r11261202
--- Diff: bin/spark-shell ---
@@ -145,7 +145,7 @@ function resolve_spark_master(){
fi
if [ -z "$MASTER" ]; then
- MASTER="$DEFAULT_MASTER"
+ export MASTER="$DEFAULT_MASTER"
--- End diff --
Thanks, added!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39483328
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: SPARK-1404: Always upgrade spark-env.sh vars t...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/310#issuecomment-39483329
All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13725/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---