You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wujianping10043419 <gi...@git.apache.org> on 2017/05/22 09:42:29 UTC
[GitHub] spark pull request #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS...
GitHub user wujianping10043419 opened a pull request:
https://github.com/apache/spark/pull/18061
[SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is needed only when use…
…rDeployMode is 'client'. If userDeployMode is not 'client', the launcher shouldn't care about DRIVER_EXTRA_JAVA_OPTIONS, nor should it exit abnormally because DRIVER_EXTRA_JAVA_OPTIONS contains Xmx configuration.
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wujianping10043419/spark-1 spark-master-20170519
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18061.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18061
----
commit e85a24d7fed6f79d60a79f7642d743f3b2b491a3
Author: Phoenix_Dad <wu...@zte.com.cn>
Date: 2017-05-22T08:40:28Z
[SPARK- ][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is needed only when userDeployMode is 'client'. If userDeployMode is not 'client', the launcher shouldn't care about DRIVER_EXTRA_JAVA_OPTIONS, nor should it exit abnormally because DRIVER_EXTRA_JAVA_OPTIONS contains Xmx configuration.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/18061
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is nee...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18061
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is nee...
Posted by wujianping10043419 <gi...@git.apache.org>.
Github user wujianping10043419 commented on the issue:
https://github.com/apache/spark/pull/18061
@srowen
The DRIVER_EXTRA_JAVA_OPTIONS is from a file. Maybe, Setting the parameter is an error. But Spark doesn't use it in non-client mode, nobody should pay for food who doesn't eat,so I think the code is not robust.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is nee...
Posted by wujianping10043419 <gi...@git.apache.org>.
Github user wujianping10043419 commented on the issue:
https://github.com/apache/spark/pull/18061
@srowen But the check result in the abnormal exit, because an irrelevant parameters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is nee...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18061
This check is still correct for cluster mode. You should set memory with `--driver-memory`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is nee...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18061
The check is truly trivial, and the check is not actually irrelevant to non-client mode.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18061: [SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is nee...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18061
Although this could be moved, probably, it's not really worth bothering. It isn't actually redundant. If you specify driver memory settings in non-client mode, that should also be an error. Please close this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org