You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by hzw19900416 <gi...@git.apache.org> on 2014/08/25 16:14:51 UTC
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
GitHub user hzw19900416 opened a pull request:
https://github.com/apache/spark/pull/2115
[SPARK-3201][YARN-Stable] Make the yarn-client support the extraJavaOptions
it's very inconvenient if we want to set the "-X" java opts in the process of "ExecutorLauncher".
So I add it in yarn-client mode.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hzw19900416/spark yarn-client
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/2115.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2115
----
commit 6a0cf8ac66ee2e0b3af2aa49f106c4e3e5405c92
Author: hzw19900416 <ca...@gmail.com>
Date: 2014-08-25T13:56:22Z
Make the yarn-client support the extraJavaOptions
commit e52501870e7bbe0bc528e8c9b6c9b6637944025c
Author: hzw19900416 <ca...@gmail.com>
Date: 2014-08-25T14:07:59Z
Update ClientBase.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2115#issuecomment-53366467
I'm not working on that issue, so if you'd like to repurpose your patch for that, feel welcome to do so.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
Posted by hzw19900416 <gi...@git.apache.org>.
Github user hzw19900416 commented on the pull request:
https://github.com/apache/spark/pull/2115#issuecomment-53361806
Hi @vanzin ,I agree with you about that "separate options for the 'client mode AM'" is the better way to do so.
And are you doing this job now?
If so, please leave the extra java opts for our users.
Nowadays, it's very inconvenient for me to use the remote debug of ExecutorLauncher
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
Posted by hzw19900416 <gi...@git.apache.org>.
Github user hzw19900416 closed the pull request at:
https://github.com/apache/spark/pull/2115
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/2115#issuecomment-53269379
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2115#issuecomment-53289230
Hi @hzw19900416,
I don't think this is the correct way to do this. Yarn should have separate options for the "client mode AM", and not reuse the driver settings, because the driver will invariably need a lot more resources than the client-mode AM.
Bug SPARK-1953 is the closes I could find to ask for this feature.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-3201][YARN-Stable] Make the yarn-client...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/2115#issuecomment-53349157
SPARK-1507 is also related (maybe both could be merged into a "create separate options to control the client-mode AM resource allocation request" bug).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org