You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by zasdfgbnm <gi...@git.apache.org> on 2016/07/02 11:31:19 UTC
[GitHub] spark pull request #14027: Spark daemon run.sh
GitHub user zasdfgbnm opened a pull request:
https://github.com/apache/spark/pull/14027
Spark daemon run.sh
## What changes were proposed in this pull request?
I add some shell scripts to support running master and slave foreground, which makes it more convenient to write systemd service and run on HPC.
## How was this patch tested?
I test it manually and it works well on my computer
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zasdfgbnm/spark spark-daemon-run.sh
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14027.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14027
----
commit 7b99b15656daa430081b1952bc090d3824d916ce
Author: Xiang Gao <qa...@gmail.com>
Date: 2016-07-01T14:22:05Z
split spark-daemon.sh to allow foreground
commit 700247e5cae5bfd33d24a54c133eef2b5a28df96
Author: Xiang Gao <qa...@gmail.com>
Date: 2016-07-01T15:09:46Z
fix bugs to make it work
commit 6a80b7986f831abaf5cdf23c42528055f4999b86
Author: Xiang Gao <qa...@gmail.com>
Date: 2016-07-01T15:26:13Z
fixes
commit 3095187162a821292eabedcd44c18932dbf73ca6
Author: Xiang Gao <qa...@gmail.com>
Date: 2016-07-01T15:27:11Z
starting->running
commit 1954640ccca1a1da7903455d67f1abdc2db0fd73
Author: Xiang Gao <qa...@gmail.com>
Date: 2016-07-01T15:29:21Z
add run-master.sh
commit b96f4e1c4a28ffc893779221100080c5d31a2508
Author: Xiang Gao <qa...@gmail.com>
Date: 2016-07-01T15:36:26Z
add support run-slave.sh
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14027
Please don't open a duplicate JIRA. You can see the discussion on the previous issue and JIRA, which is WontFix, mostly because it ultimately wasn't followed up on. It's not out of the quesiton but you should comment on the JIRA with details about why you think it's important first.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Posted by zasdfgbnm <gi...@git.apache.org>.
Github user zasdfgbnm commented on the issue:
https://github.com/apache/spark/pull/14027
I followed that PR (https://github.com/apache/spark/pull/3881)
"I was under the impression all the shell scripts were getting refactored and that this patch had become obsolete. I agree it's best to close this out." by @hellertime
I'm not sure what happened to that. It seems that the previous PR was closed because the scripts had changed a lot before that PR was merged.
I didn't see anything that support running in foreground and nobody mentioned in that issue that running in foreground is useless and will never be supported.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Posted by zasdfgbnm <gi...@git.apache.org>.
Github user zasdfgbnm commented on the issue:
https://github.com/apache/spark/pull/14027
OK, but will the status of the original JIRA keep being "won't fix", if someone decide to reopen the discussion on it?
Can anyone give a comment on whether the running on foreground feature are welcome in spark upstream?
Also, if there has already been a good solution to run in foreground easily enough (so that my PR will be useless), could anyone send me a link to the document describing this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #14027: [SPARK-16352] Spark daemon run.sh
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/14027
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: Spark daemon run.sh
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/14027
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/14027
I'm suggesting you keep the conversation together with the original JIRA, at least. I don't know if it was going to be merged or not, but it was not updated by the author in any event.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: Spark daemon run.sh
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/14027
(I think It might be nicer if this contribution follows https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #14027: [SPARK-16352] Spark daemon run.sh
Posted by zasdfgbnm <gi...@git.apache.org>.
Github user zasdfgbnm commented on the issue:
https://github.com/apache/spark/pull/14027
I see similar pull request here:
https://github.com/apache/spark/pull/3881
But I didn't get what happened to that and why it was closed without adding any support for running in foreground.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org