You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by CharlesYeh <gi...@git.apache.org> on 2015/11/03 09:16:31 UTC

[GitHub] spark pull request: show help messages for start-slave and start-m...

GitHub user CharlesYeh opened a pull request:

    https://github.com/apache/spark/pull/9432

    show help messages for start-slave and start-master

    Addressing https://issues.apache.org/jira/browse/SPARK-11218, mostly copied start-thriftserver.sh.
    ```
    charlesyeh-mbp:spark charlesyeh$ ./sbin/start-master.sh --help
    Usage: Master [options]
    
    Options:
      -i HOST, --ip HOST     Hostname to listen on (deprecated, please use --host or -h) 
      -h HOST, --host HOST   Hostname to listen on
      -p PORT, --port PORT   Port to listen on (default: 7077)
      --webui-port PORT      Port for web UI (default: 8080)
      --properties-file FILE Path to a custom Spark properties file.
                             Default is conf/spark-defaults.conf.
    ```
    ```
    charlesyeh-mbp:spark charlesyeh$ ./sbin/start-slave.sh 
    Usage: Worker [options] <master>
    
    Master must be a URL of the form spark://hostname:port
    
    Options:
      -c CORES, --cores CORES  Number of cores to use
      -m MEM, --memory MEM     Amount of memory to use (e.g. 1000M, 2G)
      -d DIR, --work-dir DIR   Directory to run apps in (default: SPARK_HOME/work)
      -i HOST, --ip IP         Hostname to listen on (deprecated, please use --host or -h)
      -h HOST, --host HOST     Hostname to listen on
      -p PORT, --port PORT     Port to listen on (default: random)
      --webui-port PORT        Port for web UI (default: 8081)
      --properties-file FILE   Path to a custom Spark properties file.
                               Default is conf/spark-defaults.conf.
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/CharlesYeh/spark helpmsg

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/9432.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #9432
    
----
commit ab1abc39fc9e19b4e2e32c431a366acbefce6972
Author: Charles Yeh <ch...@dropbox.com>
Date:   2015-11-03T08:10:01Z

    show help messages for start-slave and start-master

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-153689478
  
    Sorry @CharlesYeh this has a merge conflict now. Could you rebase? Basically the scripts have just changed to use a different constructor for finding the working directory, based on `SPARK_HOME`. See other scripts.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-154179761
  
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-154038797
  
    **[Test build #1984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1984/consoleFull)** for PR 9432 at commit [`720a61c`](https://github.com/apache/spark/commit/720a61c58eca73ba4880772cee238d361da7b0af).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:\n  * `public class JavaAssociationRulesExample `\n  * `public class JavaPrefixSpanExample `\n  * `public class JavaSimpleFPGrowth `\n  * `class StreamInterceptor implements TransportFrameDecoder.Interceptor `\n  * `public final class ChunkFetchSuccess extends ResponseWithBody `\n  * `public abstract class ResponseWithBody implements ResponseMessage `\n  * `public final class StreamFailure implements ResponseMessage `\n  * `public final class StreamRequest implements RequestMessage `\n  * `public final class StreamResponse extends ResponseWithBody `\n  * `public class TransportFrameDecoder extends ChannelInboundHandlerAdapter `\n


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-154852699
  
    **[Test build #2011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2011/consoleFull)** for PR 9432 at commit [`935efb5`](https://github.com/apache/spark/commit/935efb5ef5f35cf912820226da77da81795c21f3).
     * This patch passes all tests.
     * This patch merges cleanly.
     * This patch adds the following public classes _(experimental)_:\n  * `class LDAModel(JavaModelWrapper, JavaSaveable, Loader):`\n


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by CharlesYeh <gi...@git.apache.org>.

Github user CharlesYeh commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-153446371
  
    Changed it to ./sbin/start-worker.sh and ./sbin/start-master.sh


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/9432#discussion_r44003082
  
    --- Diff: sbin/start-master.sh ---
    @@ -23,6 +23,26 @@ if [ -z "${SPARK_HOME}" ]; then
       export SPARK_HOME="$(cd "`dirname "$0"`"/..; pwd)"
     fi
     
    +# Figure out where Spark is installed
    +FWDIR="$(cd "`dirname "$0"`"/..; pwd)"
    --- End diff --
    
    This is wrong now, though; like other scripts it should be using `SPARK_HOME` not `FWDIR`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-154003198
  
    **[Test build #1984 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1984/consoleFull)** for PR 9432 at commit [`720a61c`](https://github.com/apache/spark/commit/720a61c58eca73ba4880772cee238d361da7b0af).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by CharlesYeh <gi...@git.apache.org>.

Github user CharlesYeh commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-153902921
  
    rebased


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by CharlesYeh <gi...@git.apache.org>.

Github user CharlesYeh commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-154740151
  
    Oops my bad, also removed the unnecessary sbin/../ from the other tachyon paths


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by SparkQA <gi...@git.apache.org>.

Github user SparkQA commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-154821359
  
    **[Test build #2007 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2007/consoleFull)** for PR 9432 at commit [`935efb5`](https://github.com/apache/spark/commit/935efb5ef5f35cf912820226da77da81795c21f3).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-153452389
  
    This LGTM being basically a copy of the existing approach in `start-thriftserver.sh` et al. It does not appear to interfere with any existing usage of the scripts since nobody would have legitimately been passing `--help` or `-h` before.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by srowen <gi...@git.apache.org>.

Github user srowen commented on a diff in the pull request:

    https://github.com/apache/spark/pull/9432#discussion_r44181885
  
    --- Diff: sbin/start-master.sh ---
    @@ -30,7 +44,7 @@ START_TACHYON=false
     while (( "$#" )); do
     case $1 in
         --with-tachyon)
    -      if [ ! -e "$sbin"/../tachyon/bin/tachyon ]; then
    +      if [ ! -e "${SPARK_HOME}"/../tachyon/bin/tachyon ]; then
    --- End diff --
    
    Last question: does the `..` need to go away? before it started with `$sbin/..` which I think means the parent of `sbin` which should be the project root?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] spark pull request: [SPARK-11218] [Core] show help messages for st...

Posted by jaceklaskowski <gi...@git.apache.org>.

Github user jaceklaskowski commented on the pull request:

    https://github.com/apache/spark/pull/9432#issuecomment-153362113
  
    Could you change `Usage: Worker [options] <master>` and `Usage: Master [options]` to be the scripts' names themselves not the cryptic Master and Worker?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org