You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Andy Davidson <An...@SantaCruzIntegration.com> on 2016/07/21 22:44:03 UTC

How to submit app in cluster mode? port 7077 or 6066

I have some very long lived streaming apps. They have been running for
several months. I wonder if something has changed recently? I first started
working with spark-1.3 . I am using the stand alone cluster manager. The way
I would submit my app to run in cluster mode was port 6066


Looking at the spark-1.6 it seems like we are supposed to use port 7077 and
the  new argument 

http://spark.apache.org/docs/latest/submitting-applications.html#launching-a
pplications-with-spark-submit
* --deploy-mode: Whether to deploy your driver on the worker nodes (cluster)
or locally as an external client (client) (default: client) †

Can anyone confirm this. It took me a very long time to figure out how to
get things to run cluster mode.

Thanks

Andy

Re: How to submit app in cluster mode? port 7077 or 6066

Posted by Andy Davidson <An...@SantaCruzIntegration.com>.

Thanks

Andy

From:  Saisai Shao <sa...@gmail.com>
Date:  Thursday, July 21, 2016 at 6:11 PM
To:  Andrew Davidson <An...@SantaCruzIntegration.com>
Cc:  "user @spark" <us...@spark.apache.org>
Subject:  Re: How to submit app in cluster mode? port 7077 or 6066

> I think both 6066 and 7077 can be worked. 6066 is using the REST way to submit
> application, while 7077 is the legacy way. From user's aspect, it should be
> transparent and no need to worry about the difference.
> 
> * URL: spark://hw12100.local:7077
> * REST URL: spark://hw12100.local:6066 (cluster mode)
> 
> Thanks
> Saisai
> 
> On Fri, Jul 22, 2016 at 6:44 AM, Andy Davidson <An...@santacruzintegration.com>
> wrote:
>> I have some very long lived streaming apps. They have been running for
>> several months. I wonder if something has changed recently? I first started
>> working with spark-1.3 . I am using the stand alone cluster manager. The way
>> I would submit my app to run in cluster mode was port 6066
>> 
>> 
>> Looking at the spark-1.6 it seems like we are supposed to use port 7077 and
>> the  new argument
>> 
>> http://spark.apache.org/docs/latest/submitting-applications.html#launching-ap
>> plications-with-spark-submit
>> * --deploy-mode: Whether to deploy your driver on the worker nodes (cluster)
>> or locally as an external client (client) (default: client) †
>> 
>> Can anyone confirm this. It took me a very long time to figure out how to get
>> things to run cluster mode.
>> 
>> Thanks
>> 
>> Andy
>

Re: How to submit app in cluster mode? port 7077 or 6066

Posted by Saisai Shao <sa...@gmail.com>.

I think both 6066 and 7077 can be worked. 6066 is using the REST way to
submit application, while 7077 is the legacy way. From user's aspect, it
should be transparent and no need to worry about the difference.


   - *URL:* spark://hw12100.local:7077
   - *REST URL:* spark://hw12100.local:6066 (cluster mode)


Thanks
Saisai

On Fri, Jul 22, 2016 at 6:44 AM, Andy Davidson <
Andy@santacruzintegration.com> wrote:

> I have some very long lived streaming apps. They have been running for
> several months. I wonder if something has changed recently? I first started
> working with spark-1.3 . I am using the stand alone cluster manager. The
> way I would submit my app to run in cluster mode was port 6066
>
>
> Looking at the spark-1.6 it seems like we are supposed to use port 7077
> and the  new argument
>
>
> http://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit
>
>
>    - --deploy-mode: Whether to deploy your driver on the worker nodes (
>    cluster) or locally as an external client (client) (default: client)
>    *†*
>
>
> Can anyone confirm this. It took me a very long time to figure out how to
> get things to run cluster mode.
>
> Thanks
>
> Andy
>