You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Silvio Fiorito <si...@granturing.com> on 2015/03/11 05:39:59 UTC

Re: Is it possible to use windows service to start and stop spark standalone cluster

Have you tried Apache Daemon? http://commons.apache.org/proper/commons-daemon/procrun.html

From: <Wang>, "Ningjun (LNG-NPV)"
Date: Tuesday, March 10, 2015 at 11:47 PM
To: "user@spark.apache.org<ma...@spark.apache.org>"
Subject: Is it possible to use windows service to start and stop spark standalone cluster

We are using spark stand alone cluster on Windows 2008 R2. I can start spark clusters by open an command prompt and run the following

bin\spark-class.cmd org.apache.spark.deploy.master.Master

bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark://mywin.mydomain.com:7077

I can stop spark cluster by pressing Ctril-C.

The problem is that if the machine is reboot, I have to manually start the spark cluster again as above. Is it possible to use a windows service to start cluster? This way when the machine is reboot, the windows service will automatically restart spark cluster. How to stop spark cluster using windows service is also a challenge.

Please advise.

Thanks

Ningjun

Re: Is it possible to use windows service to start and stop spark standalone cluster

Posted by Yana Kadiyska <ya...@gmail.com>.

You might also want to see if TaskScheduler helps with that. I have not
used it with Windows 2008 R2 but it generally does allow you to schedule a
bat file to run on startup

On Wed, Mar 11, 2015 at 10:16 AM, Wang, Ningjun (LNG-NPV) <
ningjun.wang@lexisnexis.com> wrote:
>
> Thanks for the suggestion. I will try that.
>
>
>
> Ningjun
>
>
>
>
>
> From: Silvio Fiorito [mailto:silvio.fiorito@granturing.com]
> Sent: Wednesday, March 11, 2015 12:40 AM
> To: Wang, Ningjun (LNG-NPV); user@spark.apache.org
> Subject: Re: Is it possible to use windows service to start and stop
spark standalone cluster
>
>
>
> Have you tried Apache Daemon?
http://commons.apache.org/proper/commons-daemon/procrun.html
>
>
>
> From: <Wang>, "Ningjun (LNG-NPV)"
> Date: Tuesday, March 10, 2015 at 11:47 PM
> To: "user@spark.apache.org"
> Subject: Is it possible to use windows service to start and stop spark
standalone cluster
>
>
>
> We are using spark stand alone cluster on Windows 2008 R2. I can start
spark clusters by open an command prompt and run the following
>
>
>
> bin\spark-class.cmd org.apache.spark.deploy.master.Master
>
>
>
> bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark://
mywin.mydomain.com:7077
>
>
>
> I can stop spark cluster by pressing Ctril-C.
>
>
>
> The problem is that if the machine is reboot, I have to manually start
the spark cluster again as above. Is it possible to use a windows service
to start cluster? This way when the machine is reboot, the windows service
will automatically restart spark cluster. How to stop spark cluster using
windows service is also a challenge.
>
>
>
> Please advise.
>
>
>
> Thanks
>
>
>
> Ningjun

RE: Is it possible to use windows service to start and stop spark standalone cluster

Posted by "Wang, Ningjun (LNG-NPV)" <ni...@lexisnexis.com>.

Thanks for the suggestion. I will try that.

Ningjun

From: Silvio Fiorito [mailto:silvio.fiorito@granturing.com]
Sent: Wednesday, March 11, 2015 12:40 AM
To: Wang, Ningjun (LNG-NPV); user@spark.apache.org
Subject: Re: Is it possible to use windows service to start and stop spark standalone cluster

Have you tried Apache Daemon? http://commons.apache.org/proper/commons-daemon/procrun.html

From: <Wang>, "Ningjun (LNG-NPV)"
Date: Tuesday, March 10, 2015 at 11:47 PM
To: "user@spark.apache.org<ma...@spark.apache.org>"
Subject: Is it possible to use windows service to start and stop spark standalone cluster

We are using spark stand alone cluster on Windows 2008 R2. I can start spark clusters by open an command prompt and run the following

bin\spark-class.cmd org.apache.spark.deploy.master.Master

bin\spark-class.cmd org.apache.spark.deploy.worker.Worker spark://mywin.mydomain.com:7077

I can stop spark cluster by pressing Ctril-C.

The problem is that if the machine is reboot, I have to manually start the spark cluster again as above. Is it possible to use a windows service to start cluster? This way when the machine is reboot, the windows service will automatically restart spark cluster. How to stop spark cluster using windows service is also a challenge.

Please advise.

Thanks

Ningjun