How to efficiently utilize all cores?

["matha.harika" <ma...@gmail.com>]

Hi,

I have a cluster setup with three slaves, 4 cores each(12 cores in total).
When I try to run multiple applications, using 4 cores each, only the first
application is running(with 2,1,1 cores used in corresponding slaves). Every
other application is going to WAIT state. Following the solution provided 
here
<http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Worker-Core-Allocation-td7188.html>  
I set the parameter spark.deploy.spreadout to false. But the problem is not
solved.

Any suggestion in this regard is welcome.

Thanks in advance

Harika



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-efficiently-utilize-all-cores-tp21569.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: How to efficiently utilize all cores?

[Akhil Das <ak...@sigmoidanalytics.com>]

You can look at http://spark.apache.org/docs/1.2.0/job-scheduling.html

I would go with mesos
http://spark.apache.org/docs/1.2.0/running-on-mesos.html

Thanks
Best Regards

On Tue, Feb 10, 2015 at 2:59 PM, matha.harika <ma...@gmail.com>
wrote:

> Hi,
>
> I have a cluster setup with three slaves, 4 cores each(12 cores in total).
> When I try to run multiple applications, using 4 cores each, only the first
> application is running(with 2,1,1 cores used in corresponding slaves).
> Every
> other application is going to WAIT state. Following the solution provided
> here
> <
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Worker-Core-Allocation-td7188.html
> >
> I set the parameter spark.deploy.spreadout to false. But the problem is not
> solved.
>
> Any suggestion in this regard is welcome.
>
> Thanks in advance
>
> Harika
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-efficiently-utilize-all-cores-tp21569.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: How to efficiently utilize all cores?

[Harika <ma...@gmail.com>]

Hi Aplysia,

Thanks for the reply.

Could you be more specific in terms of what part of the document to look at
as I have already seen it and tried a few of the relevant settings for no
use. 







--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-efficiently-utilize-all-cores-tp21569p21597.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org