You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by rapelly kartheek <ka...@gmail.com> on 2014/07/08 18:11:13 UTC

Scheduling in spark

Hi,
  I am a post graduate student, new to spark. I want to understand how
Spark scheduler works. I just have theoretical understanding of DAG
scheduler and the underlying task scheduler.

I want to know, given a job to the framework, after the DAG scheduler
phase, how the scheduling happens??

Can someone help me out as to how to proceed in these lines.  I have some
exposure towards Hadoop schedulers and tools like Mumak simulator for
experiments.

Can someone please tell me how to perform simulations on Spark w.r.to
 schedulers.


Thanks in advance!!!!

Re: Scheduling in spark

Posted by "Kartheek.R" <ka...@gmail.com>.

Thank you Andrew for the updated link.

regards
Karthik



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Scheduling-in-spark-tp9035p9717.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Scheduling in spark

Posted by Andrew Or <an...@databricks.com>.

Here's the most updated version of the same page:
http://spark.apache.org/docs/latest/job-scheduling


2014-07-08 12:44 GMT-07:00 Sujeet Varakhedi <sv...@gopivotal.com>:

> This is a good start:
>
> http://www.eecs.berkeley.edu/~tdas/spark_docs/job-scheduling.html
>
>
> On Tue, Jul 8, 2014 at 9:11 AM, rapelly kartheek <ka...@gmail.com>
> wrote:
>
>> Hi,
>>   I am a post graduate student, new to spark. I want to understand how
>> Spark scheduler works. I just have theoretical understanding of DAG
>> scheduler and the underlying task scheduler.
>>
>> I want to know, given a job to the framework, after the DAG scheduler
>> phase, how the scheduling happens??
>>
>> Can someone help me out as to how to proceed in these lines.  I have some
>> exposure towards Hadoop schedulers and tools like Mumak simulator for
>> experiments.
>>
>> Can someone please tell me how to perform simulations on Spark w.r.to
>>  schedulers.
>>
>>
>> Thanks in advance!!!!
>>
>
>

Re: Scheduling in spark

Posted by "Kartheek.R" <ka...@gmail.com>.

Thank you so much for the link, Sujeet.

regards
Karthik



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Scheduling-in-spark-tp9035p9716.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Scheduling in spark

Posted by Sujeet Varakhedi <sv...@gopivotal.com>.

This is a good start:

http://www.eecs.berkeley.edu/~tdas/spark_docs/job-scheduling.html


On Tue, Jul 8, 2014 at 9:11 AM, rapelly kartheek <ka...@gmail.com>
wrote:

> Hi,
>   I am a post graduate student, new to spark. I want to understand how
> Spark scheduler works. I just have theoretical understanding of DAG
> scheduler and the underlying task scheduler.
>
> I want to know, given a job to the framework, after the DAG scheduler
> phase, how the scheduling happens??
>
> Can someone help me out as to how to proceed in these lines.  I have some
> exposure towards Hadoop schedulers and tools like Mumak simulator for
> experiments.
>
> Can someone please tell me how to perform simulations on Spark w.r.to
>  schedulers.
>
>
> Thanks in advance!!!!
>