Some questions about the scheduler.

[Tim Chou <ti...@gmail.com>]

Hi all,

1. I don't understand the difference between NatureOrder-Scheduler and
MRR-Scheduler well.

    It seems that MRR-Scheduler will prevent partitioner and shuffler
running until the reducer finishes; NatureOrder-Scheduler do nothing about
this and will start partitioner and shuffler at first.

2. If there are three leaf nodes in a dag and there is limited resource in
the cluster, no matter which scheduler we use, these three vertices will
run together. But the detail number of tasks of each vertex is depend on
the practical running situation.
    Am I right?

3. Why I cannot find DAGPlan class in our latest version?

Thanks,
Tim

RE: Some questions about the scheduler.

[Bikas Saha <bi...@hortonworks.com>]

Inline



*From:* Tim Chou [mailto:timchou.hit@gmail.com]
*Sent:* Wednesday, July 30, 2014 8:46 AM
*To:* user@tez.apache.org
*Subject:* Some questions about the scheduler.



Hi all,



1. I don't understand the difference between NatureOrder-Scheduler and
MRR-Scheduler well.



    It seems that MRR-Scheduler will prevent partitioner and shuffler
running until the reducer finishes; NatureOrder-Scheduler do nothing about
this and will start partitioner and shuffler at first.

*[Bikas] MRR was written for mapreduce compatibility. Its not intended to
be used for DAGs. You should only look at NaturalOrder.*



2. If there are three leaf nodes in a dag and there is limited resource in
the cluster, no matter which scheduler we use, these three vertices will
run together. But the detail number of tasks of each vertex is depend on
the practical running situation.

    Am I right?

*[Bikas] That is correct.*



3. Why I cannot find DAGPlan class in our latest version?

*[Bikas] DAGPlan is created from the proto definition. So you need to
compile the project once for it to be generated during the compilation
process.*



Thanks,

Tim

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.