You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2013/08/23 03:36:37 UTC
question about fair scheduler
hi,i have a question about fair scheduler
doc says "When there is a single app running, that app uses the entire
cluster. When other apps are submitted, resources that free up are assigned
to the new apps, so that each app gets roughly the same amount of
resources",
suppose i have only a big app running ,so it get 100 resource ,as it
running ,it release 20 resource ,that time 2 short app come to run,so each
get
10 resource ,what i think is "fair is for current available resources ,not
the whole resources" ,is it right?
Re: question about fair scheduler
Posted by Sandy Ryza <sa...@cloudera.com>.
That's right that the other 2 apps will end up getting 10 resources each,
but as more resources become released, eventually the cluster will converge
to a fair state. I.e. if the first app requested additional resources
after releasing resources, it would not receive any more until either
another app either grew to more resources than it, or no other app needed
them.
Does that make sense?
-Sandy
On Thu, Aug 22, 2013 at 6:36 PM, ch huang <ju...@gmail.com> wrote:
> hi,i have a question about fair scheduler
> doc says "When there is a single app running, that app uses the entire
> cluster. When other apps are submitted, resources that free up are assigned
> to the new apps, so that each app gets roughly the same amount of
> resources",
> suppose i have only a big app running ,so it get 100 resource ,as it
> running ,it release 20 resource ,that time 2 short app come to run,so each
> get
> 10 resource ,what i think is "fair is for current available
> resources ,not the whole resources" ,is it right?
>
Re: question about fair scheduler
Posted by Sandy Ryza <sa...@cloudera.com>.
That's right that the other 2 apps will end up getting 10 resources each,
but as more resources become released, eventually the cluster will converge
to a fair state. I.e. if the first app requested additional resources
after releasing resources, it would not receive any more until either
another app either grew to more resources than it, or no other app needed
them.
Does that make sense?
-Sandy
On Thu, Aug 22, 2013 at 6:36 PM, ch huang <ju...@gmail.com> wrote:
> hi,i have a question about fair scheduler
> doc says "When there is a single app running, that app uses the entire
> cluster. When other apps are submitted, resources that free up are assigned
> to the new apps, so that each app gets roughly the same amount of
> resources",
> suppose i have only a big app running ,so it get 100 resource ,as it
> running ,it release 20 resource ,that time 2 short app come to run,so each
> get
> 10 resource ,what i think is "fair is for current available
> resources ,not the whole resources" ,is it right?
>
Re: question about fair scheduler
Posted by Sandy Ryza <sa...@cloudera.com>.
That's right that the other 2 apps will end up getting 10 resources each,
but as more resources become released, eventually the cluster will converge
to a fair state. I.e. if the first app requested additional resources
after releasing resources, it would not receive any more until either
another app either grew to more resources than it, or no other app needed
them.
Does that make sense?
-Sandy
On Thu, Aug 22, 2013 at 6:36 PM, ch huang <ju...@gmail.com> wrote:
> hi,i have a question about fair scheduler
> doc says "When there is a single app running, that app uses the entire
> cluster. When other apps are submitted, resources that free up are assigned
> to the new apps, so that each app gets roughly the same amount of
> resources",
> suppose i have only a big app running ,so it get 100 resource ,as it
> running ,it release 20 resource ,that time 2 short app come to run,so each
> get
> 10 resource ,what i think is "fair is for current available
> resources ,not the whole resources" ,is it right?
>
Re: question about fair scheduler
Posted by Sandy Ryza <sa...@cloudera.com>.
That's right that the other 2 apps will end up getting 10 resources each,
but as more resources become released, eventually the cluster will converge
to a fair state. I.e. if the first app requested additional resources
after releasing resources, it would not receive any more until either
another app either grew to more resources than it, or no other app needed
them.
Does that make sense?
-Sandy
On Thu, Aug 22, 2013 at 6:36 PM, ch huang <ju...@gmail.com> wrote:
> hi,i have a question about fair scheduler
> doc says "When there is a single app running, that app uses the entire
> cluster. When other apps are submitted, resources that free up are assigned
> to the new apps, so that each app gets roughly the same amount of
> resources",
> suppose i have only a big app running ,so it get 100 resource ,as it
> running ,it release 20 resource ,that time 2 short app come to run,so each
> get
> 10 resource ,what i think is "fair is for current available
> resources ,not the whole resources" ,is it right?
>