You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Tobin Baker <td...@cs.washington.edu> on 2015/12/07 23:51:10 UTC
Specifying locality constraints for Driver
Hi, our initial users for Myria on REEF would like to be able to launch the
driver on a particular node (because they have just enough memory on the
non-coordinator nodes for a single Task evaluator and don't want to
allocate enough slack memory on every node for the driver in case it gets
randomly assigned to that node). I spent some time grepping REEF source (
https://github.com/apache/reef/blob/master/lang/java/reef-common/src/main/java/org/apache/reef/client/DriverConfiguration.java,
https://github.com/apache/reef/blob/master/lang/java/reef-runtime-yarn/src/main/java/org/apache/reef/runtime/yarn/driver/YarnDriverConfiguration.java)
for such a configuration option, but came up empty. Am I missing something?
(Note that there may be a YARN bug that prevents this from working even if
I dropped down to YARN:
http://stackoverflow.com/questions/29302659/run-my-own-application-master-on-a-specific-node-in-a-yarn-cluster
.)
Re: Specifying locality constraints for Driver
Posted by Byung-Gon Chun <bg...@gmail.com>.
Thanks, Tobin!
A related topic is to use node labels for evaluators.
Suppose we have heterogeneous machines (e.g., machines with or without
GPUs) in the cluster. We can use node labels to distinguish the machines.
This is another good issue to contribute. :)
On Tue, Dec 8, 2015 at 8:38 AM, Tobin Baker <td...@cs.washington.edu>
wrote:
> Thanks for the tip about node labels! I'll see if I can get this working at
> the YARN level (I did something similar with node locality before that was
> added in the last release). Created a JIRA for this issue and assigned to
> myself: https://issues.apache.org/jira/browse/REEF-1048
>
>
>
> On Mon, Dec 7, 2015 at 2:59 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
>
> > Hi Tobin,
> >
> > This's a requested feature for our customers as well in Korea. Driver
> runs
> > on a more powerful machine in the cluster.
> > There's no locality setter in YARN ApplicationSubmissionContext.
> >
> >
> https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.html#setNodeLabelExpression(java.lang.String)
> >
> > But we can achieve the effect you want by using node labels. This's not
> > currently exposed through REEF, though. A good item to contribute. :)
> >
> > Cheers,
> > Gon
> >
> > On Tue, Dec 8, 2015 at 7:51 AM, Tobin Baker <td...@cs.washington.edu>
> > wrote:
> >
> > > Hi, our initial users for Myria on REEF would like to be able to launch
> > the
> > > driver on a particular node (because they have just enough memory on
> the
> > > non-coordinator nodes for a single Task evaluator and don't want to
> > > allocate enough slack memory on every node for the driver in case it
> gets
> > > randomly assigned to that node). I spent some time grepping REEF
> source (
> > >
> > >
> >
> https://github.com/apache/reef/blob/master/lang/java/reef-common/src/main/java/org/apache/reef/client/DriverConfiguration.java
> > > ,
> > >
> > >
> >
> https://github.com/apache/reef/blob/master/lang/java/reef-runtime-yarn/src/main/java/org/apache/reef/runtime/yarn/driver/YarnDriverConfiguration.java
> > > )
> > > for such a configuration option, but came up empty. Am I missing
> > something?
> > > (Note that there may be a YARN bug that prevents this from working even
> > if
> > > I dropped down to YARN:
> > >
> > >
> >
> http://stackoverflow.com/questions/29302659/run-my-own-application-master-on-a-specific-node-in-a-yarn-cluster
> > > .)
> > >
> >
> >
> >
> > --
> > Byung-Gon Chun
> >
>
--
Byung-Gon Chun
Re: Specifying locality constraints for Driver
Posted by Tobin Baker <td...@cs.washington.edu>.
Thanks for the tip about node labels! I'll see if I can get this working at
the YARN level (I did something similar with node locality before that was
added in the last release). Created a JIRA for this issue and assigned to
myself: https://issues.apache.org/jira/browse/REEF-1048
On Mon, Dec 7, 2015 at 2:59 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
> Hi Tobin,
>
> This's a requested feature for our customers as well in Korea. Driver runs
> on a more powerful machine in the cluster.
> There's no locality setter in YARN ApplicationSubmissionContext.
>
> https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.html#setNodeLabelExpression(java.lang.String)
>
> But we can achieve the effect you want by using node labels. This's not
> currently exposed through REEF, though. A good item to contribute. :)
>
> Cheers,
> Gon
>
> On Tue, Dec 8, 2015 at 7:51 AM, Tobin Baker <td...@cs.washington.edu>
> wrote:
>
> > Hi, our initial users for Myria on REEF would like to be able to launch
> the
> > driver on a particular node (because they have just enough memory on the
> > non-coordinator nodes for a single Task evaluator and don't want to
> > allocate enough slack memory on every node for the driver in case it gets
> > randomly assigned to that node). I spent some time grepping REEF source (
> >
> >
> https://github.com/apache/reef/blob/master/lang/java/reef-common/src/main/java/org/apache/reef/client/DriverConfiguration.java
> > ,
> >
> >
> https://github.com/apache/reef/blob/master/lang/java/reef-runtime-yarn/src/main/java/org/apache/reef/runtime/yarn/driver/YarnDriverConfiguration.java
> > )
> > for such a configuration option, but came up empty. Am I missing
> something?
> > (Note that there may be a YARN bug that prevents this from working even
> if
> > I dropped down to YARN:
> >
> >
> http://stackoverflow.com/questions/29302659/run-my-own-application-master-on-a-specific-node-in-a-yarn-cluster
> > .)
> >
>
>
>
> --
> Byung-Gon Chun
>
Re: Specifying locality constraints for Driver
Posted by Byung-Gon Chun <bg...@gmail.com>.
Hi Tobin,
This's a requested feature for our customers as well in Korea. Driver runs
on a more powerful machine in the cluster.
There's no locality setter in YARN ApplicationSubmissionContext.
https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.html#setNodeLabelExpression(java.lang.String)
But we can achieve the effect you want by using node labels. This's not
currently exposed through REEF, though. A good item to contribute. :)
Cheers,
Gon
On Tue, Dec 8, 2015 at 7:51 AM, Tobin Baker <td...@cs.washington.edu>
wrote:
> Hi, our initial users for Myria on REEF would like to be able to launch the
> driver on a particular node (because they have just enough memory on the
> non-coordinator nodes for a single Task evaluator and don't want to
> allocate enough slack memory on every node for the driver in case it gets
> randomly assigned to that node). I spent some time grepping REEF source (
>
> https://github.com/apache/reef/blob/master/lang/java/reef-common/src/main/java/org/apache/reef/client/DriverConfiguration.java
> ,
>
> https://github.com/apache/reef/blob/master/lang/java/reef-runtime-yarn/src/main/java/org/apache/reef/runtime/yarn/driver/YarnDriverConfiguration.java
> )
> for such a configuration option, but came up empty. Am I missing something?
> (Note that there may be a YARN bug that prevents this from working even if
> I dropped down to YARN:
>
> http://stackoverflow.com/questions/29302659/run-my-own-application-master-on-a-specific-node-in-a-yarn-cluster
> .)
>
--
Byung-Gon Chun