Re: Adding GPUs/FPGAs into the resources

[Muhuan Huang <mh...@cs.ucla.edu>]

*--resources flag* is great. I don't know that before.

Thank you all for the pointers!

Muhuan



On Wed, Mar 26, 2014 at 3:06 PM, Erik Erlandson <ej...@redhat.com> wrote:

>
>
> ----- Original Message -----
> > I think many can agree that *this is a useful feature, but be forewarned,
> > "Thar be dragons in this space!"
> >
> > There is a high degree of variability across GPU's, their abilities, and
> > apis. That shouldn't deter folks, but it may be wise to leverage other
> > communities on how they approached this problem, and learn from their
> wisdom
> > as well as mistakes.
> >
> > - Simpler is better
> > - Think non-fungible
> > - Script for discovery
> >
> > A couple of quick refs:
> > older ref:
> https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToManageGpus
> >
> http://techblog.netflix.com/2014/02/distributed-neural-networks-with-gpus.html
> > I'll let Erik point out the other refs, he has spent a large amount of
> time
> > in this area, and has looked @ (N) resources.
>
>
>
>
> +1 to thinking about 'non-fungible' resources as early as possible.
>  Allowing resources to be assigned with individual identities requires more
> machinery.  HTCondor has recently been adding support for non-fungible
> resources:
> https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=4141
>
> I've spent more time thinking about managing heterogeneous resources, than
> non-fungible resources, per se.   Some possibly useful links to my thoughts
> (with a htcondor slant) are:
>
>
> http://erikerlandson.github.io/blog/2012/11/13/rethinking-the-semantics-of-group-quotas-and-slot-weights-for-heterogeneous-and-multidimensional-compute-resources/
>
> http://erikerlandson.github.io/blog/2012/11/15/rethinking-the-semantics-of-group-quotas-and-slot-weights-claim-capacity-model/
>
> http://erikerlandson.github.io/blog/2012/11/26/rethinking-the-semantics-of-group-quotas-and-slot-weights-computing-claim-capacity-from-consumption-policy/
>
> https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=ConsumptionPolicies
>
>


-- 
Muhuan Huang
Dept. of Computer Science, UCLA