spark / mesos and GPU resources

[Jackie Tung <ja...@drive.ai>]

Hi,

Does Spark support GPU resources reservation (much like it does for CPU and memory) on a Mesos cluster manager?

Mesos added GPU recently as a first class resource type.  I tried out the spark.mesos.constraints variable optimistically “gpu:1” which did not work for me.

This is really one of the final hurdles for us to adopt Spark for our workloads.  Spark looks really great otherwise for us.

Many Thanks,
Jackie
-- 
 

The information in this email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this email 
by anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be 
taken in reliance on it, is prohibited and may be unlawful.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org

Re: spark / mesos and GPU resources

[Timothy Chen <tn...@gmail.com>]

Hi Jackie,

That doesn't work because GPU is a first class reource for Mesos
starting with 1.0, and the patch to enable it from me is still in PR.
I've done a demo last Spark Summit SF about Spark/Mesos/GPU and you
can look at the video to see how it works.

Feel free to try out the PR
(https://github.com/apache/spark/pull/14644) and let me know if you
have any problems or questions.

Tim


On Wed, Sep 28, 2016 at 4:48 PM, Jackie Tung <ja...@drive.ai> wrote:
> Hi,
>
> Does Spark support GPU resources reservation (much like it does for CPU and memory) on a Mesos cluster manager?
>
> Mesos added GPU recently as a first class resource type.  I tried out the spark.mesos.constraints variable optimistically “gpu:1” which did not work for me.
>
> This is really one of the final hurdles for us to adopt Spark for our workloads.  Spark looks really great otherwise for us.
>
> Many Thanks,
> Jackie
> --
>
>
> The information in this email is confidential and may be legally
> privileged. It is intended solely for the addressee. Access to this email
> by anyone else is unauthorized. If you are not the intended recipient, any
> disclosure, copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org