You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crail.apache.org by Bairen YI <by...@connect.ust.hk> on 2018/02/17 12:49:28 UTC
C (or language agnostic) API for Crail
HI folks,
I have been looking into you guys’ work for a long time and it is great to see Crail accepted as an Apache Incubator project.
I authored the GPU Direct RDMA transport for TensorFlow (https://github.com/tensorflow/tensorflow/pull/11392), and I would love to see how we could design an end-to-end zero-copy dataflow from Crail to various deep learning framework such as TensorFlow (https://dl.acm.org/citation.cfm?doid=3123878.3131975).
Is there any roadmap for Crail as a standalone language-independent FileSystem/Cache service with C API? That would really ease the integration into non-JVM based third party system. It does not have to be HDFS compatible if that brings extra performance cost.
Best,
Bairen
Re: C (or language agnostic) API for Crail
Posted by Patrick Stuedi <ps...@gmail.com>.
Thanks for sharing the doc. This is very much in line with the work we
are doing since 6-7 years (https://github.com/zrlio,
https://researcher.watson.ibm.com/researcher/view.php?person=zurich-stu).
Please let us know if there is a specific aspect you would like to get
involved in the context of Crail, ideally around the topics I proposed
earlier. It would be great to get your help.
On Mon, Feb 19, 2018 at 1:37 AM, Bairen YI <by...@connect.ust.hk> wrote:
> Hi Patrick, Yuval,
>
> I’m having a broader proposal to incorporate TensorFlow into Spark to enable
> transparent e2e hardware acceleration of data flow pipeline.
>
> A zero copy distributed cache service such as Crail will play a central role
> in this design. Without it, systems like Yahoo’s TensorFlow on Spark or
> Databricks’ TensorFrame may incur large overheads transferring data across
> system boundaries.
>
> Attached please find a deck of slides describing the central idea of such
> design.
>
RE: C (or language agnostic) API for Crail
Posted by Yuval Degani <yu...@mellanox.com>.
Thanks Qingchun, actually I was already on the thread.
Patrick, Bairen, maybe we should hold a meeting and discuss how we can make this work?
Thanks,
Yuval.
From: Qingchun Song
Sent: Saturday, February 17, 2018 3:41 PM
To: Patrick Stuedi <ps...@gmail.com>; dev@crail.apache.org
Cc: kaichen@cse.ust.hk; Yuval Degani <yu...@mellanox.com>
Subject: Re: C (or language agnostic) API for Crail
Add Yuval in loop. Yuval is SparkRDMA developer.
Qingchun Song
Cell: +8613501218637
E-Mail: qingchun@mellanox.com<ma...@mellanox.com>
获取 Outlook<https://aka.ms/qtex0l> for iOS
________________________________
From: Patrick Stuedi <ps...@gmail.com>>
Sent: Saturday, February 17, 2018 11:00:50 PM
To: dev@crail.apache.org<ma...@crail.apache.org>
Cc: kaichen@cse.ust.hk<ma...@cse.ust.hk>; Qingchun Song
Subject: Re: C (or language agnostic) API for Crail
That's great, one of the main goals of crail being an apache incubator project is to get more people involved in the development of crail. I've been following your contributions to tensorflow, nice work! Collaborating in this context (incl mxnet) would be very interesting. There are multiple ways to go. Once we have the core c++ client we could need help in the developmen of the various bindings (rdma, tcp, for storage and rpc). Or we could need help in leveraging crail in tensirflow and mxnet (param server, storage of the model > dram). Let us know where you see opportinities.
On Feb 17, 2018 3:36 PM, "Bairen YI" <by...@connect.ust.hk>> wrote:
Hi Patrick,
That would be fantastic. In fact we would love to get more involved as our lab in HKUST has partnered with MLNX to codevelop datacenter scale AI software solution (TensorFlow and Apache MXNet), and we could encourage a couple of students contributing code to Crail at this very stage if we see fit. It could also bring novel system/networking research opportunities to our lab.
Let me know how we could better work together.
Best,
Bairen
> On 17 Feb 2018, at 22:19, Patrick Stuedi <ps...@gmail.com>> wrote:
>
> Hi Bairen,
>
> Your comment is just on spot. The development of a c++ Api for crail is one
> of the top items on the roadmap, in partical to facilitate the integration
> into tensorflow and serverless. In fact i started drafting a prototype two
> weeks ago that i wanted to share soon. If you are interested in helping let
> us know!
>
>
>
> On Feb 17, 2018 1:49 PM, "Bairen YI" <by...@connect.ust.hk>> wrote:
>
> HI folks,
>
> I have been looking into you guys’ work for a long time and it is great to
> see Crail accepted as an Apache Incubator project.
>
> I authored the GPU Direct RDMA transport for TensorFlow (
> https://github.com/tensorflow/tensorflow/pull/11392<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftensorflow%2Ftensorflow%2Fpull%2F11392&data=02%7C01%7Cqingchun%40mellanox.com%7C305036780e114ee781f908d576173d21%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636544764585899945&sdata=pDzYCjIYsPEdLd4Ggw6tIMUusSpPnZ01ULw5f%2F%2Bki44%3D&reserved=0>), and I would love to
> see how we could design an end-to-end zero-copy dataflow from Crail to
> various deep learning framework such as TensorFlow (
> https://dl.acm.org/citation.cfm?doid=3123878.3131975<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fcitation.cfm%3Fdoid%3D3123878.3131975&data=02%7C01%7Cqingchun%40mellanox.com%7C305036780e114ee781f908d576173d21%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636544764585899945&sdata=4qFyBQGRQRhqDcFjtLdA1J6ibloQ%2F6mI%2BZaWDST6rr8%3D&reserved=0>).
>
> Is there any roadmap for Crail as a standalone language-independent
> FileSystem/Cache service with C API? That would really ease the integration
> into non-JVM based third party system. It does not have to be HDFS
> compatible if that brings extra performance cost.
>
> Best,
> Bairen
Re: C (or language agnostic) API for Crail
Posted by Qingchun Song <Qi...@Mellanox.com>.
Add Yuval in loop. Yuval is SparkRDMA developer.
Qingchun Song
Cell: +8613501218637
E-Mail: qingchun@mellanox.com
获取 Outlook<https://aka.ms/qtex0l> for iOS
________________________________
From: Patrick Stuedi <ps...@gmail.com>
Sent: Saturday, February 17, 2018 11:00:50 PM
To: dev@crail.apache.org
Cc: kaichen@cse.ust.hk; Qingchun Song
Subject: Re: C (or language agnostic) API for Crail
That's great, one of the main goals of crail being an apache incubator project is to get more people involved in the development of crail. I've been following your contributions to tensorflow, nice work! Collaborating in this context (incl mxnet) would be very interesting. There are multiple ways to go. Once we have the core c++ client we could need help in the developmen of the various bindings (rdma, tcp, for storage and rpc). Or we could need help in leveraging crail in tensirflow and mxnet (param server, storage of the model > dram). Let us know where you see opportinities.
On Feb 17, 2018 3:36 PM, "Bairen YI" <by...@connect.ust.hk>> wrote:
Hi Patrick,
That would be fantastic. In fact we would love to get more involved as our lab in HKUST has partnered with MLNX to codevelop datacenter scale AI software solution (TensorFlow and Apache MXNet), and we could encourage a couple of students contributing code to Crail at this very stage if we see fit. It could also bring novel system/networking research opportunities to our lab.
Let me know how we could better work together.
Best,
Bairen
> On 17 Feb 2018, at 22:19, Patrick Stuedi <ps...@gmail.com>> wrote:
>
> Hi Bairen,
>
> Your comment is just on spot. The development of a c++ Api for crail is one
> of the top items on the roadmap, in partical to facilitate the integration
> into tensorflow and serverless. In fact i started drafting a prototype two
> weeks ago that i wanted to share soon. If you are interested in helping let
> us know!
>
>
>
> On Feb 17, 2018 1:49 PM, "Bairen YI" <by...@connect.ust.hk>> wrote:
>
> HI folks,
>
> I have been looking into you guys’ work for a long time and it is great to
> see Crail accepted as an Apache Incubator project.
>
> I authored the GPU Direct RDMA transport for TensorFlow (
> https://github.com/tensorflow/tensorflow/pull/11392<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftensorflow%2Ftensorflow%2Fpull%2F11392&data=02%7C01%7Cqingchun%40mellanox.com%7C305036780e114ee781f908d576173d21%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636544764585899945&sdata=pDzYCjIYsPEdLd4Ggw6tIMUusSpPnZ01ULw5f%2F%2Bki44%3D&reserved=0>), and I would love to
> see how we could design an end-to-end zero-copy dataflow from Crail to
> various deep learning framework such as TensorFlow (
> https://dl.acm.org/citation.cfm?doid=3123878.3131975<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fcitation.cfm%3Fdoid%3D3123878.3131975&data=02%7C01%7Cqingchun%40mellanox.com%7C305036780e114ee781f908d576173d21%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636544764585899945&sdata=4qFyBQGRQRhqDcFjtLdA1J6ibloQ%2F6mI%2BZaWDST6rr8%3D&reserved=0>).
>
> Is there any roadmap for Crail as a standalone language-independent
> FileSystem/Cache service with C API? That would really ease the integration
> into non-JVM based third party system. It does not have to be HDFS
> compatible if that brings extra performance cost.
>
> Best,
> Bairen
Re: C (or language agnostic) API for Crail
Posted by Patrick Stuedi <ps...@gmail.com>.
Another interesting opportunity would be the development of a gpu storage
tier based on gpu direct
On Feb 17, 2018 4:00 PM, "Patrick Stuedi" <ps...@gmail.com> wrote:
> That's great, one of the main goals of crail being an apache incubator
> project is to get more people involved in the development of crail. I've
> been following your contributions to tensorflow, nice work! Collaborating
> in this context (incl mxnet) would be very interesting. There are multiple
> ways to go. Once we have the core c++ client we could need help in the
> developmen of the various bindings (rdma, tcp, for storage and rpc). Or we
> could need help in leveraging crail in tensirflow and mxnet (param server,
> storage of the model > dram). Let us know where you see opportinities.
>
> On Feb 17, 2018 3:36 PM, "Bairen YI" <by...@connect.ust.hk> wrote:
>
>> Hi Patrick,
>>
>> That would be fantastic. In fact we would love to get more involved as
>> our lab in HKUST has partnered with MLNX to codevelop datacenter scale AI
>> software solution (TensorFlow and Apache MXNet), and we could encourage a
>> couple of students contributing code to Crail at this very stage if we see
>> fit. It could also bring novel system/networking research opportunities to
>> our lab.
>>
>> Let me know how we could better work together.
>>
>> Best,
>> Bairen
>>
>> > On 17 Feb 2018, at 22:19, Patrick Stuedi <ps...@gmail.com> wrote:
>> >
>> > Hi Bairen,
>> >
>> > Your comment is just on spot. The development of a c++ Api for crail is
>> one
>> > of the top items on the roadmap, in partical to facilitate the
>> integration
>> > into tensorflow and serverless. In fact i started drafting a prototype
>> two
>> > weeks ago that i wanted to share soon. If you are interested in helping
>> let
>> > us know!
>> >
>> >
>> >
>> > On Feb 17, 2018 1:49 PM, "Bairen YI" <by...@connect.ust.hk> wrote:
>> >
>> > HI folks,
>> >
>> > I have been looking into you guys’ work for a long time and it is great
>> to
>> > see Crail accepted as an Apache Incubator project.
>> >
>> > I authored the GPU Direct RDMA transport for TensorFlow (
>> > https://github.com/tensorflow/tensorflow/pull/11392), and I would love
>> to
>> > see how we could design an end-to-end zero-copy dataflow from Crail to
>> > various deep learning framework such as TensorFlow (
>> > https://dl.acm.org/citation.cfm?doid=3123878.3131975).
>> >
>> > Is there any roadmap for Crail as a standalone language-independent
>> > FileSystem/Cache service with C API? That would really ease the
>> integration
>> > into non-JVM based third party system. It does not have to be HDFS
>> > compatible if that brings extra performance cost.
>> >
>> > Best,
>> > Bairen
>>
>
Re: C (or language agnostic) API for Crail
Posted by Patrick Stuedi <ps...@gmail.com>.
That's great, one of the main goals of crail being an apache incubator
project is to get more people involved in the development of crail. I've
been following your contributions to tensorflow, nice work! Collaborating
in this context (incl mxnet) would be very interesting. There are multiple
ways to go. Once we have the core c++ client we could need help in the
developmen of the various bindings (rdma, tcp, for storage and rpc). Or we
could need help in leveraging crail in tensirflow and mxnet (param server,
storage of the model > dram). Let us know where you see opportinities.
On Feb 17, 2018 3:36 PM, "Bairen YI" <by...@connect.ust.hk> wrote:
> Hi Patrick,
>
> That would be fantastic. In fact we would love to get more involved as our
> lab in HKUST has partnered with MLNX to codevelop datacenter scale AI
> software solution (TensorFlow and Apache MXNet), and we could encourage a
> couple of students contributing code to Crail at this very stage if we see
> fit. It could also bring novel system/networking research opportunities to
> our lab.
>
> Let me know how we could better work together.
>
> Best,
> Bairen
>
> > On 17 Feb 2018, at 22:19, Patrick Stuedi <ps...@gmail.com> wrote:
> >
> > Hi Bairen,
> >
> > Your comment is just on spot. The development of a c++ Api for crail is
> one
> > of the top items on the roadmap, in partical to facilitate the
> integration
> > into tensorflow and serverless. In fact i started drafting a prototype
> two
> > weeks ago that i wanted to share soon. If you are interested in helping
> let
> > us know!
> >
> >
> >
> > On Feb 17, 2018 1:49 PM, "Bairen YI" <by...@connect.ust.hk> wrote:
> >
> > HI folks,
> >
> > I have been looking into you guys’ work for a long time and it is great
> to
> > see Crail accepted as an Apache Incubator project.
> >
> > I authored the GPU Direct RDMA transport for TensorFlow (
> > https://github.com/tensorflow/tensorflow/pull/11392), and I would love
> to
> > see how we could design an end-to-end zero-copy dataflow from Crail to
> > various deep learning framework such as TensorFlow (
> > https://dl.acm.org/citation.cfm?doid=3123878.3131975).
> >
> > Is there any roadmap for Crail as a standalone language-independent
> > FileSystem/Cache service with C API? That would really ease the
> integration
> > into non-JVM based third party system. It does not have to be HDFS
> > compatible if that brings extra performance cost.
> >
> > Best,
> > Bairen
>
Re: C (or language agnostic) API for Crail
Posted by Bairen YI <by...@connect.ust.hk>.
Hi Patrick,
That would be fantastic. In fact we would love to get more involved as our lab in HKUST has partnered with MLNX to codevelop datacenter scale AI software solution (TensorFlow and Apache MXNet), and we could encourage a couple of students contributing code to Crail at this very stage if we see fit. It could also bring novel system/networking research opportunities to our lab.
Let me know how we could better work together.
Best,
Bairen
> On 17 Feb 2018, at 22:19, Patrick Stuedi <ps...@gmail.com> wrote:
>
> Hi Bairen,
>
> Your comment is just on spot. The development of a c++ Api for crail is one
> of the top items on the roadmap, in partical to facilitate the integration
> into tensorflow and serverless. In fact i started drafting a prototype two
> weeks ago that i wanted to share soon. If you are interested in helping let
> us know!
>
>
>
> On Feb 17, 2018 1:49 PM, "Bairen YI" <by...@connect.ust.hk> wrote:
>
> HI folks,
>
> I have been looking into you guys’ work for a long time and it is great to
> see Crail accepted as an Apache Incubator project.
>
> I authored the GPU Direct RDMA transport for TensorFlow (
> https://github.com/tensorflow/tensorflow/pull/11392), and I would love to
> see how we could design an end-to-end zero-copy dataflow from Crail to
> various deep learning framework such as TensorFlow (
> https://dl.acm.org/citation.cfm?doid=3123878.3131975).
>
> Is there any roadmap for Crail as a standalone language-independent
> FileSystem/Cache service with C API? That would really ease the integration
> into non-JVM based third party system. It does not have to be HDFS
> compatible if that brings extra performance cost.
>
> Best,
> Bairen
Re: C (or language agnostic) API for Crail
Posted by Patrick Stuedi <ps...@gmail.com>.
Hi Bairen,
Your comment is just on spot. The development of a c++ Api for crail is one
of the top items on the roadmap, in partical to facilitate the integration
into tensorflow and serverless. In fact i started drafting a prototype two
weeks ago that i wanted to share soon. If you are interested in helping let
us know!
On Feb 17, 2018 1:49 PM, "Bairen YI" <by...@connect.ust.hk> wrote:
HI folks,
I have been looking into you guys’ work for a long time and it is great to
see Crail accepted as an Apache Incubator project.
I authored the GPU Direct RDMA transport for TensorFlow (
https://github.com/tensorflow/tensorflow/pull/11392), and I would love to
see how we could design an end-to-end zero-copy dataflow from Crail to
various deep learning framework such as TensorFlow (
https://dl.acm.org/citation.cfm?doid=3123878.3131975).
Is there any roadmap for Crail as a standalone language-independent
FileSystem/Cache service with C API? That would really ease the integration
into non-JVM based third party system. It does not have to be HDFS
compatible if that brings extra performance cost.
Best,
Bairen