You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Sergiy Matusevych <se...@gmail.com> on 2017/04/27 05:28:00 UTC
REEF runs from Spark!
Hi fellow reefers,
After almost a year of refactorings and bugfixes, I was able to run REEF
application from Spark today. Woo-hoo! All I tried so far was Hello REEF
application running from a Spark driver on YARN, but other Java/REEF/YARN
applications should also run from Spark just fine. I'll keep testing it
tomorrow.
There is a PR pending: https://github.com/apache/reef/pull/1299
We still have a lot of work to do - we need to try (and write unit tests!)
for various failure scenarios, test on different versions of YARN, fix
outstanding related bugs, such as
https://issues.apache.org/jira/browse/REEF-1782 , and, most importantly,
make sure we can launch .NET applications from Spark.
There is a lot of fun work to do - please check out the JIRA at
https://issues.apache.org/jira/browse/REEF and send us your pull requests!
Cheers,
Sergiy.
P.S. I have a sample Spark+REEF app in a private repo. We will open source
it soon, although there is not much to open source - it's basically a
REEF-on-REEF example (namely, a try block from
org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted almost
verbatim into the Spark driver.
Re: REEF runs from Spark!
Posted by Sergiy Matusevych <se...@gmail.com>.
On Thu, Apr 27, 2017 at 2:17 PM, Douglas Service <ds...@gmail.com> wrote:
I believe this needs to be tested on a spark cluster in addition to normal
> tests? If so can I use an HDInsight Ubuntu Spark cluster?
>
Hi Doug,
PR https://github.com/apache/reef/pull/1299 fixes the way REEF packages
resources for the YARN job. It does not require Spark - it just so happened
that running REEF on Spark revealed a bug in that part of code. Also, if
something breaks there, it will impact the entire project. Therefore,
before we merge that PR, we have to make sure all our unit tests still pass
locally and on YARN. Later, when we open source our Spark+REEF example, we
can add Spark-specific tests to it, but I wouldn't add Spark dependencies
to REEF itself.
Thank you,
Sergiy.
Re: REEF runs from Spark!
Posted by Douglas Service <ds...@gmail.com>.
I believe this needs to be tested on a spark cluster in addition to normal
tests? If so can I use an HDInsight Ubuntu Spark cluster?
On Thu, Apr 27, 2017 at 1:36 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> Thank you guys! Now please someone merge
> https://github.com/apache/reef/pull/1299 :-)
>
> On Thu, Apr 27, 2017 at 1:13 PM, Douglas Service <ds...@gmail.com>
> wrote:
>
> > Congratulations Sergiy!
> >
> > On Thu, Apr 27, 2017 at 6:52 PM, Markus Weimer <ma...@weimo.de> wrote:
> >
> > > Fantastic news! Thanks so much for this!
> > >
> > > Next, we should make this one of our examples :)
> > >
> > > Markus
> > >
> > > On Thu, Apr 27, 2017 at 10:13 AM, Rogan Carr <ro...@gmail.com>
> > wrote:
> > > > Nice work Sergiy!
> > > >
> > > > Best,
> > > > Rogan
> > > >
> > > > On Wed, Apr 26, 2017 at 10:55 PM, Sergiy Matusevych <
> > > > sergiy.matusevych@gmail.com> wrote:
> > > >
> > > >> Thank you Gon!
> > > >> Now we need to fix the remaining issues and we'll have an awesome
> 0.16
> > > >> release. Lean in, reef devs! :)
> > > >>
> > > >> -- Sergiy.
> > > >>
> > > >> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Great job!
> > > >> > This is awesome! Very exciting!
> > > >> >
> > > >> > -Gon
> > > >> >
> > > >> > On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> > > >> > sergiy.matusevych@gmail.com> wrote:
> > > >> >
> > > >> > > Hi fellow reefers,
> > > >> > >
> > > >> > > After almost a year of refactorings and bugfixes, I was able to
> > run
> > > >> REEF
> > > >> > > application from Spark today. Woo-hoo! All I tried so far was
> > Hello
> > > >> REEF
> > > >> > > application running from a Spark driver on YARN, but other
> > > >> Java/REEF/YARN
> > > >> > > applications should also run from Spark just fine. I'll keep
> > > testing it
> > > >> > > tomorrow.
> > > >> > >
> > > >> > > There is a PR pending: https://github.com/apache/reef/pull/1299
> > > >> > >
> > > >> > > We still have a lot of work to do - we need to try (and write
> unit
> > > >> > tests!)
> > > >> > > for various failure scenarios, test on different versions of
> YARN,
> > > fix
> > > >> > > outstanding related bugs, such as
> > > >> > > https://issues.apache.org/jira/browse/REEF-1782 , and, most
> > > >> importantly,
> > > >> > > make sure we can launch .NET applications from Spark.
> > > >> > >
> > > >> > > There is a lot of fun work to do - please check out the JIRA at
> > > >> > > https://issues.apache.org/jira/browse/REEF and send us your
> pull
> > > >> > requests!
> > > >> > >
> > > >> > > Cheers,
> > > >> > > Sergiy.
> > > >> > >
> > > >> > > P.S. I have a sample Spark+REEF app in a private repo. We will
> > open
> > > >> > source
> > > >> > > it soon, although there is not much to open source - it's
> > basically
> > > a
> > > >> > > REEF-on-REEF example (namely, a try block from
> > > >> > > org.apache.reef.examples.reefonreef.ReefOnReefDriver),
> > transplanted
> > > >> > almost
> > > >> > > verbatim into the Spark driver.
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Byung-Gon Chun
> > > >> >
> > > >>
> > >
> >
>
Re: REEF runs from Spark!
Posted by Sergiy Matusevych <se...@gmail.com>.
Thank you guys! Now please someone merge
https://github.com/apache/reef/pull/1299 :-)
On Thu, Apr 27, 2017 at 1:13 PM, Douglas Service <ds...@gmail.com> wrote:
> Congratulations Sergiy!
>
> On Thu, Apr 27, 2017 at 6:52 PM, Markus Weimer <ma...@weimo.de> wrote:
>
> > Fantastic news! Thanks so much for this!
> >
> > Next, we should make this one of our examples :)
> >
> > Markus
> >
> > On Thu, Apr 27, 2017 at 10:13 AM, Rogan Carr <ro...@gmail.com>
> wrote:
> > > Nice work Sergiy!
> > >
> > > Best,
> > > Rogan
> > >
> > > On Wed, Apr 26, 2017 at 10:55 PM, Sergiy Matusevych <
> > > sergiy.matusevych@gmail.com> wrote:
> > >
> > >> Thank you Gon!
> > >> Now we need to fix the remaining issues and we'll have an awesome 0.16
> > >> release. Lean in, reef devs! :)
> > >>
> > >> -- Sergiy.
> > >>
> > >> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com>
> > wrote:
> > >>
> > >> > Great job!
> > >> > This is awesome! Very exciting!
> > >> >
> > >> > -Gon
> > >> >
> > >> > On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> > >> > sergiy.matusevych@gmail.com> wrote:
> > >> >
> > >> > > Hi fellow reefers,
> > >> > >
> > >> > > After almost a year of refactorings and bugfixes, I was able to
> run
> > >> REEF
> > >> > > application from Spark today. Woo-hoo! All I tried so far was
> Hello
> > >> REEF
> > >> > > application running from a Spark driver on YARN, but other
> > >> Java/REEF/YARN
> > >> > > applications should also run from Spark just fine. I'll keep
> > testing it
> > >> > > tomorrow.
> > >> > >
> > >> > > There is a PR pending: https://github.com/apache/reef/pull/1299
> > >> > >
> > >> > > We still have a lot of work to do - we need to try (and write unit
> > >> > tests!)
> > >> > > for various failure scenarios, test on different versions of YARN,
> > fix
> > >> > > outstanding related bugs, such as
> > >> > > https://issues.apache.org/jira/browse/REEF-1782 , and, most
> > >> importantly,
> > >> > > make sure we can launch .NET applications from Spark.
> > >> > >
> > >> > > There is a lot of fun work to do - please check out the JIRA at
> > >> > > https://issues.apache.org/jira/browse/REEF and send us your pull
> > >> > requests!
> > >> > >
> > >> > > Cheers,
> > >> > > Sergiy.
> > >> > >
> > >> > > P.S. I have a sample Spark+REEF app in a private repo. We will
> open
> > >> > source
> > >> > > it soon, although there is not much to open source - it's
> basically
> > a
> > >> > > REEF-on-REEF example (namely, a try block from
> > >> > > org.apache.reef.examples.reefonreef.ReefOnReefDriver),
> transplanted
> > >> > almost
> > >> > > verbatim into the Spark driver.
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Byung-Gon Chun
> > >> >
> > >>
> >
>
Re: REEF runs from Spark!
Posted by Douglas Service <ds...@gmail.com>.
Congratulations Sergiy!
On Thu, Apr 27, 2017 at 6:52 PM, Markus Weimer <ma...@weimo.de> wrote:
> Fantastic news! Thanks so much for this!
>
> Next, we should make this one of our examples :)
>
> Markus
>
> On Thu, Apr 27, 2017 at 10:13 AM, Rogan Carr <ro...@gmail.com> wrote:
> > Nice work Sergiy!
> >
> > Best,
> > Rogan
> >
> > On Wed, Apr 26, 2017 at 10:55 PM, Sergiy Matusevych <
> > sergiy.matusevych@gmail.com> wrote:
> >
> >> Thank you Gon!
> >> Now we need to fix the remaining issues and we'll have an awesome 0.16
> >> release. Lean in, reef devs! :)
> >>
> >> -- Sergiy.
> >>
> >> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com>
> wrote:
> >>
> >> > Great job!
> >> > This is awesome! Very exciting!
> >> >
> >> > -Gon
> >> >
> >> > On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> >> > sergiy.matusevych@gmail.com> wrote:
> >> >
> >> > > Hi fellow reefers,
> >> > >
> >> > > After almost a year of refactorings and bugfixes, I was able to run
> >> REEF
> >> > > application from Spark today. Woo-hoo! All I tried so far was Hello
> >> REEF
> >> > > application running from a Spark driver on YARN, but other
> >> Java/REEF/YARN
> >> > > applications should also run from Spark just fine. I'll keep
> testing it
> >> > > tomorrow.
> >> > >
> >> > > There is a PR pending: https://github.com/apache/reef/pull/1299
> >> > >
> >> > > We still have a lot of work to do - we need to try (and write unit
> >> > tests!)
> >> > > for various failure scenarios, test on different versions of YARN,
> fix
> >> > > outstanding related bugs, such as
> >> > > https://issues.apache.org/jira/browse/REEF-1782 , and, most
> >> importantly,
> >> > > make sure we can launch .NET applications from Spark.
> >> > >
> >> > > There is a lot of fun work to do - please check out the JIRA at
> >> > > https://issues.apache.org/jira/browse/REEF and send us your pull
> >> > requests!
> >> > >
> >> > > Cheers,
> >> > > Sergiy.
> >> > >
> >> > > P.S. I have a sample Spark+REEF app in a private repo. We will open
> >> > source
> >> > > it soon, although there is not much to open source - it's basically
> a
> >> > > REEF-on-REEF example (namely, a try block from
> >> > > org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted
> >> > almost
> >> > > verbatim into the Spark driver.
> >> > >
> >> >
> >> >
> >> >
> >> > --
> >> > Byung-Gon Chun
> >> >
> >>
>
Re: REEF runs from Spark!
Posted by Markus Weimer <ma...@weimo.de>.
Fantastic news! Thanks so much for this!
Next, we should make this one of our examples :)
Markus
On Thu, Apr 27, 2017 at 10:13 AM, Rogan Carr <ro...@gmail.com> wrote:
> Nice work Sergiy!
>
> Best,
> Rogan
>
> On Wed, Apr 26, 2017 at 10:55 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
>> Thank you Gon!
>> Now we need to fix the remaining issues and we'll have an awesome 0.16
>> release. Lean in, reef devs! :)
>>
>> -- Sergiy.
>>
>> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
>>
>> > Great job!
>> > This is awesome! Very exciting!
>> >
>> > -Gon
>> >
>> > On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
>> > sergiy.matusevych@gmail.com> wrote:
>> >
>> > > Hi fellow reefers,
>> > >
>> > > After almost a year of refactorings and bugfixes, I was able to run
>> REEF
>> > > application from Spark today. Woo-hoo! All I tried so far was Hello
>> REEF
>> > > application running from a Spark driver on YARN, but other
>> Java/REEF/YARN
>> > > applications should also run from Spark just fine. I'll keep testing it
>> > > tomorrow.
>> > >
>> > > There is a PR pending: https://github.com/apache/reef/pull/1299
>> > >
>> > > We still have a lot of work to do - we need to try (and write unit
>> > tests!)
>> > > for various failure scenarios, test on different versions of YARN, fix
>> > > outstanding related bugs, such as
>> > > https://issues.apache.org/jira/browse/REEF-1782 , and, most
>> importantly,
>> > > make sure we can launch .NET applications from Spark.
>> > >
>> > > There is a lot of fun work to do - please check out the JIRA at
>> > > https://issues.apache.org/jira/browse/REEF and send us your pull
>> > requests!
>> > >
>> > > Cheers,
>> > > Sergiy.
>> > >
>> > > P.S. I have a sample Spark+REEF app in a private repo. We will open
>> > source
>> > > it soon, although there is not much to open source - it's basically a
>> > > REEF-on-REEF example (namely, a try block from
>> > > org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted
>> > almost
>> > > verbatim into the Spark driver.
>> > >
>> >
>> >
>> >
>> > --
>> > Byung-Gon Chun
>> >
>>
Re: REEF runs from Spark!
Posted by Rogan Carr <ro...@gmail.com>.
Nice work Sergiy!
Best,
Rogan
On Wed, Apr 26, 2017 at 10:55 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> Thank you Gon!
> Now we need to fix the remaining issues and we'll have an awesome 0.16
> release. Lean in, reef devs! :)
>
> -- Sergiy.
>
> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
>
> > Great job!
> > This is awesome! Very exciting!
> >
> > -Gon
> >
> > On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> > sergiy.matusevych@gmail.com> wrote:
> >
> > > Hi fellow reefers,
> > >
> > > After almost a year of refactorings and bugfixes, I was able to run
> REEF
> > > application from Spark today. Woo-hoo! All I tried so far was Hello
> REEF
> > > application running from a Spark driver on YARN, but other
> Java/REEF/YARN
> > > applications should also run from Spark just fine. I'll keep testing it
> > > tomorrow.
> > >
> > > There is a PR pending: https://github.com/apache/reef/pull/1299
> > >
> > > We still have a lot of work to do - we need to try (and write unit
> > tests!)
> > > for various failure scenarios, test on different versions of YARN, fix
> > > outstanding related bugs, such as
> > > https://issues.apache.org/jira/browse/REEF-1782 , and, most
> importantly,
> > > make sure we can launch .NET applications from Spark.
> > >
> > > There is a lot of fun work to do - please check out the JIRA at
> > > https://issues.apache.org/jira/browse/REEF and send us your pull
> > requests!
> > >
> > > Cheers,
> > > Sergiy.
> > >
> > > P.S. I have a sample Spark+REEF app in a private repo. We will open
> > source
> > > it soon, although there is not much to open source - it's basically a
> > > REEF-on-REEF example (namely, a try block from
> > > org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted
> > almost
> > > verbatim into the Spark driver.
> > >
> >
> >
> >
> > --
> > Byung-Gon Chun
> >
>
Re: REEF runs from Spark!
Posted by Sergiy Matusevych <se...@gmail.com>.
Hi Taegeon,
I don't know - there is a discussion about 0.16 in another thread. If you
ask me, I would much prefer to see issues
https://issues.apache.org/jira/browse/REEF-1782 and
https://issues.apache.org/jira/browse/REEF-1715 fixed before the release.
Cheers,
Sergiy.
On Thu, Apr 27, 2017 at 7:25 PM, Tae-Geon Um <ta...@gmail.com> wrote:
> Great news!
>
> So, do you want to release 0.16 after the remaining issues are resolved?
> Or, is it okay to release 0.16 right now (after the pending PR is merged)?
>
> Taegeon
>
> > On Apr 27, 2017, at 2:55 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
> >
> > Thank you Gon!
> > Now we need to fix the remaining issues and we'll have an awesome 0.16
> > release. Lean in, reef devs! :)
> >
> > -- Sergiy.
> >
> > On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com>
> wrote:
> >
> >> Great job!
> >> This is awesome! Very exciting!
> >>
> >> -Gon
> >>
> >> On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> >> sergiy.matusevych@gmail.com> wrote:
> >>
> >>> Hi fellow reefers,
> >>>
> >>> After almost a year of refactorings and bugfixes, I was able to run
> REEF
> >>> application from Spark today. Woo-hoo! All I tried so far was Hello
> REEF
> >>> application running from a Spark driver on YARN, but other
> Java/REEF/YARN
> >>> applications should also run from Spark just fine. I'll keep testing it
> >>> tomorrow.
> >>>
> >>> There is a PR pending: https://github.com/apache/reef/pull/1299
> >>>
> >>> We still have a lot of work to do - we need to try (and write unit
> >> tests!)
> >>> for various failure scenarios, test on different versions of YARN, fix
> >>> outstanding related bugs, such as
> >>> https://issues.apache.org/jira/browse/REEF-1782 , and, most
> importantly,
> >>> make sure we can launch .NET applications from Spark.
> >>>
> >>> There is a lot of fun work to do - please check out the JIRA at
> >>> https://issues.apache.org/jira/browse/REEF and send us your pull
> >> requests!
> >>>
> >>> Cheers,
> >>> Sergiy.
> >>>
> >>> P.S. I have a sample Spark+REEF app in a private repo. We will open
> >> source
> >>> it soon, although there is not much to open source - it's basically a
> >>> REEF-on-REEF example (namely, a try block from
> >>> org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted
> >> almost
> >>> verbatim into the Spark driver.
> >>>
> >>
> >>
> >>
> >> --
> >> Byung-Gon Chun
> >>
>
>
Re: REEF runs from Spark!
Posted by Tae-Geon Um <ta...@gmail.com>.
Great news!
So, do you want to release 0.16 after the remaining issues are resolved?
Or, is it okay to release 0.16 right now (after the pending PR is merged)?
Taegeon
> On Apr 27, 2017, at 2:55 PM, Sergiy Matusevych <se...@gmail.com> wrote:
>
> Thank you Gon!
> Now we need to fix the remaining issues and we'll have an awesome 0.16
> release. Lean in, reef devs! :)
>
> -- Sergiy.
>
> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
>
>> Great job!
>> This is awesome! Very exciting!
>>
>> -Gon
>>
>> On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
>> sergiy.matusevych@gmail.com> wrote:
>>
>>> Hi fellow reefers,
>>>
>>> After almost a year of refactorings and bugfixes, I was able to run REEF
>>> application from Spark today. Woo-hoo! All I tried so far was Hello REEF
>>> application running from a Spark driver on YARN, but other Java/REEF/YARN
>>> applications should also run from Spark just fine. I'll keep testing it
>>> tomorrow.
>>>
>>> There is a PR pending: https://github.com/apache/reef/pull/1299
>>>
>>> We still have a lot of work to do - we need to try (and write unit
>> tests!)
>>> for various failure scenarios, test on different versions of YARN, fix
>>> outstanding related bugs, such as
>>> https://issues.apache.org/jira/browse/REEF-1782 , and, most importantly,
>>> make sure we can launch .NET applications from Spark.
>>>
>>> There is a lot of fun work to do - please check out the JIRA at
>>> https://issues.apache.org/jira/browse/REEF and send us your pull
>> requests!
>>>
>>> Cheers,
>>> Sergiy.
>>>
>>> P.S. I have a sample Spark+REEF app in a private repo. We will open
>> source
>>> it soon, although there is not much to open source - it's basically a
>>> REEF-on-REEF example (namely, a try block from
>>> org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted
>> almost
>>> verbatim into the Spark driver.
>>>
>>
>>
>>
>> --
>> Byung-Gon Chun
>>
Re: REEF runs from Spark!
Posted by Sergiy Matusevych <se...@gmail.com>.
Thank you Gon!
Now we need to fix the remaining issues and we'll have an awesome 0.16
release. Lean in, reef devs! :)
-- Sergiy.
On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
> Great job!
> This is awesome! Very exciting!
>
> -Gon
>
> On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
> > Hi fellow reefers,
> >
> > After almost a year of refactorings and bugfixes, I was able to run REEF
> > application from Spark today. Woo-hoo! All I tried so far was Hello REEF
> > application running from a Spark driver on YARN, but other Java/REEF/YARN
> > applications should also run from Spark just fine. I'll keep testing it
> > tomorrow.
> >
> > There is a PR pending: https://github.com/apache/reef/pull/1299
> >
> > We still have a lot of work to do - we need to try (and write unit
> tests!)
> > for various failure scenarios, test on different versions of YARN, fix
> > outstanding related bugs, such as
> > https://issues.apache.org/jira/browse/REEF-1782 , and, most importantly,
> > make sure we can launch .NET applications from Spark.
> >
> > There is a lot of fun work to do - please check out the JIRA at
> > https://issues.apache.org/jira/browse/REEF and send us your pull
> requests!
> >
> > Cheers,
> > Sergiy.
> >
> > P.S. I have a sample Spark+REEF app in a private repo. We will open
> source
> > it soon, although there is not much to open source - it's basically a
> > REEF-on-REEF example (namely, a try block from
> > org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted
> almost
> > verbatim into the Spark driver.
> >
>
>
>
> --
> Byung-Gon Chun
>
Re: REEF runs from Spark!
Posted by Byung-Gon Chun <bg...@gmail.com>.
Great job!
This is awesome! Very exciting!
-Gon
On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> Hi fellow reefers,
>
> After almost a year of refactorings and bugfixes, I was able to run REEF
> application from Spark today. Woo-hoo! All I tried so far was Hello REEF
> application running from a Spark driver on YARN, but other Java/REEF/YARN
> applications should also run from Spark just fine. I'll keep testing it
> tomorrow.
>
> There is a PR pending: https://github.com/apache/reef/pull/1299
>
> We still have a lot of work to do - we need to try (and write unit tests!)
> for various failure scenarios, test on different versions of YARN, fix
> outstanding related bugs, such as
> https://issues.apache.org/jira/browse/REEF-1782 , and, most importantly,
> make sure we can launch .NET applications from Spark.
>
> There is a lot of fun work to do - please check out the JIRA at
> https://issues.apache.org/jira/browse/REEF and send us your pull requests!
>
> Cheers,
> Sergiy.
>
> P.S. I have a sample Spark+REEF app in a private repo. We will open source
> it soon, although there is not much to open source - it's basically a
> REEF-on-REEF example (namely, a try block from
> org.apache.reef.examples.reefonreef.ReefOnReefDriver), transplanted almost
> verbatim into the Spark driver.
>
--
Byung-Gon Chun