You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@reef.apache.org by Byung-Gon Chun <bg...@gmail.com> on 2017/05/10 06:55:11 UTC

Re: 0.16 release plan

Any update?
It'd be great if we can release 0.16 during the week of ApacheCon.

-Gon

On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com> wrote:

> Unfortunately, we’ve also got a recent build failure in Java side [1],
> which is not reported previously.
> I’ve created an issue [2] to track this failure, and am going to
> investigate it.
>
> Thanks,
> Taegeon
>
> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
> https://travis-ci.org/apache/reef/builds/220731026>
> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
> https://issues.apache.org/jira/browse/REEF-1770>
> > On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <ma...@microsoft.com.INVALID>
> wrote:
> >
> > At least 3 of the issues previously reported under REEF-1462 have
> re-occurred in the past two days (I've reopened corresponding JIRAs and
> attached links to failures). Unfortunately, with the transient failures
> like these one good build is insufficient.
> >
> > It is a known issue, since we're using free access to AppVeyor, our
> builds are sequential and low-priority, so sometimes when a lot of pull
> requests have to be built the build queue takes a while to drain.
> >
> > -Mariia
> >
> > -----Original Message-----
> > From: Byung-Gon Chun [mailto:bgchun@gmail.com]
> > Sent: Wednesday, April 5, 2017 12:11 AM
> > To: dev@reef.apache.org
> > Subject: Re: 0.16 release plan
> >
> > Awesome!
> >
> > There is no build failure in the .Net side with the latest build [1].
> >
> > It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis CI's
> already done. We're still waiting for Appveyor. :(
> >
> > [1]
> > https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com%
> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%2BhaBnkHjFnkwxLh6GiPubCBDb%
> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
> > [2] https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
> data=02%7C01%7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
> 7C636269730940876660&sdata=oF%2F6YP9JkpD%2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
> E9Q%3D&reserved=0
> >
> >
> > On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
> wrote:
> >
> >> Thanks Julia for the work!
> >>
> >> It looks like Java and .NET builds are almost stable, except for the
> >> recent build failure in .NET side [1].
> >> As Julia said in REEF-1406 [2], we would need to wait for time if this
> >> failure is reproduced or not.
> >>
> >> I will wait for a week and call a release vote if there are no build
> >> failures during that time.
> >> Thanks!
> >>
> >> Taegeon
> >>
> >> [1]:
> >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci.ap
> >> pveyor.com%2Fproject%2FApacheSoftwareFoundation%2Freef%2F&data=02%7C01
> >> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2fda1%7C72f98
> >> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&sdata=Y9iO%2B
> >> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
> >> build/1453-master
> >> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fci.a
> >> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com%7C90c
> >> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7
> >> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRntM6avrry5nk
> >> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
> >> [2]:
> >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue
> >> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%7Cmamykhai%40mic
> >> rosoft.com%7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d
> >> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%2F9yenZoGqe%2FkowHza7
> >> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
> >> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissue
> >> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%7Cmamykhai%40mic
> >> rosoft.com%7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d
> >> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%2F9yenZoGqe%2FkowHza7
> >> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
> >>
> >>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
> >> Qiuhe.Wang@microsoft.com.INVALID> wrote:
> >>>
> >>> I have resolved all the .Net test issues for now. The fixes contain
> >>> what
> >> I have identifies so far based on the failures.
> >>>
> >>> I agree with Marria, as they are transit failures, also they failed
> >>> for
> >> multiple reasons sometimes, we need to continue to observe if the
> >> issues come back again.
> >>>
> >>> Thanks,
> >>> Julia
> >>>
> >>> -----Original Message-----
> >>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
> >>> Sent: Thursday, March 23, 2017 5:20 PM
> >>> To: dev@reef.apache.org
> >>> Subject: Re: 0.16 release plan
> >>>
> >>> Thanks Mariia for pointing it out to me.
> >>> Yes. I agree that we need more time to fix all of the transient
> failures.
> >>> After they are resolved, I will wait for some time to ensure that
> >>> they
> >> are not reoccurred.
> >>>
> >>> Thanks!
> >>> Taegeon
> >>>
> >>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
> >>>> <ma...@microsoft.com.INVALID>
> >> wrote:
> >>>>
> >>>> Please note that due to the transient nature of .NET failures, it
> >>>> makes
> >> sense to wait for some time and to observe whether they are actually
> >> fixed or just lying low until the next reoccurrence. We had to reopen
> >> some bugs which looked resolved in the past but then reoccurred.
> >>>>
> >>>>
> >>>> -Mariia
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ________________________________
> >>>> From: Tae-Geon Um <ta...@gmail.com>
> >>>> Sent: Thursday, March 23, 2017 6:51:24 AM
> >>>> To: dev@reef.apache.org
> >>>> Subject: Re: 0.16 release plan
> >>>>
> >>>> Hi,
> >>>>
> >>>> Julia has been doing a great work to resolve the .NET side issues.
> >>>> It looks like she has resolved 3 issues recently (and now 3 issues
> >> remain in .NET side with 1 pending PR).
> >>>>
> >>>> Sergiy and I also have worked for the java side issues, and we've
> >> resolved 1 issue (and 1 issue still remains with 1 pending PR).
> >>>>
> >>>> Because of the unresolved issues (3 .NET side and 1 java side), I
> >>>> think
> >> it would be good to delay the release vote.
> >>>> However, judging from the progress we made, I think all of the
> >>>> issues
> >> could be resolved until at the end of this week or begging of next week.
> >>>>
> >>>> I will call a vote as soon as possible after they are resolved.
> >>>> Thanks!
> >>>>
> >>>> Taegeon
> >>>>
> >>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
> wrote:
> >>>>>
> >>>>> Thank you for all the efforts to make release 0.16 happen!
> >>>>>
> >>>>> Taegeon, could you give us status update? Thanks.
> >>>>>
> >>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
> >>>>> <bg...@gmail.com>
> >> wrote:
> >>>>>
> >>>>>> Thanks for looking at .Net CI failures, Julia!
> >>>>>>
> >>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
> >>>>>>
> >>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
> >>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
> >>>>>>
> >>>>>>> I am working on some of the .Net AppVeyor test failures now.
> >>>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
> >>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
> >>>>>>> To: dev@reef.apache.org
> >>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>
> >>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
> >>>>>>> <ta...@gmail.com>
> >> wrote:
> >>>>>>>
> >>>>>>> Oh, never mind.
> >>>>>>>> I thought that we still need some time to make sure that
> >>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
> >>>>>>>>
> >>>>>>>
> >>>>>>> Great, then we have only two items left! :-) I can confirm that
> >>>>>>> Unmanaged AM works on proper version of YARN; still, we need to
> >>>>>>> address the Java issues (that is, item #1) as they are related
> >>>>>>> to the Unmanaged AM mode. For example, we must make sure close
> >>>>>>> all threads before exiting REEF Driver - otherwise,
> >>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
> >>>>>>> currently
> >> have a System.exit() call at the end.
> >>>>>>>
> >>>>>>> Thanks for help!
> >>>>>>> Sergiy.
> >>>>>>>
> >>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
> >>>>>>>> sergiy.matusevych@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Taegeon,
> >>>>>>>>>
> >>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
> >>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
> >>>>>>>>> class), and it works fine on YARN 2.7.3. We also have several
> >>>>>>>>> examples and unit tests that check
> >>>>>>>> the
> >>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
> >>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
> >>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think we
> >>>>>>>>> should unit test? (I am saying that our unit tests are
> >>>>>>>>> comprehensive (they are not!), but I would love to know
> >>>>>>>> what
> >>>>>>>>> area you think we should focus on for 0.16 release)
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Sergiy.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
> >>>>>>>>> <ta...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi all,
> >>>>>>>>>>
> >>>>>>>>>> It's been about 10 months since we've released the latest
> >>>>>>>>>> version
> >>>>>>>>>> (0.15
> >>>>>>>>> version).
> >>>>>>>>>> In order not to delay the release any longer, I want to call
> >>>>>>>>>> a release
> >>>>>>>>> vote as soon as possible.
> >>>>>>>>>>
> >>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
> >>>>>>>>>> next
> >>>>>>>> Thursday
> >>>>>>>>> (23th)?
> >>>>>>>>>> I know there still remain several blocking issues:
> >>>>>>>>>> 1) Java side CI failures
> >>>>>>>>>> 2) .NET side CI failures
> >>>>>>>>>> 3) Unmanaged AM test
> >>>>>>>>>>
> >>>>>>>>>> I want to know if it is possible that they can be resolved
> >>>>>>>>>> until next
> >>>>>>>>> Thursday.
> >>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
> >>>>>>>>>> the due date
> >>>>>>>>> is ok to me.
> >>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
> >>>>>>>>>> Sergiy is
> >>>>>>>>> working on 3).
> >>>>>>>>>> If the plan seems not ok, could you please share the ETA of
> them?
> >>>>>>>>>>
> >>>>>>>>>> Thanks,
> >>>>>>>>>> Taegeon
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Byung-Gon Chun
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Byung-Gon Chun
> >>>>
> >>>
> >>
> >>
> >
> >
> > --
> > Byung-Gon Chun
>
>


-- 
Byung-Gon Chun

RE: 0.16 release plan

Posted by Taegeon Um <ta...@gmail.com>.

Thanks Markus and Mariia for the comments!

I will document the transient failures in the release notes and update
REEF-1462 :)

Thanks,
Taegeon

2017. 7. 22. 오전 3:22에 "Mariia Mykhailova" <ma...@microsoft.com.invalid>님이
작성:

If we're releasing with the transient failures, the release manager needs
to go through the recent (~1 month) AppVeyor builds and update our list of
transient failures in REEF-1462 is. It has been a while since I updated
that list, and if we're making this part of release documentation it needs
to be up-to-date.

-Mariia

-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Friday, July 21, 2017 10:03 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: 0.16 release plan

Looks good overall. I'd like to add a goal for the release: Document the
transient failures in the release notes. That way, they are not a surprise
to future users. --Markus

RE: 0.16 release plan

Posted by Mariia Mykhailova <ma...@microsoft.com.INVALID>.

If we're releasing with the transient failures, the release manager needs to go through the recent (~1 month) AppVeyor builds and update our list of transient failures in REEF-1462 is. It has been a while since I updated that list, and if we're making this part of release documentation it needs to be up-to-date.

-Mariia

-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de] 
Sent: Friday, July 21, 2017 10:03 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: 0.16 release plan

Looks good overall. I'd like to add a goal for the release: Document the transient failures in the release notes. That way, they are not a surprise to future users. --Markus

Re: 0.16 release plan

Posted by Markus Weimer <ma...@weimo.de>.

Looks good overall. I'd like to add a goal for the release: Document
the transient failures in the release notes. That way, they are not a
surprise to future users. --Markus

Re: 0.16 release plan

Posted by Byung-Gon Chun <bg...@gmail.com>.

Sounds good!

- Gon

Sent from my iPhone

2017. 7. 21. 오후 1:34 Tae-Geon Um <ta...@gmail.com> 작성:

> Hi all, 
> 
> Again, I think it’s time to release 0.16; I would like to share the release plan :)
> 
> For the release 0.16, it looks like we have a consensus on the following items: 
> 1) releasing 0.16 with known transient failures (.NET side) 
> 2) upgrading to hadoop 2.7.0 for the new release
> 
> If you have any objection to these things, please share your opinion. 
> 
> Here is the release plan. 
> 1) I will wait *one week* for resolving the remaining transient failures
> 2) I will upgrade to hadoop 2.7.0 during the week
> 3) Although transient failures still remain after the deadline, I will call a release vote with the remaining transient issues. 
> 
> What do you think about this plan? 
> 
> Thanks,
> Taegeon
> 
>> On May 11, 2017, at 12:56 PM, Sergiy Matusevych <se...@gmail.com> wrote:
>> 
>> Hi Taegeon,
>> 
>> I am afraid I won't be able to look at either of these issues this week. I
>> am very busy working on my slides on Distributed Factorization Machines,
>> and I have a presentation on Friday.
>> 
>> We can release 0.16-preview (or beta, or rc1, or whatever you call it) - I
>> think we are good in terms of features, and there is a finite number of
>> bugs that should not stop early adopters from using 0.16-preview
>> 
>> What do you guys think?
>> 
>> Cheers,
>> Sergiy.
>> 
>> 
>> 
>> 
>>> On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <ta...@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> Thanks Sergiy for taking a look at them!
>>> As far as I know, ApacheCon opens next week (May 16-18), so I think we
>>> need to resolve the issues until the end of this week.
>>> 
>>> I think I can help you in investigating REEF-1770. However, I’m not sure I
>>> can fix it until the end of this week.
>>> REEF-1770 is a transient failure, but there is no transient failure during
>>> the past 30 days in our Java-side Travis CI (0 transient failure out of 69
>>> builds).
>>> So, maybe it could be hard to reproduce it.
>>> 
>>> Sergiy, do you think you can resolve REEF-1796 until the end of this week?
>>> If not, we have two options.
>>> 
>>> 1) release 0.16 during the week of ApacheCon without fixing them
>>> (Actually, the .NET side CI is unstable, but just release 0.16)
>>> 2) do not release 0.16 until the bugs (in addition to the .NET side CI
>>> failures) are resolved
>>> 
>>> What do you guys think?
>>> 
>>> Thanks,
>>> Taegeon
>>> 
>>>> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
>>> sergiy.matusevych@gmail.com> wrote:
>>>> 
>>>> Hi guys,
>>>> 
>>>> It surely would be great to announce 0.16 at the conference, and we have
>>>> some awesome features to brag about - most notably, REEF-on-Spark. Still,
>>>> there are a few bugs that we need to fix before the release. I am mostly
>>>> concerned with https://issues.apache.org/jira/browse/REEF-1770 and
>>>> especially the https://issues.apache.org/jira/browse/REEF-1796 I am
>>> looking
>>>> at them now, but any help would be greatly appreciated!
>>>> 
>>>> Cheers,
>>>> Sergiy.
>>>> 
>>>> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com>
>>> wrote:
>>>> 
>>>>> Any update?
>>>>> It'd be great if we can release 0.16 during the week of ApacheCon.
>>>>> 
>>>>> -Gon
>>>>> 
>>>>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com>
>>> wrote:
>>>>> 
>>>>>> Unfortunately, we’ve also got a recent build failure in Java side [1],
>>>>>> which is not reported previously.
>>>>>> I’ve created an issue [2] to track this failure, and am going to
>>>>>> investigate it.
>>>>>> 
>>>>>> Thanks,
>>>>>> Taegeon
>>>>>> 
>>>>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
>>>>>> https://travis-ci.org/apache/reef/builds/220731026>
>>>>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
>>>>>> https://issues.apache.org/jira/browse/REEF-1770>
>>>>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <mamykhai@microsoft.com
>>> .
>>>>> INVALID>
>>>>>> wrote:
>>>>>>> 
>>>>>>> At least 3 of the issues previously reported under REEF-1462 have
>>>>>> re-occurred in the past two days (I've reopened corresponding JIRAs and
>>>>>> attached links to failures). Unfortunately, with the transient failures
>>>>>> like these one good build is insufficient.
>>>>>>> 
>>>>>>> It is a known issue, since we're using free access to AppVeyor, our
>>>>>> builds are sequential and low-priority, so sometimes when a lot of pull
>>>>>> requests have to be built the build queue takes a while to drain.
>>>>>>> 
>>>>>>> -Mariia
>>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
>>>>>>> Sent: Wednesday, April 5, 2017 12:11 AM
>>>>>>> To: dev@reef.apache.org
>>>>>>> Subject: Re: 0.16 release plan
>>>>>>> 
>>>>>>> Awesome!
>>>>>>> 
>>>>>>> There is no build failure in the .Net side with the latest build [1].
>>>>>>> 
>>>>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis CI's
>>>>>> already done. We're still waiting for Appveyor. :(
>>>>>>> 
>>>>>>> [1]
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com%
>>>>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
>>> 2BhaBnkHjFnkwxLh6GiPubCBDb%
>>>>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
>>>>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
>>>>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
>>>>>> data=02%7C01%7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
>>>>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
>>> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
>>>>>> E9Q%3D&reserved=0
>>>>>>> 
>>>>>>> 
>>>>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Thanks Julia for the work!
>>>>>>>> 
>>>>>>>> It looks like Java and .NET builds are almost stable, except for the
>>>>>>>> recent build failure in .NET side [1].
>>>>>>>> As Julia said in REEF-1406 [2], we would need to wait for time if
>>> this
>>>>>>>> failure is reproduced or not.
>>>>>>>> 
>>>>>>>> I will wait for a week and call a release vote if there are no build
>>>>>>>> failures during that time.
>>>>>>>> Thanks!
>>>>>>>> 
>>>>>>>> Taegeon
>>>>>>>> 
>>>>>>>> [1]:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>> https%3A%2F%2Fci.ap
>>>>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>> 2Freef%2F&data=02%7C01
>>>>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>> fda1%7C72f98
>>>>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
>>>>> sdata=Y9iO%2B
>>>>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
>>>>>>>> build/1453-master
>>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>>>>> https%3A%2F%2Fci.a
>>>>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
>>>>> %7C90c
>>>>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>> db47%7C1%7
>>>>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
>>>>> M6avrry5nk
>>>>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
>>>>>>>> [2]:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>> https%3A%2F%2Fissue
>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>>> 7Cmamykhai%40mic
>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>> https%3A%2F%2Fissue
>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>>> 7Cmamykhai%40mic
>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
>>>>>>>> 
>>>>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
>>>>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
>>>>>>>>> 
>>>>>>>>> I have resolved all the .Net test issues for now. The fixes contain
>>>>>>>>> what
>>>>>>>> I have identifies so far based on the failures.
>>>>>>>>> 
>>>>>>>>> I agree with Marria, as they are transit failures, also they failed
>>>>>>>>> for
>>>>>>>> multiple reasons sometimes, we need to continue to observe if the
>>>>>>>> issues come back again.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Julia
>>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
>>>>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>> 
>>>>>>>>> Thanks Mariia for pointing it out to me.
>>>>>>>>> Yes. I agree that we need more time to fix all of the transient
>>>>>> failures.
>>>>>>>>> After they are resolved, I will wait for some time to ensure that
>>>>>>>>> they
>>>>>>>> are not reoccurred.
>>>>>>>>> 
>>>>>>>>> Thanks!
>>>>>>>>> Taegeon
>>>>>>>>> 
>>>>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
>>>>>>>>>> <ma...@microsoft.com.INVALID>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Please note that due to the transient nature of .NET failures, it
>>>>>>>>>> makes
>>>>>>>> sense to wait for some time and to observe whether they are actually
>>>>>>>> fixed or just lying low until the next reoccurrence. We had to reopen
>>>>>>>> some bugs which looked resolved in the past but then reoccurred.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> -Mariia
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> ________________________________
>>>>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
>>>>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>> 
>>>>>>>>>> Hi,
>>>>>>>>>> 
>>>>>>>>>> Julia has been doing a great work to resolve the .NET side issues.
>>>>>>>>>> It looks like she has resolved 3 issues recently (and now 3 issues
>>>>>>>> remain in .NET side with 1 pending PR).
>>>>>>>>>> 
>>>>>>>>>> Sergiy and I also have worked for the java side issues, and we've
>>>>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
>>>>>>>>>> 
>>>>>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
>>>>>>>>>> think
>>>>>>>> it would be good to delay the release vote.
>>>>>>>>>> However, judging from the progress we made, I think all of the
>>>>>>>>>> issues
>>>>>>>> could be resolved until at the end of this week or begging of next
>>>>> week.
>>>>>>>>>> 
>>>>>>>>>> I will call a vote as soon as possible after they are resolved.
>>>>>>>>>> Thanks!
>>>>>>>>>> 
>>>>>>>>>> Taegeon
>>>>>>>>>> 
>>>>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
>>>>>>>>>>> 
>>>>>>>>>>> Taegeon, could you give us status update? Thanks.
>>>>>>>>>>> 
>>>>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
>>>>>>>>>>> <bg...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
>>>>>>>>>>>> 
>>>>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
>>>>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
>>>>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
>>>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Oh, never mind.
>>>>>>>>>>>>>> I thought that we still need some time to make sure that
>>>>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Great, then we have only two items left! :-) I can confirm that
>>>>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need to
>>>>>>>>>>>>> address the Java issues (that is, item #1) as they are related
>>>>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
>>>>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
>>>>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
>>>>>>>>>>>>> currently
>>>>>>>> have a System.exit() call at the end.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for help!
>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
>>>>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi Taegeon,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
>>>>>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
>>>>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have several
>>>>>>>>>>>>>>> examples and unit tests that check
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
>>>>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
>>>>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think we
>>>>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
>>>>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>> area you think we should focus on for 0.16 release)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
>>>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> It's been about 10 months since we've released the latest
>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> (0.15
>>>>>>>>>>>>>>> version).
>>>>>>>>>>>>>>>> In order not to delay the release any longer, I want to call
>>>>>>>>>>>>>>>> a release
>>>>>>>>>>>>>>> vote as soon as possible.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
>>>>>>>>>>>>>>>> next
>>>>>>>>>>>>>> Thursday
>>>>>>>>>>>>>>> (23th)?
>>>>>>>>>>>>>>>> I know there still remain several blocking issues:
>>>>>>>>>>>>>>>> 1) Java side CI failures
>>>>>>>>>>>>>>>> 2) .NET side CI failures
>>>>>>>>>>>>>>>> 3) Unmanaged AM test
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I want to know if it is possible that they can be resolved
>>>>>>>>>>>>>>>> until next
>>>>>>>>>>>>>>> Thursday.
>>>>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
>>>>>>>>>>>>>>>> the due date
>>>>>>>>>>>>>>> is ok to me.
>>>>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
>>>>>>>>>>>>>>>> Sergiy is
>>>>>>>>>>>>>>> working on 3).
>>>>>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
>>>>>> them?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Taegeon
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Byung-Gon Chun
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Byung-Gon Chun
>>>>> 
>>> 
>>> 
>

Re: 0.16 release plan

Posted by Markus Weimer <ma...@weimo.de>.

> I am inclined to release 0.16 now - I will run the tests and cast my vote in a few hours.

+1

If we enter the realm of in-flight features blocking releases, we will
never release. We have 10+ PRs in flight at all times, after all :)

Markus

Re: 0.16 release plan

Posted by Sergiy Matusevych <se...@gmail.com>.

Hi guys,

I am all for getting 0.16 out of the door ASAP :) REEF looks pretty stable
now (save the functional tests that we all know of) and builds with VS2017
just fine. My only concern is that the Avro bridge functionality is halfway
there: C# serialization part of it is already in master, Java serialization
is pending another round of PR reviews (but I expect I'll be able to merge
it one of these days), and the bridge itself is on another branch and
is not even a PR yet.

So we can either wait for all this functionality to appear in 0.16, or
release 0.16 as is, with C# serialization not used anywhere, and do another
minor release once the entire Avro bridge is working and we can run REEF.NET
from Spark.

I am inclined to release 0.16 now - I will run the tests and cast my vote
in a few hours.

Cheers,
Sergiy.

On Tue, Aug 1, 2017 at 7:40 PM, Taegeon Um <ta...@gmail.com> wrote:

> Hi all,
>
> Sorry for the spam!
> I’ve updated JIRA issues in which the fix version was not set to 0.16.
>
> I will call a 0.16 release vote soon :)
>
> Thanks,
> Taegeon
>
>
> > On Jul 29, 2017, at 12:27 AM, Tae-Geon Um <ta...@gmail.com> wrote:
> >
> > Sure. Not a problem.
> >
> > I’ve went through the recent AppVeyor builds and updated REEF-1462,
> > and also created a PR for upgrading to Hadoop 2.7.0. Please someone
> takes a look at PR#1347 (https://github.com/apache/reef/pull/1347 <
> https://github.com/apache/reef/pull/1347>) and merge it.
> >
> > If PR#1347 is merged, I will going to call a release vote.
> >
> > Thanks,
> > Taegeon
> >
> >> On Jul 28, 2017, at 8:25 PM, Byung-Gon Chun <bgchun@gmail.com <mailto:
> bgchun@gmail.com>> wrote:
> >>
> >> Hi Taegeon,
> >>
> >> Can you share the current release progress with the community?
> >>
> >> Thanks.
> >> -Gon
> >>
> >>
> >> On Fri, Jul 21, 2017 at 1:34 PM, Tae-Geon Um <taegeonum@gmail.com
> <ma...@gmail.com>> wrote:
> >>
> >>> Hi all,
> >>>
> >>> Again, I think it’s time to release 0.16; I would like to share the
> >>> release plan :)
> >>>
> >>> For the release 0.16, it looks like we have a consensus on the
> following
> >>> items:
> >>> 1) releasing 0.16 with known transient failures (.NET side)
> >>> 2) upgrading to hadoop 2.7.0 for the new release
> >>>
> >>> If you have any objection to these things, please share your opinion.
> >>>
> >>> Here is the release plan.
> >>> 1) I will wait *one week* for resolving the remaining transient
> failures
> >>> 2) I will upgrade to hadoop 2.7.0 during the week
> >>> 3) Although transient failures still remain after the deadline, I will
> >>> call a release vote with the remaining transient issues.
> >>>
> >>> What do you think about this plan?
> >>>
> >>> Thanks,
> >>> Taegeon
> >>>
> >>>> On May 11, 2017, at 12:56 PM, Sergiy Matusevych <
> >>> sergiy.matusevych@gmail.com <ma...@gmail.com>>
> wrote:
> >>>>
> >>>> Hi Taegeon,
> >>>>
> >>>> I am afraid I won't be able to look at either of these issues this
> week.
> >>> I
> >>>> am very busy working on my slides on Distributed Factorization
> Machines,
> >>>> and I have a presentation on Friday.
> >>>>
> >>>> We can release 0.16-preview (or beta, or rc1, or whatever you call
> it) -
> >>> I
> >>>> think we are good in terms of features, and there is a finite number
> of
> >>>> bugs that should not stop early adopters from using 0.16-preview
> >>>>
> >>>> What do you guys think?
> >>>>
> >>>> Cheers,
> >>>> Sergiy.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <taegeonum@gmail.com
> <ma...@gmail.com>>
> >>> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> Thanks Sergiy for taking a look at them!
> >>>>> As far as I know, ApacheCon opens next week (May 16-18), so I think
> we
> >>>>> need to resolve the issues until the end of this week.
> >>>>>
> >>>>> I think I can help you in investigating REEF-1770. However, I’m not
> >>> sure I
> >>>>> can fix it until the end of this week.
> >>>>> REEF-1770 is a transient failure, but there is no transient failure
> >>> during
> >>>>> the past 30 days in our Java-side Travis CI (0 transient failure out
> of
> >>> 69
> >>>>> builds).
> >>>>> So, maybe it could be hard to reproduce it.
> >>>>>
> >>>>> Sergiy, do you think you can resolve REEF-1796 until the end of this
> >>> week?
> >>>>> If not, we have two options.
> >>>>>
> >>>>> 1) release 0.16 during the week of ApacheCon without fixing them
> >>>>> (Actually, the .NET side CI is unstable, but just release 0.16)
> >>>>> 2) do not release 0.16 until the bugs (in addition to the .NET side
> CI
> >>>>> failures) are resolved
> >>>>>
> >>>>> What do you guys think?
> >>>>>
> >>>>> Thanks,
> >>>>> Taegeon
> >>>>>
> >>>>>> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
> >>>>> sergiy.matusevych@gmail.com <ma...@gmail.com>>
> wrote:
> >>>>>>
> >>>>>> Hi guys,
> >>>>>>
> >>>>>> It surely would be great to announce 0.16 at the conference, and we
> >>> have
> >>>>>> some awesome features to brag about - most notably, REEF-on-Spark.
> >>> Still,
> >>>>>> there are a few bugs that we need to fix before the release. I am
> >>> mostly
> >>>>>> concerned with https://issues.apache.org/jira/browse/REEF-1770 <
> https://issues.apache.org/jira/browse/REEF-1770> and
> >>>>>> especially the https://issues.apache.org/jira/browse/REEF-1796 <
> https://issues.apache.org/jira/browse/REEF-1796> I am
> >>>>> looking
> >>>>>> at them now, but any help would be greatly appreciated!
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Sergiy.
> >>>>>>
> >>>>>> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bgchun@gmail.com
> <ma...@gmail.com>>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Any update?
> >>>>>>> It'd be great if we can release 0.16 during the week of ApacheCon.
> >>>>>>>
> >>>>>>> -Gon
> >>>>>>>
> >>>>>>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <taegeonum@gmail.com
> <ma...@gmail.com>>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Unfortunately, we’ve also got a recent build failure in Java side
> >>> [1],
> >>>>>>>> which is not reported previously.
> >>>>>>>> I’ve created an issue [2] to track this failure, and am going to
> >>>>>>>> investigate it.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Taegeon
> >>>>>>>>
> >>>>>>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
> https://travis-ci.org/apache/reef/builds/220731026> <
> >>>>>>>> https://travis-ci.org/apache/reef/builds/220731026 <
> https://travis-ci.org/apache/reef/builds/220731026>>
> >>>>>>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
> https://issues.apache.org/jira/browse/REEF-1770> <
> >>>>>>>> https://issues.apache.org/jira/browse/REEF-1770 <
> https://issues.apache.org/jira/browse/REEF-1770>>
> >>>>>>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <
> >>> mamykhai@microsoft.com <ma...@microsoft.com>
> >>>>> .
> >>>>>>> INVALID>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> At least 3 of the issues previously reported under REEF-1462 have
> >>>>>>>> re-occurred in the past two days (I've reopened corresponding
> JIRAs
> >>> and
> >>>>>>>> attached links to failures). Unfortunately, with the transient
> >>> failures
> >>>>>>>> like these one good build is insufficient.
> >>>>>>>>>
> >>>>>>>>> It is a known issue, since we're using free access to AppVeyor,
> our
> >>>>>>>> builds are sequential and low-priority, so sometimes when a lot of
> >>> pull
> >>>>>>>> requests have to be built the build queue takes a while to drain.
> >>>>>>>>>
> >>>>>>>>> -Mariia
> >>>>>>>>>
> >>>>>>>>> -----Original Message-----
> >>>>>>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
> >>>>>>>>> Sent: Wednesday, April 5, 2017 12:11 AM
> >>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>
> >>>>>>>>> Awesome!
> >>>>>>>>>
> >>>>>>>>> There is no build failure in the .Net side with the latest build
> >>> [1].
> >>>>>>>>>
> >>>>>>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis
> >>> CI's
> >>>>>>>> already done. We're still waiting for Appveyor. :(
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>>>>>> https%3A%2F%2Fci.appveyor.com%2Fproject%
> 2FApacheSoftwareFoundation%
> >>>>>>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40micr
> osoft.com
> >>> %
> >>>>>>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> >>>>>>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
> >>>>> 2BhaBnkHjFnkwxLh6GiPubCBDb%
> >>>>>>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
> >>>>>>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
> >>>>>>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
> >>>>>>>> data=02%7C01%7Cmamykhai%40microsoft.com%
> >>> 7C90c5159366d44f7efdd308d47bf2
> >>>>>>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
> >>>>>>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
> >>>>> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
> >>>>>>>> E9Q%3D&reserved=0
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <
> taegeonum@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks Julia for the work!
> >>>>>>>>>>
> >>>>>>>>>> It looks like Java and .NET builds are almost stable, except for
> >>> the
> >>>>>>>>>> recent build failure in .NET side [1].
> >>>>>>>>>> As Julia said in REEF-1406 [2], we would need to wait for time
> if
> >>>>> this
> >>>>>>>>>> failure is reproduced or not.
> >>>>>>>>>>
> >>>>>>>>>> I will wait for a week and call a release vote if there are no
> >>> build
> >>>>>>>>>> failures during that time.
> >>>>>>>>>> Thanks!
> >>>>>>>>>>
> >>>>>>>>>> Taegeon
> >>>>>>>>>>
> >>>>>>>>>> [1]:
> >>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>>>>> https%3A%2F%2Fci.ap
> >>>>>>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
> >>>>>>> 2Freef%2F&data=02%7C01
> >>>>>>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> >>>>>>> fda1%7C72f98
> >>>>>>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
> >>>>>>> sdata=Y9iO%2B
> >>>>>>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
> >>>>>>>>>> build/1453-master
> >>>>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
> >>>>>>> https%3A%2F%2Fci.a
> >>>>>>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft
> .com
> >>>>>>> %7C90c
> >>>>>>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> >>>>>>> db47%7C1%7
> >>>>>>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
> >>>>>>> M6avrry5nk
> >>>>>>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/
> build/1453-master>
> >>>>>>>>>> [2]:
> >>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>>>>> https%3A%2F%2Fissue
> >>>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> >>>>>>> 7Cmamykhai%40mic
> >>>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> >>>>>>> fda1%7C72f988bf86f141af91ab2d
> >>>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> >>>>>>> 2F9yenZoGqe%2FkowHza7
> >>>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
> >>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>>>>> https%3A%2F%2Fissue
> >>>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> >>>>>>> 7Cmamykhai%40mic
> >>>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> >>>>>>> fda1%7C72f988bf86f141af91ab2d
> >>>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> >>>>>>> 2F9yenZoGqe%2FkowHza7
> >>>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
> >>>>>>>>>>
> >>>>>>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
> >>>>>>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> I have resolved all the .Net test issues for now. The fixes
> >>> contain
> >>>>>>>>>>> what
> >>>>>>>>>> I have identifies so far based on the failures.
> >>>>>>>>>>>
> >>>>>>>>>>> I agree with Marria, as they are transit failures, also they
> >>> failed
> >>>>>>>>>>> for
> >>>>>>>>>> multiple reasons sometimes, we need to continue to observe if
> the
> >>>>>>>>>> issues come back again.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Julia
> >>>>>>>>>>>
> >>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
> >>>>>>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
> >>>>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks Mariia for pointing it out to me.
> >>>>>>>>>>> Yes. I agree that we need more time to fix all of the transient
> >>>>>>>> failures.
> >>>>>>>>>>> After they are resolved, I will wait for some time to ensure
> that
> >>>>>>>>>>> they
> >>>>>>>>>> are not reoccurred.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>> Taegeon
> >>>>>>>>>>>
> >>>>>>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
> >>>>>>>>>>>> <ma...@microsoft.com.INVALID>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please note that due to the transient nature of .NET
> failures, it
> >>>>>>>>>>>> makes
> >>>>>>>>>> sense to wait for some time and to observe whether they are
> >>> actually
> >>>>>>>>>> fixed or just lying low until the next reoccurrence. We had to
> >>> reopen
> >>>>>>>>>> some bugs which looked resolved in the past but then reoccurred.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> -Mariia
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> ________________________________
> >>>>>>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
> >>>>>>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
> >>>>>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Julia has been doing a great work to resolve the .NET side
> >>> issues.
> >>>>>>>>>>>> It looks like she has resolved 3 issues recently (and now 3
> >>> issues
> >>>>>>>>>> remain in .NET side with 1 pending PR).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sergiy and I also have worked for the java side issues, and
> we've
> >>>>>>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
> >>>>>>>>>>>>
> >>>>>>>>>>>> Because of the unresolved issues (3 .NET side and 1 java
> side), I
> >>>>>>>>>>>> think
> >>>>>>>>>> it would be good to delay the release vote.
> >>>>>>>>>>>> However, judging from the progress we made, I think all of the
> >>>>>>>>>>>> issues
> >>>>>>>>>> could be resolved until at the end of this week or begging of
> next
> >>>>>>> week.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I will call a vote as soon as possible after they are
> resolved.
> >>>>>>>>>>>> Thanks!
> >>>>>>>>>>>>
> >>>>>>>>>>>> Taegeon
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <
> bgchun@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Taegeon, could you give us status update? Thanks.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
> >>>>>>>>>>>>> <bg...@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
> >>>>>>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I am working on some of the .Net AppVeyor test failures
> now.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@
> gmail.com]
> >>>>>>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
> >>>>>>>>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
> >>>>>>>>>>>>>>> <ta...@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Oh, never mind.
> >>>>>>>>>>>>>>>> I thought that we still need some time to make sure that
> >>>>>>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Great, then we have only two items left! :-) I can confirm
> >>> that
> >>>>>>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we
> need
> >>> to
> >>>>>>>>>>>>>>> address the Java issues (that is, item #1) as they are
> related
> >>>>>>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure
> close
> >>>>>>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
> >>>>>>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
> >>>>>>>>>>>>>>> currently
> >>>>>>>>>> have a System.exit() call at the end.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks for help!
> >>>>>>>>>>>>>>> Sergiy.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
> >>>>>>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi Taegeon,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF
> example
> >>>>>>>>>>>>>>>>> running in Unmanaged AM mode (see
> HelloREEFYarnUnmanagedAM
> >>>>>>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have
> >>> several
> >>>>>>>>>>>>>>>>> examples and unit tests that check
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
> >>>>>>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
> >>>>>>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you
> think
> >>> we
> >>>>>>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
> >>>>>>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
> >>>>>>>>>>>>>>>> what
> >>>>>>>>>>>>>>>>> area you think we should focus on for 0.16 release)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>>> Sergiy.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
> >>>>>>>>>>>>>>>>> <ta...@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> It's been about 10 months since we've released the
> latest
> >>>>>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>>>> (0.15
> >>>>>>>>>>>>>>>>> version).
> >>>>>>>>>>>>>>>>>> In order not to delay the release any longer, I want to
> >>> call
> >>>>>>>>>>>>>>>>>> a release
> >>>>>>>>>>>>>>>>> vote as soon as possible.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release
> vote on
> >>>>>>>>>>>>>>>>>> next
> >>>>>>>>>>>>>>>> Thursday
> >>>>>>>>>>>>>>>>> (23th)?
> >>>>>>>>>>>>>>>>>> I know there still remain several blocking issues:
> >>>>>>>>>>>>>>>>>> 1) Java side CI failures
> >>>>>>>>>>>>>>>>>> 2) .NET side CI failures
> >>>>>>>>>>>>>>>>>> 3) Unmanaged AM test
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I want to know if it is possible that they can be
> resolved
> >>>>>>>>>>>>>>>>>> until next
> >>>>>>>>>>>>>>>>> Thursday.
> >>>>>>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help),
> and
> >>>>>>>>>>>>>>>>>> the due date
> >>>>>>>>>>>>>>>>> is ok to me.
> >>>>>>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia
> and
> >>>>>>>>>>>>>>>>>> Sergiy is
> >>>>>>>>>>>>>>>>> working on 3).
> >>>>>>>>>>>>>>>>>> If the plan seems not ok, could you please share the
> ETA of
> >>>>>>>> them?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>>>> Taegeon
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Byung-Gon Chun
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Byung-Gon Chun
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Byung-Gon Chun
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Byung-Gon Chun
> >>>>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Byung-Gon Chun
> >
>
>

Re: 0.16 release plan

Posted by Taegeon Um <ta...@gmail.com>.

Hi all, 

Sorry for the spam! 
I’ve updated JIRA issues in which the fix version was not set to 0.16.  

I will call a 0.16 release vote soon :)

Thanks,
Taegeon 


> On Jul 29, 2017, at 12:27 AM, Tae-Geon Um <ta...@gmail.com> wrote:
> 
> Sure. Not a problem.
> 
> I’ve went through the recent AppVeyor builds and updated REEF-1462,
> and also created a PR for upgrading to Hadoop 2.7.0. Please someone takes a look at PR#1347 (https://github.com/apache/reef/pull/1347 <https://github.com/apache/reef/pull/1347>) and merge it. 
> 
> If PR#1347 is merged, I will going to call a release vote.
> 
> Thanks,
> Taegeon 
> 
>> On Jul 28, 2017, at 8:25 PM, Byung-Gon Chun <bgchun@gmail.com <ma...@gmail.com>> wrote:
>> 
>> Hi Taegeon,
>> 
>> Can you share the current release progress with the community?
>> 
>> Thanks.
>> -Gon
>> 
>> 
>> On Fri, Jul 21, 2017 at 1:34 PM, Tae-Geon Um <taegeonum@gmail.com <ma...@gmail.com>> wrote:
>> 
>>> Hi all,
>>> 
>>> Again, I think it’s time to release 0.16; I would like to share the
>>> release plan :)
>>> 
>>> For the release 0.16, it looks like we have a consensus on the following
>>> items:
>>> 1) releasing 0.16 with known transient failures (.NET side)
>>> 2) upgrading to hadoop 2.7.0 for the new release
>>> 
>>> If you have any objection to these things, please share your opinion.
>>> 
>>> Here is the release plan.
>>> 1) I will wait *one week* for resolving the remaining transient failures
>>> 2) I will upgrade to hadoop 2.7.0 during the week
>>> 3) Although transient failures still remain after the deadline, I will
>>> call a release vote with the remaining transient issues.
>>> 
>>> What do you think about this plan?
>>> 
>>> Thanks,
>>> Taegeon
>>> 
>>>> On May 11, 2017, at 12:56 PM, Sergiy Matusevych <
>>> sergiy.matusevych@gmail.com <ma...@gmail.com>> wrote:
>>>> 
>>>> Hi Taegeon,
>>>> 
>>>> I am afraid I won't be able to look at either of these issues this week.
>>> I
>>>> am very busy working on my slides on Distributed Factorization Machines,
>>>> and I have a presentation on Friday.
>>>> 
>>>> We can release 0.16-preview (or beta, or rc1, or whatever you call it) -
>>> I
>>>> think we are good in terms of features, and there is a finite number of
>>>> bugs that should not stop early adopters from using 0.16-preview
>>>> 
>>>> What do you guys think?
>>>> 
>>>> Cheers,
>>>> Sergiy.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <taegeonum@gmail.com <ma...@gmail.com>>
>>> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> Thanks Sergiy for taking a look at them!
>>>>> As far as I know, ApacheCon opens next week (May 16-18), so I think we
>>>>> need to resolve the issues until the end of this week.
>>>>> 
>>>>> I think I can help you in investigating REEF-1770. However, I’m not
>>> sure I
>>>>> can fix it until the end of this week.
>>>>> REEF-1770 is a transient failure, but there is no transient failure
>>> during
>>>>> the past 30 days in our Java-side Travis CI (0 transient failure out of
>>> 69
>>>>> builds).
>>>>> So, maybe it could be hard to reproduce it.
>>>>> 
>>>>> Sergiy, do you think you can resolve REEF-1796 until the end of this
>>> week?
>>>>> If not, we have two options.
>>>>> 
>>>>> 1) release 0.16 during the week of ApacheCon without fixing them
>>>>> (Actually, the .NET side CI is unstable, but just release 0.16)
>>>>> 2) do not release 0.16 until the bugs (in addition to the .NET side CI
>>>>> failures) are resolved
>>>>> 
>>>>> What do you guys think?
>>>>> 
>>>>> Thanks,
>>>>> Taegeon
>>>>> 
>>>>>> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
>>>>> sergiy.matusevych@gmail.com <ma...@gmail.com>> wrote:
>>>>>> 
>>>>>> Hi guys,
>>>>>> 
>>>>>> It surely would be great to announce 0.16 at the conference, and we
>>> have
>>>>>> some awesome features to brag about - most notably, REEF-on-Spark.
>>> Still,
>>>>>> there are a few bugs that we need to fix before the release. I am
>>> mostly
>>>>>> concerned with https://issues.apache.org/jira/browse/REEF-1770 <https://issues.apache.org/jira/browse/REEF-1770> and
>>>>>> especially the https://issues.apache.org/jira/browse/REEF-1796 <https://issues.apache.org/jira/browse/REEF-1796> I am
>>>>> looking
>>>>>> at them now, but any help would be greatly appreciated!
>>>>>> 
>>>>>> Cheers,
>>>>>> Sergiy.
>>>>>> 
>>>>>> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bgchun@gmail.com <ma...@gmail.com>>
>>>>> wrote:
>>>>>> 
>>>>>>> Any update?
>>>>>>> It'd be great if we can release 0.16 during the week of ApacheCon.
>>>>>>> 
>>>>>>> -Gon
>>>>>>> 
>>>>>>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <taegeonum@gmail.com <ma...@gmail.com>>
>>>>> wrote:
>>>>>>> 
>>>>>>>> Unfortunately, we’ve also got a recent build failure in Java side
>>> [1],
>>>>>>>> which is not reported previously.
>>>>>>>> I’ve created an issue [2] to track this failure, and am going to
>>>>>>>> investigate it.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Taegeon
>>>>>>>> 
>>>>>>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <https://travis-ci.org/apache/reef/builds/220731026> <
>>>>>>>> https://travis-ci.org/apache/reef/builds/220731026 <https://travis-ci.org/apache/reef/builds/220731026>>
>>>>>>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <https://issues.apache.org/jira/browse/REEF-1770> <
>>>>>>>> https://issues.apache.org/jira/browse/REEF-1770 <https://issues.apache.org/jira/browse/REEF-1770>>
>>>>>>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <
>>> mamykhai@microsoft.com <ma...@microsoft.com>
>>>>> .
>>>>>>> INVALID>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> At least 3 of the issues previously reported under REEF-1462 have
>>>>>>>> re-occurred in the past two days (I've reopened corresponding JIRAs
>>> and
>>>>>>>> attached links to failures). Unfortunately, with the transient
>>> failures
>>>>>>>> like these one good build is insufficient.
>>>>>>>>> 
>>>>>>>>> It is a known issue, since we're using free access to AppVeyor, our
>>>>>>>> builds are sequential and low-priority, so sometimes when a lot of
>>> pull
>>>>>>>> requests have to be built the build queue takes a while to drain.
>>>>>>>>> 
>>>>>>>>> -Mariia
>>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
>>>>>>>>> Sent: Wednesday, April 5, 2017 12:11 AM
>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>> 
>>>>>>>>> Awesome!
>>>>>>>>> 
>>>>>>>>> There is no build failure in the .Net side with the latest build
>>> [1].
>>>>>>>>> 
>>>>>>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis
>>> CI's
>>>>>>>> already done. We're still waiting for Appveyor. :(
>>>>>>>>> 
>>>>>>>>> [1]
>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>>>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>>>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com
>>> %
>>>>>>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>>>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
>>>>> 2BhaBnkHjFnkwxLh6GiPubCBDb%
>>>>>>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
>>>>>>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
>>>>>>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
>>>>>>>> data=02%7C01%7Cmamykhai%40microsoft.com%
>>> 7C90c5159366d44f7efdd308d47bf2
>>>>>>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
>>>>>>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
>>>>> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
>>>>>>>> E9Q%3D&reserved=0
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Thanks Julia for the work!
>>>>>>>>>> 
>>>>>>>>>> It looks like Java and .NET builds are almost stable, except for
>>> the
>>>>>>>>>> recent build failure in .NET side [1].
>>>>>>>>>> As Julia said in REEF-1406 [2], we would need to wait for time if
>>>>> this
>>>>>>>>>> failure is reproduced or not.
>>>>>>>>>> 
>>>>>>>>>> I will wait for a week and call a release vote if there are no
>>> build
>>>>>>>>>> failures during that time.
>>>>>>>>>> Thanks!
>>>>>>>>>> 
>>>>>>>>>> Taegeon
>>>>>>>>>> 
>>>>>>>>>> [1]:
>>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>>> https%3A%2F%2Fci.ap
>>>>>>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>>>> 2Freef%2F&data=02%7C01
>>>>>>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>>> fda1%7C72f98
>>>>>>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
>>>>>>> sdata=Y9iO%2B
>>>>>>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
>>>>>>>>>> build/1453-master
>>>>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>>>>>>> https%3A%2F%2Fci.a
>>>>>>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
>>>>>>> %7C90c
>>>>>>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>>>> db47%7C1%7
>>>>>>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
>>>>>>> M6avrry5nk
>>>>>>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
>>>>>>>>>> [2]:
>>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>>> https%3A%2F%2Fissue
>>>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>>>>> 7Cmamykhai%40mic
>>>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
>>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>>> https%3A%2F%2Fissue
>>>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>>>>> 7Cmamykhai%40mic
>>>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
>>>>>>>>>> 
>>>>>>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
>>>>>>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> I have resolved all the .Net test issues for now. The fixes
>>> contain
>>>>>>>>>>> what
>>>>>>>>>> I have identifies so far based on the failures.
>>>>>>>>>>> 
>>>>>>>>>>> I agree with Marria, as they are transit failures, also they
>>> failed
>>>>>>>>>>> for
>>>>>>>>>> multiple reasons sometimes, we need to continue to observe if the
>>>>>>>>>> issues come back again.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Julia
>>>>>>>>>>> 
>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
>>>>>>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>> 
>>>>>>>>>>> Thanks Mariia for pointing it out to me.
>>>>>>>>>>> Yes. I agree that we need more time to fix all of the transient
>>>>>>>> failures.
>>>>>>>>>>> After they are resolved, I will wait for some time to ensure that
>>>>>>>>>>> they
>>>>>>>>>> are not reoccurred.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks!
>>>>>>>>>>> Taegeon
>>>>>>>>>>> 
>>>>>>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
>>>>>>>>>>>> <ma...@microsoft.com.INVALID>
>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Please note that due to the transient nature of .NET failures, it
>>>>>>>>>>>> makes
>>>>>>>>>> sense to wait for some time and to observe whether they are
>>> actually
>>>>>>>>>> fixed or just lying low until the next reoccurrence. We had to
>>> reopen
>>>>>>>>>> some bugs which looked resolved in the past but then reoccurred.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> -Mariia
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> ________________________________
>>>>>>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
>>>>>>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
>>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> 
>>>>>>>>>>>> Julia has been doing a great work to resolve the .NET side
>>> issues.
>>>>>>>>>>>> It looks like she has resolved 3 issues recently (and now 3
>>> issues
>>>>>>>>>> remain in .NET side with 1 pending PR).
>>>>>>>>>>>> 
>>>>>>>>>>>> Sergiy and I also have worked for the java side issues, and we've
>>>>>>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
>>>>>>>>>>>> 
>>>>>>>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
>>>>>>>>>>>> think
>>>>>>>>>> it would be good to delay the release vote.
>>>>>>>>>>>> However, judging from the progress we made, I think all of the
>>>>>>>>>>>> issues
>>>>>>>>>> could be resolved until at the end of this week or begging of next
>>>>>>> week.
>>>>>>>>>>>> 
>>>>>>>>>>>> I will call a vote as soon as possible after they are resolved.
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> 
>>>>>>>>>>>> Taegeon
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Taegeon, could you give us status update? Thanks.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
>>>>>>>>>>>>> <bg...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
>>>>>>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
>>>>>>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
>>>>>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
>>>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Oh, never mind.
>>>>>>>>>>>>>>>> I thought that we still need some time to make sure that
>>>>>>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Great, then we have only two items left! :-) I can confirm
>>> that
>>>>>>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need
>>> to
>>>>>>>>>>>>>>> address the Java issues (that is, item #1) as they are related
>>>>>>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
>>>>>>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
>>>>>>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
>>>>>>>>>>>>>>> currently
>>>>>>>>>> have a System.exit() call at the end.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks for help!
>>>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
>>>>>>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi Taegeon,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
>>>>>>>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
>>>>>>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have
>>> several
>>>>>>>>>>>>>>>>> examples and unit tests that check
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
>>>>>>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
>>>>>>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think
>>> we
>>>>>>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
>>>>>>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>> area you think we should focus on for 0.16 release)
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
>>>>>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> It's been about 10 months since we've released the latest
>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>> (0.15
>>>>>>>>>>>>>>>>> version).
>>>>>>>>>>>>>>>>>> In order not to delay the release any longer, I want to
>>> call
>>>>>>>>>>>>>>>>>> a release
>>>>>>>>>>>>>>>>> vote as soon as possible.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
>>>>>>>>>>>>>>>>>> next
>>>>>>>>>>>>>>>> Thursday
>>>>>>>>>>>>>>>>> (23th)?
>>>>>>>>>>>>>>>>>> I know there still remain several blocking issues:
>>>>>>>>>>>>>>>>>> 1) Java side CI failures
>>>>>>>>>>>>>>>>>> 2) .NET side CI failures
>>>>>>>>>>>>>>>>>> 3) Unmanaged AM test
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I want to know if it is possible that they can be resolved
>>>>>>>>>>>>>>>>>> until next
>>>>>>>>>>>>>>>>> Thursday.
>>>>>>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
>>>>>>>>>>>>>>>>>> the due date
>>>>>>>>>>>>>>>>> is ok to me.
>>>>>>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
>>>>>>>>>>>>>>>>>> Sergiy is
>>>>>>>>>>>>>>>>> working on 3).
>>>>>>>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
>>>>>>>> them?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>> Taegeon
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Byung-Gon Chun
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Byung-Gon Chun
>>>>>>> 
>>>>> 
>>>>> 
>>> 
>>> 
>> 
>> 
>> -- 
>> Byung-Gon Chun
>

Re: 0.16 release plan

Posted by Tae-Geon Um <ta...@gmail.com>.

Sure. Not a problem.

I’ve went through the recent AppVeyor builds and updated REEF-1462,
and also created a PR for upgrading to Hadoop 2.7.0. Please someone takes a look at PR#1347 (https://github.com/apache/reef/pull/1347 <https://github.com/apache/reef/pull/1347>) and merge it. 

If PR#1347 is merged, I will going to call a release vote.

Thanks,
Taegeon 

> On Jul 28, 2017, at 8:25 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
> 
> Hi Taegeon,
> 
> Can you share the current release progress with the community?
> 
> Thanks.
> -Gon
> 
> 
> On Fri, Jul 21, 2017 at 1:34 PM, Tae-Geon Um <ta...@gmail.com> wrote:
> 
>> Hi all,
>> 
>> Again, I think it’s time to release 0.16; I would like to share the
>> release plan :)
>> 
>> For the release 0.16, it looks like we have a consensus on the following
>> items:
>> 1) releasing 0.16 with known transient failures (.NET side)
>> 2) upgrading to hadoop 2.7.0 for the new release
>> 
>> If you have any objection to these things, please share your opinion.
>> 
>> Here is the release plan.
>> 1) I will wait *one week* for resolving the remaining transient failures
>> 2) I will upgrade to hadoop 2.7.0 during the week
>> 3) Although transient failures still remain after the deadline, I will
>> call a release vote with the remaining transient issues.
>> 
>> What do you think about this plan?
>> 
>> Thanks,
>> Taegeon
>> 
>>> On May 11, 2017, at 12:56 PM, Sergiy Matusevych <
>> sergiy.matusevych@gmail.com> wrote:
>>> 
>>> Hi Taegeon,
>>> 
>>> I am afraid I won't be able to look at either of these issues this week.
>> I
>>> am very busy working on my slides on Distributed Factorization Machines,
>>> and I have a presentation on Friday.
>>> 
>>> We can release 0.16-preview (or beta, or rc1, or whatever you call it) -
>> I
>>> think we are good in terms of features, and there is a finite number of
>>> bugs that should not stop early adopters from using 0.16-preview
>>> 
>>> What do you guys think?
>>> 
>>> Cheers,
>>> Sergiy.
>>> 
>>> 
>>> 
>>> 
>>> On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <ta...@gmail.com>
>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> Thanks Sergiy for taking a look at them!
>>>> As far as I know, ApacheCon opens next week (May 16-18), so I think we
>>>> need to resolve the issues until the end of this week.
>>>> 
>>>> I think I can help you in investigating REEF-1770. However, I’m not
>> sure I
>>>> can fix it until the end of this week.
>>>> REEF-1770 is a transient failure, but there is no transient failure
>> during
>>>> the past 30 days in our Java-side Travis CI (0 transient failure out of
>> 69
>>>> builds).
>>>> So, maybe it could be hard to reproduce it.
>>>> 
>>>> Sergiy, do you think you can resolve REEF-1796 until the end of this
>> week?
>>>> If not, we have two options.
>>>> 
>>>> 1) release 0.16 during the week of ApacheCon without fixing them
>>>> (Actually, the .NET side CI is unstable, but just release 0.16)
>>>> 2) do not release 0.16 until the bugs (in addition to the .NET side CI
>>>> failures) are resolved
>>>> 
>>>> What do you guys think?
>>>> 
>>>> Thanks,
>>>> Taegeon
>>>> 
>>>>> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
>>>> sergiy.matusevych@gmail.com> wrote:
>>>>> 
>>>>> Hi guys,
>>>>> 
>>>>> It surely would be great to announce 0.16 at the conference, and we
>> have
>>>>> some awesome features to brag about - most notably, REEF-on-Spark.
>> Still,
>>>>> there are a few bugs that we need to fix before the release. I am
>> mostly
>>>>> concerned with https://issues.apache.org/jira/browse/REEF-1770 and
>>>>> especially the https://issues.apache.org/jira/browse/REEF-1796 I am
>>>> looking
>>>>> at them now, but any help would be greatly appreciated!
>>>>> 
>>>>> Cheers,
>>>>> Sergiy.
>>>>> 
>>>>> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com>
>>>> wrote:
>>>>> 
>>>>>> Any update?
>>>>>> It'd be great if we can release 0.16 during the week of ApacheCon.
>>>>>> 
>>>>>> -Gon
>>>>>> 
>>>>>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com>
>>>> wrote:
>>>>>> 
>>>>>>> Unfortunately, we’ve also got a recent build failure in Java side
>> [1],
>>>>>>> which is not reported previously.
>>>>>>> I’ve created an issue [2] to track this failure, and am going to
>>>>>>> investigate it.
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> Taegeon
>>>>>>> 
>>>>>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
>>>>>>> https://travis-ci.org/apache/reef/builds/220731026>
>>>>>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
>>>>>>> https://issues.apache.org/jira/browse/REEF-1770>
>>>>>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <
>> mamykhai@microsoft.com
>>>> .
>>>>>> INVALID>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> At least 3 of the issues previously reported under REEF-1462 have
>>>>>>> re-occurred in the past two days (I've reopened corresponding JIRAs
>> and
>>>>>>> attached links to failures). Unfortunately, with the transient
>> failures
>>>>>>> like these one good build is insufficient.
>>>>>>>> 
>>>>>>>> It is a known issue, since we're using free access to AppVeyor, our
>>>>>>> builds are sequential and low-priority, so sometimes when a lot of
>> pull
>>>>>>> requests have to be built the build queue takes a while to drain.
>>>>>>>> 
>>>>>>>> -Mariia
>>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
>>>>>>>> Sent: Wednesday, April 5, 2017 12:11 AM
>>>>>>>> To: dev@reef.apache.org
>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>> 
>>>>>>>> Awesome!
>>>>>>>> 
>>>>>>>> There is no build failure in the .Net side with the latest build
>> [1].
>>>>>>>> 
>>>>>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis
>> CI's
>>>>>>> already done. We're still waiting for Appveyor. :(
>>>>>>>> 
>>>>>>>> [1]
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com
>> %
>>>>>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
>>>> 2BhaBnkHjFnkwxLh6GiPubCBDb%
>>>>>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
>>>>>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
>>>>>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
>>>>>>> data=02%7C01%7Cmamykhai%40microsoft.com%
>> 7C90c5159366d44f7efdd308d47bf2
>>>>>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
>>>>>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
>>>> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
>>>>>>> E9Q%3D&reserved=0
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Thanks Julia for the work!
>>>>>>>>> 
>>>>>>>>> It looks like Java and .NET builds are almost stable, except for
>> the
>>>>>>>>> recent build failure in .NET side [1].
>>>>>>>>> As Julia said in REEF-1406 [2], we would need to wait for time if
>>>> this
>>>>>>>>> failure is reproduced or not.
>>>>>>>>> 
>>>>>>>>> I will wait for a week and call a release vote if there are no
>> build
>>>>>>>>> failures during that time.
>>>>>>>>> Thanks!
>>>>>>>>> 
>>>>>>>>> Taegeon
>>>>>>>>> 
>>>>>>>>> [1]:
>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>> https%3A%2F%2Fci.ap
>>>>>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>>> 2Freef%2F&data=02%7C01
>>>>>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>> fda1%7C72f98
>>>>>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
>>>>>> sdata=Y9iO%2B
>>>>>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
>>>>>>>>> build/1453-master
>>>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>>>>>> https%3A%2F%2Fci.a
>>>>>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
>>>>>> %7C90c
>>>>>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>>> db47%7C1%7
>>>>>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
>>>>>> M6avrry5nk
>>>>>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
>>>>>>>>> [2]:
>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>> https%3A%2F%2Fissue
>>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>>>> 7Cmamykhai%40mic
>>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>>> https%3A%2F%2Fissue
>>>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>>>> 7Cmamykhai%40mic
>>>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
>>>>>>>>> 
>>>>>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
>>>>>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
>>>>>>>>>> 
>>>>>>>>>> I have resolved all the .Net test issues for now. The fixes
>> contain
>>>>>>>>>> what
>>>>>>>>> I have identifies so far based on the failures.
>>>>>>>>>> 
>>>>>>>>>> I agree with Marria, as they are transit failures, also they
>> failed
>>>>>>>>>> for
>>>>>>>>> multiple reasons sometimes, we need to continue to observe if the
>>>>>>>>> issues come back again.
>>>>>>>>>> 
>>>>>>>>>> Thanks,
>>>>>>>>>> Julia
>>>>>>>>>> 
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
>>>>>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>> 
>>>>>>>>>> Thanks Mariia for pointing it out to me.
>>>>>>>>>> Yes. I agree that we need more time to fix all of the transient
>>>>>>> failures.
>>>>>>>>>> After they are resolved, I will wait for some time to ensure that
>>>>>>>>>> they
>>>>>>>>> are not reoccurred.
>>>>>>>>>> 
>>>>>>>>>> Thanks!
>>>>>>>>>> Taegeon
>>>>>>>>>> 
>>>>>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
>>>>>>>>>>> <ma...@microsoft.com.INVALID>
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Please note that due to the transient nature of .NET failures, it
>>>>>>>>>>> makes
>>>>>>>>> sense to wait for some time and to observe whether they are
>> actually
>>>>>>>>> fixed or just lying low until the next reoccurrence. We had to
>> reopen
>>>>>>>>> some bugs which looked resolved in the past but then reoccurred.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> -Mariia
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> ________________________________
>>>>>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
>>>>>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>> 
>>>>>>>>>>> Hi,
>>>>>>>>>>> 
>>>>>>>>>>> Julia has been doing a great work to resolve the .NET side
>> issues.
>>>>>>>>>>> It looks like she has resolved 3 issues recently (and now 3
>> issues
>>>>>>>>> remain in .NET side with 1 pending PR).
>>>>>>>>>>> 
>>>>>>>>>>> Sergiy and I also have worked for the java side issues, and we've
>>>>>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
>>>>>>>>>>> 
>>>>>>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
>>>>>>>>>>> think
>>>>>>>>> it would be good to delay the release vote.
>>>>>>>>>>> However, judging from the progress we made, I think all of the
>>>>>>>>>>> issues
>>>>>>>>> could be resolved until at the end of this week or begging of next
>>>>>> week.
>>>>>>>>>>> 
>>>>>>>>>>> I will call a vote as soon as possible after they are resolved.
>>>>>>>>>>> Thanks!
>>>>>>>>>>> 
>>>>>>>>>>> Taegeon
>>>>>>>>>>> 
>>>>>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
>>>>>>>>>>>> 
>>>>>>>>>>>> Taegeon, could you give us status update? Thanks.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
>>>>>>>>>>>> <bg...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
>>>>>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
>>>>>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
>>>>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
>>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Oh, never mind.
>>>>>>>>>>>>>>> I thought that we still need some time to make sure that
>>>>>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Great, then we have only two items left! :-) I can confirm
>> that
>>>>>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need
>> to
>>>>>>>>>>>>>> address the Java issues (that is, item #1) as they are related
>>>>>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
>>>>>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
>>>>>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
>>>>>>>>>>>>>> currently
>>>>>>>>> have a System.exit() call at the end.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks for help!
>>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
>>>>>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Hi Taegeon,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
>>>>>>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
>>>>>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have
>> several
>>>>>>>>>>>>>>>> examples and unit tests that check
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
>>>>>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
>>>>>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think
>> we
>>>>>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
>>>>>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>> area you think we should focus on for 0.16 release)
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
>>>>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> It's been about 10 months since we've released the latest
>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>> (0.15
>>>>>>>>>>>>>>>> version).
>>>>>>>>>>>>>>>>> In order not to delay the release any longer, I want to
>> call
>>>>>>>>>>>>>>>>> a release
>>>>>>>>>>>>>>>> vote as soon as possible.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
>>>>>>>>>>>>>>>>> next
>>>>>>>>>>>>>>> Thursday
>>>>>>>>>>>>>>>> (23th)?
>>>>>>>>>>>>>>>>> I know there still remain several blocking issues:
>>>>>>>>>>>>>>>>> 1) Java side CI failures
>>>>>>>>>>>>>>>>> 2) .NET side CI failures
>>>>>>>>>>>>>>>>> 3) Unmanaged AM test
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I want to know if it is possible that they can be resolved
>>>>>>>>>>>>>>>>> until next
>>>>>>>>>>>>>>>> Thursday.
>>>>>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
>>>>>>>>>>>>>>>>> the due date
>>>>>>>>>>>>>>>> is ok to me.
>>>>>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
>>>>>>>>>>>>>>>>> Sergiy is
>>>>>>>>>>>>>>>> working on 3).
>>>>>>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
>>>>>>> them?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>> Taegeon
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> --
>>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Byung-Gon Chun
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Byung-Gon Chun
>>>>>> 
>>>> 
>>>> 
>> 
>> 
> 
> 
> -- 
> Byung-Gon Chun

Re: 0.16 release plan

Posted by Byung-Gon Chun <bg...@gmail.com>.

Hi Taegeon,

Can you share the current release progress with the community?

Thanks.
-Gon


On Fri, Jul 21, 2017 at 1:34 PM, Tae-Geon Um <ta...@gmail.com> wrote:

> Hi all,
>
> Again, I think it’s time to release 0.16; I would like to share the
> release plan :)
>
> For the release 0.16, it looks like we have a consensus on the following
> items:
> 1) releasing 0.16 with known transient failures (.NET side)
> 2) upgrading to hadoop 2.7.0 for the new release
>
> If you have any objection to these things, please share your opinion.
>
> Here is the release plan.
> 1) I will wait *one week* for resolving the remaining transient failures
> 2) I will upgrade to hadoop 2.7.0 during the week
> 3) Although transient failures still remain after the deadline, I will
> call a release vote with the remaining transient issues.
>
> What do you think about this plan?
>
> Thanks,
> Taegeon
>
> > On May 11, 2017, at 12:56 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
> >
> > Hi Taegeon,
> >
> > I am afraid I won't be able to look at either of these issues this week.
> I
> > am very busy working on my slides on Distributed Factorization Machines,
> > and I have a presentation on Friday.
> >
> > We can release 0.16-preview (or beta, or rc1, or whatever you call it) -
> I
> > think we are good in terms of features, and there is a finite number of
> > bugs that should not stop early adopters from using 0.16-preview
> >
> > What do you guys think?
> >
> > Cheers,
> > Sergiy.
> >
> >
> >
> >
> > On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <ta...@gmail.com>
> wrote:
> >
> >> Hi,
> >>
> >> Thanks Sergiy for taking a look at them!
> >> As far as I know, ApacheCon opens next week (May 16-18), so I think we
> >> need to resolve the issues until the end of this week.
> >>
> >> I think I can help you in investigating REEF-1770. However, I’m not
> sure I
> >> can fix it until the end of this week.
> >> REEF-1770 is a transient failure, but there is no transient failure
> during
> >> the past 30 days in our Java-side Travis CI (0 transient failure out of
> 69
> >> builds).
> >> So, maybe it could be hard to reproduce it.
> >>
> >> Sergiy, do you think you can resolve REEF-1796 until the end of this
> week?
> >> If not, we have two options.
> >>
> >> 1) release 0.16 during the week of ApacheCon without fixing them
> >> (Actually, the .NET side CI is unstable, but just release 0.16)
> >> 2) do not release 0.16 until the bugs (in addition to the .NET side CI
> >> failures) are resolved
> >>
> >> What do you guys think?
> >>
> >> Thanks,
> >> Taegeon
> >>
> >>> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
> >> sergiy.matusevych@gmail.com> wrote:
> >>>
> >>> Hi guys,
> >>>
> >>> It surely would be great to announce 0.16 at the conference, and we
> have
> >>> some awesome features to brag about - most notably, REEF-on-Spark.
> Still,
> >>> there are a few bugs that we need to fix before the release. I am
> mostly
> >>> concerned with https://issues.apache.org/jira/browse/REEF-1770 and
> >>> especially the https://issues.apache.org/jira/browse/REEF-1796 I am
> >> looking
> >>> at them now, but any help would be greatly appreciated!
> >>>
> >>> Cheers,
> >>> Sergiy.
> >>>
> >>> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com>
> >> wrote:
> >>>
> >>>> Any update?
> >>>> It'd be great if we can release 0.16 during the week of ApacheCon.
> >>>>
> >>>> -Gon
> >>>>
> >>>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com>
> >> wrote:
> >>>>
> >>>>> Unfortunately, we’ve also got a recent build failure in Java side
> [1],
> >>>>> which is not reported previously.
> >>>>> I’ve created an issue [2] to track this failure, and am going to
> >>>>> investigate it.
> >>>>>
> >>>>> Thanks,
> >>>>> Taegeon
> >>>>>
> >>>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
> >>>>> https://travis-ci.org/apache/reef/builds/220731026>
> >>>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
> >>>>> https://issues.apache.org/jira/browse/REEF-1770>
> >>>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <
> mamykhai@microsoft.com
> >> .
> >>>> INVALID>
> >>>>> wrote:
> >>>>>>
> >>>>>> At least 3 of the issues previously reported under REEF-1462 have
> >>>>> re-occurred in the past two days (I've reopened corresponding JIRAs
> and
> >>>>> attached links to failures). Unfortunately, with the transient
> failures
> >>>>> like these one good build is insufficient.
> >>>>>>
> >>>>>> It is a known issue, since we're using free access to AppVeyor, our
> >>>>> builds are sequential and low-priority, so sometimes when a lot of
> pull
> >>>>> requests have to be built the build queue takes a while to drain.
> >>>>>>
> >>>>>> -Mariia
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
> >>>>>> Sent: Wednesday, April 5, 2017 12:11 AM
> >>>>>> To: dev@reef.apache.org
> >>>>>> Subject: Re: 0.16 release plan
> >>>>>>
> >>>>>> Awesome!
> >>>>>>
> >>>>>> There is no build failure in the .Net side with the latest build
> [1].
> >>>>>>
> >>>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis
> CI's
> >>>>> already done. We're still waiting for Appveyor. :(
> >>>>>>
> >>>>>> [1]
> >>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
> >>>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com
> %
> >>>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> >>>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
> >> 2BhaBnkHjFnkwxLh6GiPubCBDb%
> >>>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
> >>>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
> >>>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
> >>>>> data=02%7C01%7Cmamykhai%40microsoft.com%
> 7C90c5159366d44f7efdd308d47bf2
> >>>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
> >>>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
> >> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
> >>>>> E9Q%3D&reserved=0
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Thanks Julia for the work!
> >>>>>>>
> >>>>>>> It looks like Java and .NET builds are almost stable, except for
> the
> >>>>>>> recent build failure in .NET side [1].
> >>>>>>> As Julia said in REEF-1406 [2], we would need to wait for time if
> >> this
> >>>>>>> failure is reproduced or not.
> >>>>>>>
> >>>>>>> I will wait for a week and call a release vote if there are no
> build
> >>>>>>> failures during that time.
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>> Taegeon
> >>>>>>>
> >>>>>>> [1]:
> >>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>> https%3A%2F%2Fci.ap
> >>>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
> >>>> 2Freef%2F&data=02%7C01
> >>>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> >>>> fda1%7C72f98
> >>>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
> >>>> sdata=Y9iO%2B
> >>>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
> >>>>>>> build/1453-master
> >>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
> >>>> https%3A%2F%2Fci.a
> >>>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
> >>>> %7C90c
> >>>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> >>>> db47%7C1%7
> >>>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
> >>>> M6avrry5nk
> >>>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
> >>>>>>> [2]:
> >>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>> https%3A%2F%2Fissue
> >>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> >>>> 7Cmamykhai%40mic
> >>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> >>>> fda1%7C72f988bf86f141af91ab2d
> >>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> >>>> 2F9yenZoGqe%2FkowHza7
> >>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
> >>>>>>> https://na01.safelinks.protection.outlook.com/?url=
> >>>> https%3A%2F%2Fissue
> >>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> >>>> 7Cmamykhai%40mic
> >>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> >>>> fda1%7C72f988bf86f141af91ab2d
> >>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> >>>> 2F9yenZoGqe%2FkowHza7
> >>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
> >>>>>>>
> >>>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
> >>>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
> >>>>>>>>
> >>>>>>>> I have resolved all the .Net test issues for now. The fixes
> contain
> >>>>>>>> what
> >>>>>>> I have identifies so far based on the failures.
> >>>>>>>>
> >>>>>>>> I agree with Marria, as they are transit failures, also they
> failed
> >>>>>>>> for
> >>>>>>> multiple reasons sometimes, we need to continue to observe if the
> >>>>>>> issues come back again.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Julia
> >>>>>>>>
> >>>>>>>> -----Original Message-----
> >>>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
> >>>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
> >>>>>>>> To: dev@reef.apache.org
> >>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>
> >>>>>>>> Thanks Mariia for pointing it out to me.
> >>>>>>>> Yes. I agree that we need more time to fix all of the transient
> >>>>> failures.
> >>>>>>>> After they are resolved, I will wait for some time to ensure that
> >>>>>>>> they
> >>>>>>> are not reoccurred.
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>> Taegeon
> >>>>>>>>
> >>>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
> >>>>>>>>> <ma...@microsoft.com.INVALID>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Please note that due to the transient nature of .NET failures, it
> >>>>>>>>> makes
> >>>>>>> sense to wait for some time and to observe whether they are
> actually
> >>>>>>> fixed or just lying low until the next reoccurrence. We had to
> reopen
> >>>>>>> some bugs which looked resolved in the past but then reoccurred.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> -Mariia
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> ________________________________
> >>>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
> >>>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
> >>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Julia has been doing a great work to resolve the .NET side
> issues.
> >>>>>>>>> It looks like she has resolved 3 issues recently (and now 3
> issues
> >>>>>>> remain in .NET side with 1 pending PR).
> >>>>>>>>>
> >>>>>>>>> Sergiy and I also have worked for the java side issues, and we've
> >>>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
> >>>>>>>>>
> >>>>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
> >>>>>>>>> think
> >>>>>>> it would be good to delay the release vote.
> >>>>>>>>> However, judging from the progress we made, I think all of the
> >>>>>>>>> issues
> >>>>>>> could be resolved until at the end of this week or begging of next
> >>>> week.
> >>>>>>>>>
> >>>>>>>>> I will call a vote as soon as possible after they are resolved.
> >>>>>>>>> Thanks!
> >>>>>>>>>
> >>>>>>>>> Taegeon
> >>>>>>>>>
> >>>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
> >>>>>>>>>>
> >>>>>>>>>> Taegeon, could you give us status update? Thanks.
> >>>>>>>>>>
> >>>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
> >>>>>>>>>> <bg...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
> >>>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
> >>>>>>>>>>>>
> >>>>>>>>>>>> -----Original Message-----
> >>>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
> >>>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
> >>>>>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
> >>>>>>>>>>>> <ta...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Oh, never mind.
> >>>>>>>>>>>>> I thought that we still need some time to make sure that
> >>>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Great, then we have only two items left! :-) I can confirm
> that
> >>>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need
> to
> >>>>>>>>>>>> address the Java issues (that is, item #1) as they are related
> >>>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
> >>>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
> >>>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
> >>>>>>>>>>>> currently
> >>>>>>> have a System.exit() call at the end.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks for help!
> >>>>>>>>>>>> Sergiy.
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
> >>>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Taegeon,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
> >>>>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
> >>>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have
> several
> >>>>>>>>>>>>>> examples and unit tests that check
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
> >>>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
> >>>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think
> we
> >>>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
> >>>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
> >>>>>>>>>>>>> what
> >>>>>>>>>>>>>> area you think we should focus on for 0.16 release)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>> Sergiy.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
> >>>>>>>>>>>>>> <ta...@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> It's been about 10 months since we've released the latest
> >>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>> (0.15
> >>>>>>>>>>>>>> version).
> >>>>>>>>>>>>>>> In order not to delay the release any longer, I want to
> call
> >>>>>>>>>>>>>>> a release
> >>>>>>>>>>>>>> vote as soon as possible.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
> >>>>>>>>>>>>>>> next
> >>>>>>>>>>>>> Thursday
> >>>>>>>>>>>>>> (23th)?
> >>>>>>>>>>>>>>> I know there still remain several blocking issues:
> >>>>>>>>>>>>>>> 1) Java side CI failures
> >>>>>>>>>>>>>>> 2) .NET side CI failures
> >>>>>>>>>>>>>>> 3) Unmanaged AM test
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I want to know if it is possible that they can be resolved
> >>>>>>>>>>>>>>> until next
> >>>>>>>>>>>>>> Thursday.
> >>>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
> >>>>>>>>>>>>>>> the due date
> >>>>>>>>>>>>>> is ok to me.
> >>>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
> >>>>>>>>>>>>>>> Sergiy is
> >>>>>>>>>>>>>> working on 3).
> >>>>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
> >>>>> them?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Taegeon
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Byung-Gon Chun
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Byung-Gon Chun
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Byung-Gon Chun
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Byung-Gon Chun
> >>>>
> >>
> >>
>
>


-- 
Byung-Gon Chun

Re: 0.16 release plan

Posted by Tae-Geon Um <ta...@gmail.com>.

Hi all, 

Again, I think it’s time to release 0.16; I would like to share the release plan :)

For the release 0.16, it looks like we have a consensus on the following items: 
1) releasing 0.16 with known transient failures (.NET side) 
2) upgrading to hadoop 2.7.0 for the new release

If you have any objection to these things, please share your opinion. 

Here is the release plan. 
1) I will wait *one week* for resolving the remaining transient failures
2) I will upgrade to hadoop 2.7.0 during the week
3) Although transient failures still remain after the deadline, I will call a release vote with the remaining transient issues. 

What do you think about this plan? 

Thanks,
Taegeon

> On May 11, 2017, at 12:56 PM, Sergiy Matusevych <se...@gmail.com> wrote:
> 
> Hi Taegeon,
> 
> I am afraid I won't be able to look at either of these issues this week. I
> am very busy working on my slides on Distributed Factorization Machines,
> and I have a presentation on Friday.
> 
> We can release 0.16-preview (or beta, or rc1, or whatever you call it) - I
> think we are good in terms of features, and there is a finite number of
> bugs that should not stop early adopters from using 0.16-preview
> 
> What do you guys think?
> 
> Cheers,
> Sergiy.
> 
> 
> 
> 
> On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <ta...@gmail.com> wrote:
> 
>> Hi,
>> 
>> Thanks Sergiy for taking a look at them!
>> As far as I know, ApacheCon opens next week (May 16-18), so I think we
>> need to resolve the issues until the end of this week.
>> 
>> I think I can help you in investigating REEF-1770. However, I’m not sure I
>> can fix it until the end of this week.
>> REEF-1770 is a transient failure, but there is no transient failure during
>> the past 30 days in our Java-side Travis CI (0 transient failure out of 69
>> builds).
>> So, maybe it could be hard to reproduce it.
>> 
>> Sergiy, do you think you can resolve REEF-1796 until the end of this week?
>> If not, we have two options.
>> 
>> 1) release 0.16 during the week of ApacheCon without fixing them
>> (Actually, the .NET side CI is unstable, but just release 0.16)
>> 2) do not release 0.16 until the bugs (in addition to the .NET side CI
>> failures) are resolved
>> 
>> What do you guys think?
>> 
>> Thanks,
>> Taegeon
>> 
>>> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
>> sergiy.matusevych@gmail.com> wrote:
>>> 
>>> Hi guys,
>>> 
>>> It surely would be great to announce 0.16 at the conference, and we have
>>> some awesome features to brag about - most notably, REEF-on-Spark. Still,
>>> there are a few bugs that we need to fix before the release. I am mostly
>>> concerned with https://issues.apache.org/jira/browse/REEF-1770 and
>>> especially the https://issues.apache.org/jira/browse/REEF-1796 I am
>> looking
>>> at them now, but any help would be greatly appreciated!
>>> 
>>> Cheers,
>>> Sergiy.
>>> 
>>> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com>
>> wrote:
>>> 
>>>> Any update?
>>>> It'd be great if we can release 0.16 during the week of ApacheCon.
>>>> 
>>>> -Gon
>>>> 
>>>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com>
>> wrote:
>>>> 
>>>>> Unfortunately, we’ve also got a recent build failure in Java side [1],
>>>>> which is not reported previously.
>>>>> I’ve created an issue [2] to track this failure, and am going to
>>>>> investigate it.
>>>>> 
>>>>> Thanks,
>>>>> Taegeon
>>>>> 
>>>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
>>>>> https://travis-ci.org/apache/reef/builds/220731026>
>>>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
>>>>> https://issues.apache.org/jira/browse/REEF-1770>
>>>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <mamykhai@microsoft.com
>> .
>>>> INVALID>
>>>>> wrote:
>>>>>> 
>>>>>> At least 3 of the issues previously reported under REEF-1462 have
>>>>> re-occurred in the past two days (I've reopened corresponding JIRAs and
>>>>> attached links to failures). Unfortunately, with the transient failures
>>>>> like these one good build is insufficient.
>>>>>> 
>>>>>> It is a known issue, since we're using free access to AppVeyor, our
>>>>> builds are sequential and low-priority, so sometimes when a lot of pull
>>>>> requests have to be built the build queue takes a while to drain.
>>>>>> 
>>>>>> -Mariia
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
>>>>>> Sent: Wednesday, April 5, 2017 12:11 AM
>>>>>> To: dev@reef.apache.org
>>>>>> Subject: Re: 0.16 release plan
>>>>>> 
>>>>>> Awesome!
>>>>>> 
>>>>>> There is no build failure in the .Net side with the latest build [1].
>>>>>> 
>>>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis CI's
>>>>> already done. We're still waiting for Appveyor. :(
>>>>>> 
>>>>>> [1]
>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com%
>>>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
>> 2BhaBnkHjFnkwxLh6GiPubCBDb%
>>>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
>>>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
>>>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
>>>>> data=02%7C01%7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
>>>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
>> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
>>>>> E9Q%3D&reserved=0
>>>>>> 
>>>>>> 
>>>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>>> Thanks Julia for the work!
>>>>>>> 
>>>>>>> It looks like Java and .NET builds are almost stable, except for the
>>>>>>> recent build failure in .NET side [1].
>>>>>>> As Julia said in REEF-1406 [2], we would need to wait for time if
>> this
>>>>>>> failure is reproduced or not.
>>>>>>> 
>>>>>>> I will wait for a week and call a release vote if there are no build
>>>>>>> failures during that time.
>>>>>>> Thanks!
>>>>>>> 
>>>>>>> Taegeon
>>>>>>> 
>>>>>>> [1]:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>> https%3A%2F%2Fci.ap
>>>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>>> 2Freef%2F&data=02%7C01
>>>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>>> fda1%7C72f98
>>>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
>>>> sdata=Y9iO%2B
>>>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
>>>>>>> build/1453-master
>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>>>> https%3A%2F%2Fci.a
>>>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
>>>> %7C90c
>>>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>>> db47%7C1%7
>>>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
>>>> M6avrry5nk
>>>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
>>>>>>> [2]:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>> https%3A%2F%2Fissue
>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>> 7Cmamykhai%40mic
>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=
>>>> https%3A%2F%2Fissue
>>>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>>>> 7Cmamykhai%40mic
>>>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>>>> fda1%7C72f988bf86f141af91ab2d
>>>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>>>> 2F9yenZoGqe%2FkowHza7
>>>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
>>>>>>> 
>>>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
>>>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
>>>>>>>> 
>>>>>>>> I have resolved all the .Net test issues for now. The fixes contain
>>>>>>>> what
>>>>>>> I have identifies so far based on the failures.
>>>>>>>> 
>>>>>>>> I agree with Marria, as they are transit failures, also they failed
>>>>>>>> for
>>>>>>> multiple reasons sometimes, we need to continue to observe if the
>>>>>>> issues come back again.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> Julia
>>>>>>>> 
>>>>>>>> -----Original Message-----
>>>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
>>>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
>>>>>>>> To: dev@reef.apache.org
>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>> 
>>>>>>>> Thanks Mariia for pointing it out to me.
>>>>>>>> Yes. I agree that we need more time to fix all of the transient
>>>>> failures.
>>>>>>>> After they are resolved, I will wait for some time to ensure that
>>>>>>>> they
>>>>>>> are not reoccurred.
>>>>>>>> 
>>>>>>>> Thanks!
>>>>>>>> Taegeon
>>>>>>>> 
>>>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
>>>>>>>>> <ma...@microsoft.com.INVALID>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Please note that due to the transient nature of .NET failures, it
>>>>>>>>> makes
>>>>>>> sense to wait for some time and to observe whether they are actually
>>>>>>> fixed or just lying low until the next reoccurrence. We had to reopen
>>>>>>> some bugs which looked resolved in the past but then reoccurred.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> -Mariia
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> ________________________________
>>>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
>>>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>> 
>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>> Julia has been doing a great work to resolve the .NET side issues.
>>>>>>>>> It looks like she has resolved 3 issues recently (and now 3 issues
>>>>>>> remain in .NET side with 1 pending PR).
>>>>>>>>> 
>>>>>>>>> Sergiy and I also have worked for the java side issues, and we've
>>>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
>>>>>>>>> 
>>>>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
>>>>>>>>> think
>>>>>>> it would be good to delay the release vote.
>>>>>>>>> However, judging from the progress we made, I think all of the
>>>>>>>>> issues
>>>>>>> could be resolved until at the end of this week or begging of next
>>>> week.
>>>>>>>>> 
>>>>>>>>> I will call a vote as soon as possible after they are resolved.
>>>>>>>>> Thanks!
>>>>>>>>> 
>>>>>>>>> Taegeon
>>>>>>>>> 
>>>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
>>>>>>>>>> 
>>>>>>>>>> Taegeon, could you give us status update? Thanks.
>>>>>>>>>> 
>>>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
>>>>>>>>>> <bg...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
>>>>>>>>>>> 
>>>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
>>>>>>>>>>> 
>>>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
>>>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
>>>>>>>>>>>> 
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
>>>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
>>>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Oh, never mind.
>>>>>>>>>>>>> I thought that we still need some time to make sure that
>>>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Great, then we have only two items left! :-) I can confirm that
>>>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need to
>>>>>>>>>>>> address the Java issues (that is, item #1) as they are related
>>>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
>>>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
>>>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
>>>>>>>>>>>> currently
>>>>>>> have a System.exit() call at the end.
>>>>>>>>>>>> 
>>>>>>>>>>>> Thanks for help!
>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>> 
>>>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
>>>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Hi Taegeon,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
>>>>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
>>>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have several
>>>>>>>>>>>>>> examples and unit tests that check
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
>>>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
>>>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think we
>>>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
>>>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
>>>>>>>>>>>>> what
>>>>>>>>>>>>>> area you think we should focus on for 0.16 release)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
>>>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> It's been about 10 months since we've released the latest
>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>> (0.15
>>>>>>>>>>>>>> version).
>>>>>>>>>>>>>>> In order not to delay the release any longer, I want to call
>>>>>>>>>>>>>>> a release
>>>>>>>>>>>>>> vote as soon as possible.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
>>>>>>>>>>>>>>> next
>>>>>>>>>>>>> Thursday
>>>>>>>>>>>>>> (23th)?
>>>>>>>>>>>>>>> I know there still remain several blocking issues:
>>>>>>>>>>>>>>> 1) Java side CI failures
>>>>>>>>>>>>>>> 2) .NET side CI failures
>>>>>>>>>>>>>>> 3) Unmanaged AM test
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I want to know if it is possible that they can be resolved
>>>>>>>>>>>>>>> until next
>>>>>>>>>>>>>> Thursday.
>>>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
>>>>>>>>>>>>>>> the due date
>>>>>>>>>>>>>> is ok to me.
>>>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
>>>>>>>>>>>>>>> Sergiy is
>>>>>>>>>>>>>> working on 3).
>>>>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
>>>>> them?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Taegeon
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Byung-Gon Chun
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Byung-Gon Chun
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Byung-Gon Chun
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Byung-Gon Chun
>>>> 
>> 
>>

Re: 0.16 release plan

Posted by Sergiy Matusevych <se...@gmail.com>.

Hi Taegeon,

I am afraid I won't be able to look at either of these issues this week. I
am very busy working on my slides on Distributed Factorization Machines,
and I have a presentation on Friday.

We can release 0.16-preview (or beta, or rc1, or whatever you call it) - I
think we are good in terms of features, and there is a finite number of
bugs that should not stop early adopters from using 0.16-preview

What do you guys think?

Cheers,
Sergiy.




On Wed, May 10, 2017 at 7:23 PM, Tae-Geon Um <ta...@gmail.com> wrote:

> Hi,
>
> Thanks Sergiy for taking a look at them!
> As far as I know, ApacheCon opens next week (May 16-18), so I think we
> need to resolve the issues until the end of this week.
>
> I think I can help you in investigating REEF-1770. However, I’m not sure I
> can fix it until the end of this week.
> REEF-1770 is a transient failure, but there is no transient failure during
> the past 30 days in our Java-side Travis CI (0 transient failure out of 69
> builds).
> So, maybe it could be hard to reproduce it.
>
> Sergiy, do you think you can resolve REEF-1796 until the end of this week?
> If not, we have two options.
>
> 1) release 0.16 during the week of ApacheCon without fixing them
> (Actually, the .NET side CI is unstable, but just release 0.16)
> 2) do not release 0.16 until the bugs (in addition to the .NET side CI
> failures) are resolved
>
> What do you guys think?
>
> Thanks,
> Taegeon
>
> > On May 11, 2017, at 6:55 AM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
> >
> > Hi guys,
> >
> > It surely would be great to announce 0.16 at the conference, and we have
> > some awesome features to brag about - most notably, REEF-on-Spark. Still,
> > there are a few bugs that we need to fix before the release. I am mostly
> > concerned with https://issues.apache.org/jira/browse/REEF-1770 and
> > especially the https://issues.apache.org/jira/browse/REEF-1796 I am
> looking
> > at them now, but any help would be greatly appreciated!
> >
> > Cheers,
> > Sergiy.
> >
> > On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com>
> wrote:
> >
> >> Any update?
> >> It'd be great if we can release 0.16 during the week of ApacheCon.
> >>
> >> -Gon
> >>
> >> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com>
> wrote:
> >>
> >>> Unfortunately, we’ve also got a recent build failure in Java side [1],
> >>> which is not reported previously.
> >>> I’ve created an issue [2] to track this failure, and am going to
> >>> investigate it.
> >>>
> >>> Thanks,
> >>> Taegeon
> >>>
> >>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
> >>> https://travis-ci.org/apache/reef/builds/220731026>
> >>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
> >>> https://issues.apache.org/jira/browse/REEF-1770>
> >>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <mamykhai@microsoft.com
> .
> >> INVALID>
> >>> wrote:
> >>>>
> >>>> At least 3 of the issues previously reported under REEF-1462 have
> >>> re-occurred in the past two days (I've reopened corresponding JIRAs and
> >>> attached links to failures). Unfortunately, with the transient failures
> >>> like these one good build is insufficient.
> >>>>
> >>>> It is a known issue, since we're using free access to AppVeyor, our
> >>> builds are sequential and low-priority, so sometimes when a lot of pull
> >>> requests have to be built the build queue takes a while to drain.
> >>>>
> >>>> -Mariia
> >>>>
> >>>> -----Original Message-----
> >>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
> >>>> Sent: Wednesday, April 5, 2017 12:11 AM
> >>>> To: dev@reef.apache.org
> >>>> Subject: Re: 0.16 release plan
> >>>>
> >>>> Awesome!
> >>>>
> >>>> There is no build failure in the .Net side with the latest build [1].
> >>>>
> >>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis CI's
> >>> already done. We're still waiting for Appveyor. :(
> >>>>
> >>>> [1]
> >>>> https://na01.safelinks.protection.outlook.com/?url=
> >>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
> >>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com%
> >>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> >>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%
> 2BhaBnkHjFnkwxLh6GiPubCBDb%
> >>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
> >>>> [2] https://na01.safelinks.protection.outlook.com/?url=
> >>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
> >>> data=02%7C01%7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> >>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
> >>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%
> 2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
> >>> E9Q%3D&reserved=0
> >>>>
> >>>>
> >>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
> >>> wrote:
> >>>>
> >>>>> Thanks Julia for the work!
> >>>>>
> >>>>> It looks like Java and .NET builds are almost stable, except for the
> >>>>> recent build failure in .NET side [1].
> >>>>> As Julia said in REEF-1406 [2], we would need to wait for time if
> this
> >>>>> failure is reproduced or not.
> >>>>>
> >>>>> I will wait for a week and call a release vote if there are no build
> >>>>> failures during that time.
> >>>>> Thanks!
> >>>>>
> >>>>> Taegeon
> >>>>>
> >>>>> [1]:
> >>>>> https://na01.safelinks.protection.outlook.com/?url=
> >> https%3A%2F%2Fci.ap
> >>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
> >> 2Freef%2F&data=02%7C01
> >>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> >> fda1%7C72f98
> >>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
> >> sdata=Y9iO%2B
> >>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
> >>>>> build/1453-master
> >>>>> <https://na01.safelinks.protection.outlook.com/?url=
> >> https%3A%2F%2Fci.a
> >>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
> >> %7C90c
> >>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> >> db47%7C1%7
> >>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
> >> M6avrry5nk
> >>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
> >>>>> [2]:
> >>>>> https://na01.safelinks.protection.outlook.com/?url=
> >> https%3A%2F%2Fissue
> >>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> >> 7Cmamykhai%40mic
> >>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> >> fda1%7C72f988bf86f141af91ab2d
> >>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> >> 2F9yenZoGqe%2FkowHza7
> >>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
> >>>>> https://na01.safelinks.protection.outlook.com/?url=
> >> https%3A%2F%2Fissue
> >>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> >> 7Cmamykhai%40mic
> >>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> >> fda1%7C72f988bf86f141af91ab2d
> >>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> >> 2F9yenZoGqe%2FkowHza7
> >>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
> >>>>>
> >>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
> >>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
> >>>>>>
> >>>>>> I have resolved all the .Net test issues for now. The fixes contain
> >>>>>> what
> >>>>> I have identifies so far based on the failures.
> >>>>>>
> >>>>>> I agree with Marria, as they are transit failures, also they failed
> >>>>>> for
> >>>>> multiple reasons sometimes, we need to continue to observe if the
> >>>>> issues come back again.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Julia
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
> >>>>>> Sent: Thursday, March 23, 2017 5:20 PM
> >>>>>> To: dev@reef.apache.org
> >>>>>> Subject: Re: 0.16 release plan
> >>>>>>
> >>>>>> Thanks Mariia for pointing it out to me.
> >>>>>> Yes. I agree that we need more time to fix all of the transient
> >>> failures.
> >>>>>> After they are resolved, I will wait for some time to ensure that
> >>>>>> they
> >>>>> are not reoccurred.
> >>>>>>
> >>>>>> Thanks!
> >>>>>> Taegeon
> >>>>>>
> >>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
> >>>>>>> <ma...@microsoft.com.INVALID>
> >>>>> wrote:
> >>>>>>>
> >>>>>>> Please note that due to the transient nature of .NET failures, it
> >>>>>>> makes
> >>>>> sense to wait for some time and to observe whether they are actually
> >>>>> fixed or just lying low until the next reoccurrence. We had to reopen
> >>>>> some bugs which looked resolved in the past but then reoccurred.
> >>>>>>>
> >>>>>>>
> >>>>>>> -Mariia
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> ________________________________
> >>>>>>> From: Tae-Geon Um <ta...@gmail.com>
> >>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
> >>>>>>> To: dev@reef.apache.org
> >>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> Julia has been doing a great work to resolve the .NET side issues.
> >>>>>>> It looks like she has resolved 3 issues recently (and now 3 issues
> >>>>> remain in .NET side with 1 pending PR).
> >>>>>>>
> >>>>>>> Sergiy and I also have worked for the java side issues, and we've
> >>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
> >>>>>>>
> >>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
> >>>>>>> think
> >>>>> it would be good to delay the release vote.
> >>>>>>> However, judging from the progress we made, I think all of the
> >>>>>>> issues
> >>>>> could be resolved until at the end of this week or begging of next
> >> week.
> >>>>>>>
> >>>>>>> I will call a vote as soon as possible after they are resolved.
> >>>>>>> Thanks!
> >>>>>>>
> >>>>>>> Taegeon
> >>>>>>>
> >>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>> Thank you for all the efforts to make release 0.16 happen!
> >>>>>>>>
> >>>>>>>> Taegeon, could you give us status update? Thanks.
> >>>>>>>>
> >>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
> >>>>>>>> <bg...@gmail.com>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Thanks for looking at .Net CI failures, Julia!
> >>>>>>>>>
> >>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
> >>>>>>>>>
> >>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
> >>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
> >>>>>>>>>
> >>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
> >>>>>>>>>>
> >>>>>>>>>> -----Original Message-----
> >>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
> >>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
> >>>>>>>>>> To: dev@reef.apache.org
> >>>>>>>>>> Subject: Re: 0.16 release plan
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
> >>>>>>>>>> <ta...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Oh, never mind.
> >>>>>>>>>>> I thought that we still need some time to make sure that
> >>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Great, then we have only two items left! :-) I can confirm that
> >>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need to
> >>>>>>>>>> address the Java issues (that is, item #1) as they are related
> >>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
> >>>>>>>>>> all threads before exiting REEF Driver - otherwise,
> >>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
> >>>>>>>>>> currently
> >>>>> have a System.exit() call at the end.
> >>>>>>>>>>
> >>>>>>>>>> Thanks for help!
> >>>>>>>>>> Sergiy.
> >>>>>>>>>>
> >>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
> >>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi Taegeon,
> >>>>>>>>>>>>
> >>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
> >>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
> >>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have several
> >>>>>>>>>>>> examples and unit tests that check
> >>>>>>>>>>> the
> >>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
> >>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
> >>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think we
> >>>>>>>>>>>> should unit test? (I am saying that our unit tests are
> >>>>>>>>>>>> comprehensive (they are not!), but I would love to know
> >>>>>>>>>>> what
> >>>>>>>>>>>> area you think we should focus on for 0.16 release)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Cheers,
> >>>>>>>>>>>> Sergiy.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
> >>>>>>>>>>>> <ta...@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> It's been about 10 months since we've released the latest
> >>>>>>>>>>>>> version
> >>>>>>>>>>>>> (0.15
> >>>>>>>>>>>> version).
> >>>>>>>>>>>>> In order not to delay the release any longer, I want to call
> >>>>>>>>>>>>> a release
> >>>>>>>>>>>> vote as soon as possible.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
> >>>>>>>>>>>>> next
> >>>>>>>>>>> Thursday
> >>>>>>>>>>>> (23th)?
> >>>>>>>>>>>>> I know there still remain several blocking issues:
> >>>>>>>>>>>>> 1) Java side CI failures
> >>>>>>>>>>>>> 2) .NET side CI failures
> >>>>>>>>>>>>> 3) Unmanaged AM test
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I want to know if it is possible that they can be resolved
> >>>>>>>>>>>>> until next
> >>>>>>>>>>>> Thursday.
> >>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
> >>>>>>>>>>>>> the due date
> >>>>>>>>>>>> is ok to me.
> >>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
> >>>>>>>>>>>>> Sergiy is
> >>>>>>>>>>>> working on 3).
> >>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
> >>> them?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>> Taegeon
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Byung-Gon Chun
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Byung-Gon Chun
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Byung-Gon Chun
> >>>
> >>>
> >>
> >>
> >> --
> >> Byung-Gon Chun
> >>
>
>

Re: 0.16 release plan

Posted by Tae-Geon Um <ta...@gmail.com>.

Hi, 

Thanks Sergiy for taking a look at them! 
As far as I know, ApacheCon opens next week (May 16-18), so I think we need to resolve the issues until the end of this week. 

I think I can help you in investigating REEF-1770. However, I’m not sure I can fix it until the end of this week.
REEF-1770 is a transient failure, but there is no transient failure during the past 30 days in our Java-side Travis CI (0 transient failure out of 69 builds). 
So, maybe it could be hard to reproduce it. 

Sergiy, do you think you can resolve REEF-1796 until the end of this week? 
If not, we have two options.

1) release 0.16 during the week of ApacheCon without fixing them (Actually, the .NET side CI is unstable, but just release 0.16) 
2) do not release 0.16 until the bugs (in addition to the .NET side CI failures) are resolved 

What do you guys think? 

Thanks,
Taegeon

> On May 11, 2017, at 6:55 AM, Sergiy Matusevych <se...@gmail.com> wrote:
> 
> Hi guys,
> 
> It surely would be great to announce 0.16 at the conference, and we have
> some awesome features to brag about - most notably, REEF-on-Spark. Still,
> there are a few bugs that we need to fix before the release. I am mostly
> concerned with https://issues.apache.org/jira/browse/REEF-1770 and
> especially the https://issues.apache.org/jira/browse/REEF-1796 I am looking
> at them now, but any help would be greatly appreciated!
> 
> Cheers,
> Sergiy.
> 
> On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com> wrote:
> 
>> Any update?
>> It'd be great if we can release 0.16 during the week of ApacheCon.
>> 
>> -Gon
>> 
>> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com> wrote:
>> 
>>> Unfortunately, we’ve also got a recent build failure in Java side [1],
>>> which is not reported previously.
>>> I’ve created an issue [2] to track this failure, and am going to
>>> investigate it.
>>> 
>>> Thanks,
>>> Taegeon
>>> 
>>> [1]: https://travis-ci.org/apache/reef/builds/220731026 <
>>> https://travis-ci.org/apache/reef/builds/220731026>
>>> [2]: https://issues.apache.org/jira/browse/REEF-1770 <
>>> https://issues.apache.org/jira/browse/REEF-1770>
>>>> On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <mamykhai@microsoft.com.
>> INVALID>
>>> wrote:
>>>> 
>>>> At least 3 of the issues previously reported under REEF-1462 have
>>> re-occurred in the past two days (I've reopened corresponding JIRAs and
>>> attached links to failures). Unfortunately, with the transient failures
>>> like these one good build is insufficient.
>>>> 
>>>> It is a known issue, since we're using free access to AppVeyor, our
>>> builds are sequential and low-priority, so sometimes when a lot of pull
>>> requests have to be built the build queue takes a while to drain.
>>>> 
>>>> -Mariia
>>>> 
>>>> -----Original Message-----
>>>> From: Byung-Gon Chun [mailto:bgchun@gmail.com]
>>>> Sent: Wednesday, April 5, 2017 12:11 AM
>>>> To: dev@reef.apache.org
>>>> Subject: Re: 0.16 release plan
>>>> 
>>>> Awesome!
>>>> 
>>>> There is no build failure in the .Net side with the latest build [1].
>>>> 
>>>> It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis CI's
>>> already done. We're still waiting for Appveyor. :(
>>>> 
>>>> [1]
>>>> https://na01.safelinks.protection.outlook.com/?url=
>>> https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
>>> 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com%
>>> 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>>> db47%7C1%7C0%7C636269730940876660&sdata=j2h9%2BhaBnkHjFnkwxLh6GiPubCBDb%
>>> 2B5%2B3S8Ok6aU2dc%3D&reserved=0
>>>> [2] https://na01.safelinks.protection.outlook.com/?url=
>>> https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
>>> data=02%7C01%7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>>> fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
>>> 7C636269730940876660&sdata=oF%2F6YP9JkpD%2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
>>> E9Q%3D&reserved=0
>>>> 
>>>> 
>>>> On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
>>> wrote:
>>>> 
>>>>> Thanks Julia for the work!
>>>>> 
>>>>> It looks like Java and .NET builds are almost stable, except for the
>>>>> recent build failure in .NET side [1].
>>>>> As Julia said in REEF-1406 [2], we would need to wait for time if this
>>>>> failure is reproduced or not.
>>>>> 
>>>>> I will wait for a week and call a release vote if there are no build
>>>>> failures during that time.
>>>>> Thanks!
>>>>> 
>>>>> Taegeon
>>>>> 
>>>>> [1]:
>>>>> https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fci.ap
>>>>> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
>> 2Freef%2F&data=02%7C01
>>>>> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
>> fda1%7C72f98
>>>>> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
>> sdata=Y9iO%2B
>>>>> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
>>>>> build/1453-master
>>>>> <https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fci.a
>>>>> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
>> %7C90c
>>>>> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
>> db47%7C1%7
>>>>> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
>> M6avrry5nk
>>>>> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
>>>>> [2]:
>>>>> https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fissue
>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>> 7Cmamykhai%40mic
>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>> fda1%7C72f988bf86f141af91ab2d
>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>> 2F9yenZoGqe%2FkowHza7
>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
>>>>> https://na01.safelinks.protection.outlook.com/?url=
>> https%3A%2F%2Fissue
>>>>> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
>> 7Cmamykhai%40mic
>>>>> rosoft.com%7C90c5159366d44f7efdd308d47bf2
>> fda1%7C72f988bf86f141af91ab2d
>>>>> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
>> 2F9yenZoGqe%2FkowHza7
>>>>> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
>>>>> 
>>>>>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
>>>>> Qiuhe.Wang@microsoft.com.INVALID> wrote:
>>>>>> 
>>>>>> I have resolved all the .Net test issues for now. The fixes contain
>>>>>> what
>>>>> I have identifies so far based on the failures.
>>>>>> 
>>>>>> I agree with Marria, as they are transit failures, also they failed
>>>>>> for
>>>>> multiple reasons sometimes, we need to continue to observe if the
>>>>> issues come back again.
>>>>>> 
>>>>>> Thanks,
>>>>>> Julia
>>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
>>>>>> Sent: Thursday, March 23, 2017 5:20 PM
>>>>>> To: dev@reef.apache.org
>>>>>> Subject: Re: 0.16 release plan
>>>>>> 
>>>>>> Thanks Mariia for pointing it out to me.
>>>>>> Yes. I agree that we need more time to fix all of the transient
>>> failures.
>>>>>> After they are resolved, I will wait for some time to ensure that
>>>>>> they
>>>>> are not reoccurred.
>>>>>> 
>>>>>> Thanks!
>>>>>> Taegeon
>>>>>> 
>>>>>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
>>>>>>> <ma...@microsoft.com.INVALID>
>>>>> wrote:
>>>>>>> 
>>>>>>> Please note that due to the transient nature of .NET failures, it
>>>>>>> makes
>>>>> sense to wait for some time and to observe whether they are actually
>>>>> fixed or just lying low until the next reoccurrence. We had to reopen
>>>>> some bugs which looked resolved in the past but then reoccurred.
>>>>>>> 
>>>>>>> 
>>>>>>> -Mariia
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ________________________________
>>>>>>> From: Tae-Geon Um <ta...@gmail.com>
>>>>>>> Sent: Thursday, March 23, 2017 6:51:24 AM
>>>>>>> To: dev@reef.apache.org
>>>>>>> Subject: Re: 0.16 release plan
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> Julia has been doing a great work to resolve the .NET side issues.
>>>>>>> It looks like she has resolved 3 issues recently (and now 3 issues
>>>>> remain in .NET side with 1 pending PR).
>>>>>>> 
>>>>>>> Sergiy and I also have worked for the java side issues, and we've
>>>>> resolved 1 issue (and 1 issue still remains with 1 pending PR).
>>>>>>> 
>>>>>>> Because of the unresolved issues (3 .NET side and 1 java side), I
>>>>>>> think
>>>>> it would be good to delay the release vote.
>>>>>>> However, judging from the progress we made, I think all of the
>>>>>>> issues
>>>>> could be resolved until at the end of this week or begging of next
>> week.
>>>>>>> 
>>>>>>> I will call a vote as soon as possible after they are resolved.
>>>>>>> Thanks!
>>>>>>> 
>>>>>>> Taegeon
>>>>>>> 
>>>>>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
>>> wrote:
>>>>>>>> 
>>>>>>>> Thank you for all the efforts to make release 0.16 happen!
>>>>>>>> 
>>>>>>>> Taegeon, could you give us status update? Thanks.
>>>>>>>> 
>>>>>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
>>>>>>>> <bg...@gmail.com>
>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Thanks for looking at .Net CI failures, Julia!
>>>>>>>>> 
>>>>>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
>>>>>>>>> 
>>>>>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
>>>>>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
>>>>>>>>> 
>>>>>>>>>> I am working on some of the .Net AppVeyor test failures now.
>>>>>>>>>> 
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
>>>>>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
>>>>>>>>>> To: dev@reef.apache.org
>>>>>>>>>> Subject: Re: 0.16 release plan
>>>>>>>>>> 
>>>>>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
>>>>>>>>>> <ta...@gmail.com>
>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Oh, never mind.
>>>>>>>>>>> I thought that we still need some time to make sure that
>>>>>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Great, then we have only two items left! :-) I can confirm that
>>>>>>>>>> Unmanaged AM works on proper version of YARN; still, we need to
>>>>>>>>>> address the Java issues (that is, item #1) as they are related
>>>>>>>>>> to the Unmanaged AM mode. For example, we must make sure close
>>>>>>>>>> all threads before exiting REEF Driver - otherwise,
>>>>>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
>>>>>>>>>> currently
>>>>> have a System.exit() call at the end.
>>>>>>>>>> 
>>>>>>>>>> Thanks for help!
>>>>>>>>>> Sergiy.
>>>>>>>>>> 
>>>>>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
>>>>>>>>>>> sergiy.matusevych@gmail.com> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Hi Taegeon,
>>>>>>>>>>>> 
>>>>>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
>>>>>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
>>>>>>>>>>>> class), and it works fine on YARN 2.7.3. We also have several
>>>>>>>>>>>> examples and unit tests that check
>>>>>>>>>>> the
>>>>>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
>>>>>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
>>>>>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think we
>>>>>>>>>>>> should unit test? (I am saying that our unit tests are
>>>>>>>>>>>> comprehensive (they are not!), but I would love to know
>>>>>>>>>>> what
>>>>>>>>>>>> area you think we should focus on for 0.16 release)
>>>>>>>>>>>> 
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Sergiy.
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
>>>>>>>>>>>> <ta...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> It's been about 10 months since we've released the latest
>>>>>>>>>>>>> version
>>>>>>>>>>>>> (0.15
>>>>>>>>>>>> version).
>>>>>>>>>>>>> In order not to delay the release any longer, I want to call
>>>>>>>>>>>>> a release
>>>>>>>>>>>> vote as soon as possible.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
>>>>>>>>>>>>> next
>>>>>>>>>>> Thursday
>>>>>>>>>>>> (23th)?
>>>>>>>>>>>>> I know there still remain several blocking issues:
>>>>>>>>>>>>> 1) Java side CI failures
>>>>>>>>>>>>> 2) .NET side CI failures
>>>>>>>>>>>>> 3) Unmanaged AM test
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I want to know if it is possible that they can be resolved
>>>>>>>>>>>>> until next
>>>>>>>>>>>> Thursday.
>>>>>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
>>>>>>>>>>>>> the due date
>>>>>>>>>>>> is ok to me.
>>>>>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
>>>>>>>>>>>>> Sergiy is
>>>>>>>>>>>> working on 3).
>>>>>>>>>>>>> If the plan seems not ok, could you please share the ETA of
>>> them?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Taegeon
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Byung-Gon Chun
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Byung-Gon Chun
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Byung-Gon Chun
>>> 
>>> 
>> 
>> 
>> --
>> Byung-Gon Chun
>>

Re: 0.16 release plan

Posted by Sergiy Matusevych <se...@gmail.com>.

Hi guys,

It surely would be great to announce 0.16 at the conference, and we have
some awesome features to brag about - most notably, REEF-on-Spark. Still,
there are a few bugs that we need to fix before the release. I am mostly
concerned with https://issues.apache.org/jira/browse/REEF-1770 and
especially the https://issues.apache.org/jira/browse/REEF-1796 I am looking
at them now, but any help would be greatly appreciated!

Cheers,
Sergiy.

On Tue, May 9, 2017 at 11:55 PM, Byung-Gon Chun <bg...@gmail.com> wrote:

> Any update?
> It'd be great if we can release 0.16 during the week of ApacheCon.
>
> -Gon
>
> On Tue, Apr 11, 2017 at 10:27 AM, Tae-Geon Um <ta...@gmail.com> wrote:
>
> > Unfortunately, we’ve also got a recent build failure in Java side [1],
> > which is not reported previously.
> > I’ve created an issue [2] to track this failure, and am going to
> > investigate it.
> >
> > Thanks,
> > Taegeon
> >
> > [1]: https://travis-ci.org/apache/reef/builds/220731026 <
> > https://travis-ci.org/apache/reef/builds/220731026>
> > [2]: https://issues.apache.org/jira/browse/REEF-1770 <
> > https://issues.apache.org/jira/browse/REEF-1770>
> > > On Apr 6, 2017, at 3:13 AM, Mariia Mykhailova <mamykhai@microsoft.com.
> INVALID>
> > wrote:
> > >
> > > At least 3 of the issues previously reported under REEF-1462 have
> > re-occurred in the past two days (I've reopened corresponding JIRAs and
> > attached links to failures). Unfortunately, with the transient failures
> > like these one good build is insufficient.
> > >
> > > It is a known issue, since we're using free access to AppVeyor, our
> > builds are sequential and low-priority, so sometimes when a lot of pull
> > requests have to be built the build queue takes a while to drain.
> > >
> > > -Mariia
> > >
> > > -----Original Message-----
> > > From: Byung-Gon Chun [mailto:bgchun@gmail.com]
> > > Sent: Wednesday, April 5, 2017 12:11 AM
> > > To: dev@reef.apache.org
> > > Subject: Re: 0.16 release plan
> > >
> > > Awesome!
> > >
> > > There is no build failure in the .Net side with the latest build [1].
> > >
> > > It looks like Appveyor's quite slow. Regarding PR1284 [2], Travis CI's
> > already done. We're still waiting for Appveyor. :(
> > >
> > > [1]
> > > https://na01.safelinks.protection.outlook.com/?url=
> > https%3A%2F%2Fci.appveyor.com%2Fproject%2FApacheSoftwareFoundation%
> > 2Freef%2Fbuild%2F1455-master&data=02%7C01%7Cmamykhai%40microsoft.com%
> > 7C90c5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> > db47%7C1%7C0%7C636269730940876660&sdata=j2h9%2BhaBnkHjFnkwxLh6GiPubCBDb%
> > 2B5%2B3S8Ok6aU2dc%3D&reserved=0
> > > [2] https://na01.safelinks.protection.outlook.com/?url=
> > https%3A%2F%2Fgithub.com%2Fapache%2Freef%2Fpull%2F1284&
> > data=02%7C01%7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> > fda1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
> > 7C636269730940876660&sdata=oF%2F6YP9JkpD%2FahyiP9Yu6MpIrOEd48MEVQQ8w1Iw
> > E9Q%3D&reserved=0
> > >
> > >
> > > On Tue, Apr 4, 2017 at 10:29 AM, Tae-Geon Um <ta...@gmail.com>
> > wrote:
> > >
> > >> Thanks Julia for the work!
> > >>
> > >> It looks like Java and .NET builds are almost stable, except for the
> > >> recent build failure in .NET side [1].
> > >> As Julia said in REEF-1406 [2], we would need to wait for time if this
> > >> failure is reproduced or not.
> > >>
> > >> I will wait for a week and call a release vote if there are no build
> > >> failures during that time.
> > >> Thanks!
> > >>
> > >> Taegeon
> > >>
> > >> [1]:
> > >> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fci.ap
> > >> pveyor.com%2Fproject%2FApacheSoftwareFoundation%
> 2Freef%2F&data=02%7C01
> > >> %7Cmamykhai%40microsoft.com%7C90c5159366d44f7efdd308d47bf2
> fda1%7C72f98
> > >> 8bf86f141af91ab2d7cd011db47%7C1%7C0%7C636269730940876660&
> sdata=Y9iO%2B
> > >> YRbPNirr38T%2BNJtxEQg0xm65lOb0P%2Bc5w6agYI%3D&reserved=0
> > >> build/1453-master
> > >> <https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fci.a
> > >> ppveyor.com%2Fproject%2F&data=02%7C01%7Cmamykhai%40microsoft.com
> %7C90c
> > >> 5159366d44f7efdd308d47bf2fda1%7C72f988bf86f141af91ab2d7cd011
> db47%7C1%7
> > >> C0%7C636269730940876660&sdata=exulSYnqM0PxkRJBTpxAd825tbhRnt
> M6avrry5nk
> > >> Nfw%3D&reserved=0 ApacheSoftwareFoundation/reef/build/1453-master>
> > >> [2]:
> > >> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissue
> > >> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> 7Cmamykhai%40mic
> > >> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> fda1%7C72f988bf86f141af91ab2d
> > >> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> 2F9yenZoGqe%2FkowHza7
> > >> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0 <
> > >> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Fissue
> > >> s.apache.org%2Fjira%2Fbrowse%2FREEF-1406&data=02%7C01%
> 7Cmamykhai%40mic
> > >> rosoft.com%7C90c5159366d44f7efdd308d47bf2
> fda1%7C72f988bf86f141af91ab2d
> > >> 7cd011db47%7C1%7C0%7C636269730940876660&sdata=oS%
> 2F9yenZoGqe%2FkowHza7
> > >> m2T531qmGySb7q1qGmX%2FTJA%3D&reserved=0>
> > >>
> > >>> On Mar 30, 2017, at 10:28 AM, Julia Wang (QIUHE) <
> > >> Qiuhe.Wang@microsoft.com.INVALID> wrote:
> > >>>
> > >>> I have resolved all the .Net test issues for now. The fixes contain
> > >>> what
> > >> I have identifies so far based on the failures.
> > >>>
> > >>> I agree with Marria, as they are transit failures, also they failed
> > >>> for
> > >> multiple reasons sometimes, we need to continue to observe if the
> > >> issues come back again.
> > >>>
> > >>> Thanks,
> > >>> Julia
> > >>>
> > >>> -----Original Message-----
> > >>> From: Tae-Geon Um [mailto:taegeonum@gmail.com]
> > >>> Sent: Thursday, March 23, 2017 5:20 PM
> > >>> To: dev@reef.apache.org
> > >>> Subject: Re: 0.16 release plan
> > >>>
> > >>> Thanks Mariia for pointing it out to me.
> > >>> Yes. I agree that we need more time to fix all of the transient
> > failures.
> > >>> After they are resolved, I will wait for some time to ensure that
> > >>> they
> > >> are not reoccurred.
> > >>>
> > >>> Thanks!
> > >>> Taegeon
> > >>>
> > >>>> On Mar 24, 2017, at 2:54 AM, Mariia Mykhailova
> > >>>> <ma...@microsoft.com.INVALID>
> > >> wrote:
> > >>>>
> > >>>> Please note that due to the transient nature of .NET failures, it
> > >>>> makes
> > >> sense to wait for some time and to observe whether they are actually
> > >> fixed or just lying low until the next reoccurrence. We had to reopen
> > >> some bugs which looked resolved in the past but then reoccurred.
> > >>>>
> > >>>>
> > >>>> -Mariia
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> ________________________________
> > >>>> From: Tae-Geon Um <ta...@gmail.com>
> > >>>> Sent: Thursday, March 23, 2017 6:51:24 AM
> > >>>> To: dev@reef.apache.org
> > >>>> Subject: Re: 0.16 release plan
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> Julia has been doing a great work to resolve the .NET side issues.
> > >>>> It looks like she has resolved 3 issues recently (and now 3 issues
> > >> remain in .NET side with 1 pending PR).
> > >>>>
> > >>>> Sergiy and I also have worked for the java side issues, and we've
> > >> resolved 1 issue (and 1 issue still remains with 1 pending PR).
> > >>>>
> > >>>> Because of the unresolved issues (3 .NET side and 1 java side), I
> > >>>> think
> > >> it would be good to delay the release vote.
> > >>>> However, judging from the progress we made, I think all of the
> > >>>> issues
> > >> could be resolved until at the end of this week or begging of next
> week.
> > >>>>
> > >>>> I will call a vote as soon as possible after they are resolved.
> > >>>> Thanks!
> > >>>>
> > >>>> Taegeon
> > >>>>
> > >>>>> On Mar 23, 2017, at 5:47 PM, Byung-Gon Chun <bg...@gmail.com>
> > wrote:
> > >>>>>
> > >>>>> Thank you for all the efforts to make release 0.16 happen!
> > >>>>>
> > >>>>> Taegeon, could you give us status update? Thanks.
> > >>>>>
> > >>>>> On Sat, Mar 18, 2017 at 10:01 AM, Byung-Gon Chun
> > >>>>> <bg...@gmail.com>
> > >> wrote:
> > >>>>>
> > >>>>>> Thanks for looking at .Net CI failures, Julia!
> > >>>>>>
> > >>>>>> Thanks for handling Java CI failures, Sergiy and Taegeon!
> > >>>>>>
> > >>>>>> On Fri, Mar 17, 2017 at 11:20 AM, Julia Wang (QIUHE) <
> > >>>>>> Qiuhe.Wang@microsoft.com.invalid> wrote:
> > >>>>>>
> > >>>>>>> I am working on some of the .Net AppVeyor test failures now.
> > >>>>>>>
> > >>>>>>> -----Original Message-----
> > >>>>>>> From: Sergiy Matusevych [mailto:sergiy.matusevych@gmail.com]
> > >>>>>>> Sent: Wednesday, March 15, 2017 5:12 PM
> > >>>>>>> To: dev@reef.apache.org
> > >>>>>>> Subject: Re: 0.16 release plan
> > >>>>>>>
> > >>>>>>> On Wed, Mar 15, 2017 at 4:21 PM, Tae-Geon Um
> > >>>>>>> <ta...@gmail.com>
> > >> wrote:
> > >>>>>>>
> > >>>>>>> Oh, never mind.
> > >>>>>>>> I thought that we still need some time to make sure that
> > >>>>>>>> Unmanaged AM works properly on Hadoop 2.7.3 :)
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> Great, then we have only two items left! :-) I can confirm that
> > >>>>>>> Unmanaged AM works on proper version of YARN; still, we need to
> > >>>>>>> address the Java issues (that is, item #1) as they are related
> > >>>>>>> to the Unmanaged AM mode. For example, we must make sure close
> > >>>>>>> all threads before exiting REEF Driver - otherwise,
> > >>>>>>> HelloREEFYarnUnmanagedAM example can hang as it does not
> > >>>>>>> currently
> > >> have a System.exit() call at the end.
> > >>>>>>>
> > >>>>>>> Thanks for help!
> > >>>>>>> Sergiy.
> > >>>>>>>
> > >>>>>>>> On Mar 16, 2017, at 6:07 AM, Sergiy Matusevych <
> > >>>>>>>> sergiy.matusevych@gmail.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Hi Taegeon,
> > >>>>>>>>>
> > >>>>>>>>> What exactly do you mean by #3? We have a HelloREEF example
> > >>>>>>>>> running in Unmanaged AM mode (see HelloREEFYarnUnmanagedAM
> > >>>>>>>>> class), and it works fine on YARN 2.7.3. We also have several
> > >>>>>>>>> examples and unit tests that check
> > >>>>>>>> the
> > >>>>>>>>> Unmanaged AM and REEF-as-a-library functionality, e.g.
> > >>>>>>>>> HelloREEFEnvironment, ReefOnReefDriver,
> > >>>>>>>>> REEFEnvironmentDriverTest, and such. What else do you think we
> > >>>>>>>>> should unit test? (I am saying that our unit tests are
> > >>>>>>>>> comprehensive (they are not!), but I would love to know
> > >>>>>>>> what
> > >>>>>>>>> area you think we should focus on for 0.16 release)
> > >>>>>>>>>
> > >>>>>>>>> Cheers,
> > >>>>>>>>> Sergiy.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 15, 2017 at 7:10 AM, Tae-Geon Um
> > >>>>>>>>> <ta...@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi all,
> > >>>>>>>>>>
> > >>>>>>>>>> It's been about 10 months since we've released the latest
> > >>>>>>>>>> version
> > >>>>>>>>>> (0.15
> > >>>>>>>>> version).
> > >>>>>>>>>> In order not to delay the release any longer, I want to call
> > >>>>>>>>>> a release
> > >>>>>>>>> vote as soon as possible.
> > >>>>>>>>>>
> > >>>>>>>>>> Do you think it is ok for me to call a 0.16 release vote on
> > >>>>>>>>>> next
> > >>>>>>>> Thursday
> > >>>>>>>>> (23th)?
> > >>>>>>>>>> I know there still remain several blocking issues:
> > >>>>>>>>>> 1) Java side CI failures
> > >>>>>>>>>> 2) .NET side CI failures
> > >>>>>>>>>> 3) Unmanaged AM test
> > >>>>>>>>>>
> > >>>>>>>>>> I want to know if it is possible that they can be resolved
> > >>>>>>>>>> until next
> > >>>>>>>>> Thursday.
> > >>>>>>>>>> I'm currently taking a look at 1) (with Sergiy's help), and
> > >>>>>>>>>> the due date
> > >>>>>>>>> is ok to me.
> > >>>>>>>>>> How about 2) and 3) ? As far as I know, 2) is on Julia and
> > >>>>>>>>>> Sergiy is
> > >>>>>>>>> working on 3).
> > >>>>>>>>>> If the plan seems not ok, could you please share the ETA of
> > them?
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks,
> > >>>>>>>>>> Taegeon
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Byung-Gon Chun
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Byung-Gon Chun
> > >>>>
> > >>>
> > >>
> > >>
> > >
> > >
> > > --
> > > Byung-Gon Chun
> >
> >
>
>
> --
> Byung-Gon Chun
>