You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Bolke de Bruin <bd...@gmail.com> on 2017/04/03 09:05:09 UTC
Re: 1.8.1 release
Hey Chris,
AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the impact is though.
AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the scheduler log files. I did downgrade from blocker to critical.
AIRFLOW-719 Has a PR, including much needed increased test coverage, that I am pretty sure is working, but needs verification (plz @Alex)
I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around at the moment.
Furthermore, do you have any timelines for getting to the release? Can I help in any way? You might want to chase a couple of times ;-).
Bolke
> On 30 Mar 2017, at 19:48, siddharth anand <sa...@apache.org> wrote:
>
> Chris,
> I've submitted PRs for :
>
> - PR [AIRFLOW-1013] :
> https://github.com/apache/incubator-airflow/pull/2203
> - PR [AIRFLOW-1054]:
> https://github.com/apache/incubator-airflow/pull/2201
>
> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
> created if catchup=False :
> https://issues.apache.org/jira/browse/AIRFLOW-1055
>
> I have a PR that works for this, but will need to add unit tests for it as
> well as for AIRFLOW-1013.
>
> -s
>
> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org> wrote:
>
>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
>> blocker. I will have a PR shortly.
>> -s
>>
>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>> are listed as part of the 1.8.1 release:
>>>
>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>
>>> I'm going to merge them in now.
>>>
>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <cr...@apache.org>
>>> wrote:
>>>
>>>> Hey Bolke,
>>>>
>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>
>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>
>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bd...@gmail.com>
>>> wrote:
>>>>
>>>>> Hi Chris,
>>>>>
>>>>> I have a PR out for
>>>>>
>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>>>>> blockers list and just a new feature.
>>>>>
>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>>
>>>>> Cc: @alexvanboxel
>>>>>
>>>>> Bolke
>>>>>
>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <cr...@apache.org>
>>>>> wrote:
>>>>>>
>>>>>> Hey all,
>>>>>>
>>>>>> I've let this thread sit for a while. Here are a list of the issues
>>> that
>>>>>> were raised:
>>>>>>
>>>>>> BLOCKERS:
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>>>>>
>>>>>> NICE TO HAVE:
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>
>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>>>>>>
>>>>>> The rest remain open. I will wait on the release until the remaining
>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
>>>>>>
>>>>>> Ruslan, if you want to work on your nice to haves, and submit
>>> patches,
>>>>>> that's great, otherwise I don't believe they'll get fixed as part of
>>>>> 1.8.1.
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>>>>> dautkhanov@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Thank you Sid!
>>>>>>>
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Ruslan
>>>>>>>
>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>>> sanand@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Ruslan,
>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
>>> should
>>>>> aim
>>>>>>> to
>>>>>>>> get some of them into 1.8.1.
>>>>>>>>
>>>>>>>> -s
>>>>>>>>
>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>>>>> dautkhanov@gmail.com
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
>>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It would be great to have at least some of them fixed in 1.8.1.
>>>>>>>>>
>>>>>>>>> Thank you.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>
>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
>>> dan.davydov@airbnb.com
>>>>> .
>>>>>>>>> invalid
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
>>> general
>>>>>>> the
>>>>>>>>>> slow
>>>>>>>>>> startup time from this new logic of orphaned/reset task)
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
>>>>>>>>> hopefully
>>>>>>>>>> have a fix out for soon just finishing up tests)
>>>>>>>>>>
>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that we
>>>>> weren't
>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang (had
>>> to
>>>>>>> roll
>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon
>>> which
>>>>>>>>> should
>>>>>>>>>> be on the list too.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>>>>>>>> criccomini@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we want
>>> in
>>>>>>> the
>>>>>>>>>>> bugfix release.
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>>>>>>> bdbruin@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <bd...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) no new features in the 1.8 branch
>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
>>>>>>>>>>>>> 3) new features to land in 1.9
>>>>>>>>>>>>>
>>>>>>>>>>>>> This allows companies to
>>>>>>>>>>>>
>>>>>>>>>>>> Have a "known" version and can move to the new branch when they
>>>>>>>> want
>>>>>>>>> to
>>>>>>>>>>>> get new features. Obviously we only support N-1, so when 1.10
>>>>>>> comes
>>>>>>>>> out
>>>>>>>>>>> we
>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The goal
>>>>>>>>> would
>>>>>>>>>>> be:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) get a second release under our belt
>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that Airbnb
>>>>>>>> might
>>>>>>>>>>> want
>>>>>>>>>>>> to.
>>>>>>>>>>>>>> @Max et al, can you comment?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs to be
>>>>>>> in
>>>>>>>>> the
>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
Re: 1.8.1 release
Posted by Bolke de Bruin <bd...@gmail.com>.
Done.
> On 3 Apr 2017, at 22:22, Chris Riccomini <cr...@apache.org> wrote:
>
> Hey Bolke,
>
> > Furthermore, do you have any timelines for getting to the release
>
> Let's target April 17 to begin a vote for RC.
>
> > Can I help in any way
>
> I need people to review things. The more reviewers, the better.
>
> Also, can you please reach a conclusion on whether the issues you raised (AIRFLOW-1019 and AIRFLOW-1013) are blockers or not (or just change the status unilaterally if people aren't responding)?
>
> Cheers,
> Chris
>
> On Mon, Apr 3, 2017 at 1:06 PM, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>> wrote:
> I have a PR out for
>
> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 <https://github.com/apache/incubator-airflow/pull/2213>
> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 <https://github.com/apache/incubator-airflow/pull/2212>
>
> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t think it is a blocker: it happens only with @once dags that have a SLA, hardly very common. Nevertheless a fix would be nice obviously.
>
> Bolke
>
>> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hey Chris,
>>
>> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the impact is though.
>> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the scheduler log files. I did downgrade from blocker to critical.
>> AIRFLOW-719 Has a PR, including much needed increased test coverage, that I am pretty sure is working, but needs verification (plz @Alex)
>>
>> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around at the moment.
>>
>> Furthermore, do you have any timelines for getting to the release? Can I help in any way? You might want to chase a couple of times ;-).
>>
>> Bolke
>>
>>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org <ma...@apache.org>> wrote:
>>>
>>> Chris,
>>> I've submitted PRs for :
>>>
>>> - PR [AIRFLOW-1013] :
>>> https://github.com/apache/incubator-airflow/pull/2203 <https://github.com/apache/incubator-airflow/pull/2203>
>>> - PR [AIRFLOW-1054]:
>>> https://github.com/apache/incubator-airflow/pull/2201 <https://github.com/apache/incubator-airflow/pull/2201>
>>>
>>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
>>> created if catchup=False :
>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <https://issues.apache.org/jira/browse/AIRFLOW-1055>
>>>
>>> I have a PR that works for this, but will need to add unit tests for it as
>>> well as for AIRFLOW-1013.
>>>
>>> -s
>>>
>>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sanand@apache.org <ma...@apache.org>> wrote:
>>>
>>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013> was a
>>>> blocker. I will have a PR shortly.
>>>> -s
>>>>
>>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <criccomini@apache.org <ma...@apache.org>>
>>>> wrote:
>>>>
>>>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>>>> are listed as part of the 1.8.1 release:
>>>>>
>>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>>
>>>>> I'm going to merge them in now.
>>>>>
>>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <criccomini@apache.org <ma...@apache.org>>
>>>>> wrote:
>>>>>
>>>>>> Hey Bolke,
>>>>>>
>>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000 <https://issues.apache.org/jira/browse/AIRFLOW-1000>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001 <https://issues.apache.org/jira/browse/AIRFLOW-1001>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018 <https://issues.apache.org/jira/browse/AIRFLOW-1018>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 <https://issues.apache.org/jira/browse/AIRFLOW-1019>
>>>>>>
>>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20% <https://issues.apache.org/jira/issues/?jql=project%20%>
>>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>>>
>>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>>
>>>>> wrote:
>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> I have a PR out for
>>>>>>>
>>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>>>>>>> blockers list and just a new feature.
>>>>>>>
>>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <https://github.com/apache/incubator-airflow/pull/2195> <
>>>>>>> https://github.com/apache/incubator-airflow/pull/2195 <https://github.com/apache/incubator-airflow/pull/2195>>
>>>>>>>
>>>>>>> Cc: @alexvanboxel
>>>>>>>
>>>>>>> Bolke
>>>>>>>
>>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <criccomini@apache.org <ma...@apache.org>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hey all,
>>>>>>>>
>>>>>>>> I've let this thread sit for a while. Here are a list of the issues
>>>>> that
>>>>>>>> were raised:
>>>>>>>>
>>>>>>>> BLOCKERS:
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982 <https://issues.apache.org/jira/browse/AIRFLOW-982>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983 <https://issues.apache.org/jira/browse/AIRFLOW-983>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 <https://issues.apache.org/jira/browse/AIRFLOW-1019>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 <https://issues.apache.org/jira/browse/AIRFLOW-1017>
>>>>>>>>
>>>>>>>> NICE TO HAVE:
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015 <https://issues.apache.org/jira/browse/AIRFLOW-1015>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004 <https://issues.apache.org/jira/browse/AIRFLOW-1004>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003 <https://issues.apache.org/jira/browse/AIRFLOW-1003>
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001 <https://issues.apache.org/jira/browse/AIRFLOW-1001>
>>>>>>>>
>>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>>>>>>>>
>>>>>>>> The rest remain open. I will wait on the release until the remaining
>>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
>>>>>>>>
>>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
>>>>> patches,
>>>>>>>> that's great, otherwise I don't believe they'll get fixed as part of
>>>>>>> 1.8.1.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Chris
>>>>>>>>
>>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>>>>>>> dautkhanov@gmail.com <ma...@gmail.com>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thank you Sid!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Ruslan
>>>>>>>>>
>>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>>>>> sanand@apache.org <ma...@apache.org>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Ruslan,
>>>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
>>>>> should
>>>>>>> aim
>>>>>>>>> to
>>>>>>>>>> get some of them into 1.8.1.
>>>>>>>>>>
>>>>>>>>>> -s
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>>>>>>> dautkhanov@gmail.com <ma...@gmail.com>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015 <https://issues.apache.org/jira/browse/AIRFLOW-1015>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013 <https://issues.apache.org/jira/browse/AIRFLOW-1013>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004 <https://issues.apache.org/jira/browse/AIRFLOW-1004>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003 <https://issues.apache.org/jira/browse/AIRFLOW-1003>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001 <https://issues.apache.org/jira/browse/AIRFLOW-1001>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015 <https://issues.apache.org/jira/browse/AIRFLOW-1015>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It would be great to have at least some of them fixed in 1.8.1.
>>>>>>>>>>>
>>>>>>>>>>> Thank you.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
>>>>> dan.davydov@airbnb.com <ma...@airbnb.com>
>>>>>>> .
>>>>>>>>>>> invalid
>>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982 <https://issues.apache.org/jira/browse/AIRFLOW-982>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983 <https://issues.apache.org/jira/browse/AIRFLOW-983>
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 <https://issues.apache.org/jira/browse/AIRFLOW-1019> (and in
>>>>> general
>>>>>>>>> the
>>>>>>>>>>>> slow
>>>>>>>>>>>> startup time from this new logic of orphaned/reset task)
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 <https://issues.apache.org/jira/browse/AIRFLOW-1017> (which I will
>>>>>>>>>>> hopefully
>>>>>>>>>>>> have a fix out for soon just finishing up tests)
>>>>>>>>>>>>
>>>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that we
>>>>>>> weren't
>>>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang (had
>>>>> to
>>>>>>>>> roll
>>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon
>>>>> which
>>>>>>>>>>> should
>>>>>>>>>>>> be on the list too.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>>>>>>>>>> criccomini@apache.org <ma...@apache.org>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we want
>>>>> in
>>>>>>>>> the
>>>>>>>>>>>>> bugfix release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>>>>>>>>> bdbruin@gmail.com <ma...@gmail.com>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) no new features in the 1.8 branch
>>>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
>>>>>>>>>>>>>>> 3) new features to land in 1.9
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This allows companies to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Have a "known" version and can move to the new branch when they
>>>>>>>>>> want
>>>>>>>>>>> to
>>>>>>>>>>>>>> get new features. Obviously we only support N-1, so when 1.10
>>>>>>>>> comes
>>>>>>>>>>> out
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
>>>>>>>>>> criccomini@apache.org <ma...@apache.org>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The goal
>>>>>>>>>>> would
>>>>>>>>>>>>> be:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) get a second release under our belt
>>>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that Airbnb
>>>>>>>>>> might
>>>>>>>>>>>>> want
>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>> @Max et al, can you comment?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs to be
>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>
>
Re: 1.8.1 release
Posted by Chris Riccomini <cr...@apache.org>.
Hey Bolke,
> Furthermore, do you have any timelines for getting to the release
Let's target April 17 to begin a vote for RC.
> Can I help in any way
I need people to review things. The more reviewers, the better.
Also, can you please reach a conclusion on whether the issues you raised
(AIRFLOW-1019 and AIRFLOW-1013) are blockers or not (or just change the
status unilaterally if people aren't responding)?
Cheers,
Chris
On Mon, Apr 3, 2017 at 1:06 PM, Bolke de Bruin <bd...@gmail.com> wrote:
> I have a PR out for
>
> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213
> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212
>
> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t
> think it is a blocker: it happens only with @once dags that have a SLA,
> hardly very common. Nevertheless a fix would be nice obviously.
>
> Bolke
>
> On 3 Apr 2017, at 11:05, Bolke de Bruin <bd...@gmail.com> wrote:
>
> Hey Chris,
>
> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what
> the impact is though.
> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the
> scheduler log files. I did downgrade from blocker to critical.
> AIRFLOW-719 Has a PR, including much needed increased test coverage, that
> I am pretty sure is working, but needs verification (plz @Alex)
>
> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
> around at the moment.
>
> Furthermore, do you have any timelines for getting to the release? Can I
> help in any way? You might want to chase a couple of times ;-).
>
> Bolke
>
> On 30 Mar 2017, at 19:48, siddharth anand <sa...@apache.org> wrote:
>
> Chris,
> I've submitted PRs for :
>
> - PR [AIRFLOW-1013] :
> https://github.com/apache/incubator-airflow/pull/2203
> - PR [AIRFLOW-1054]:
> https://github.com/apache/incubator-airflow/pull/2201
>
> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
> created if catchup=False :
> https://issues.apache.org/jira/browse/AIRFLOW-1055
>
> I have a PR that works for this, but will need to add unit tests for it as
> well as for AIRFLOW-1013.
>
> -s
>
> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org>
> wrote:
>
> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
> blocker. I will have a PR shortly.
> -s
>
> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> The following three JIRAs were not merged into the v1-8-test branch, but
> are listed as part of the 1.8.1 release:
>
> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>
> I'm going to merge them in now.
>
> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> Hey Bolke,
>
> Great. Assuming your PR is committed, that leaves five blockers:
>
> https://issues.apache.org/jira/browse/AIRFLOW-1000
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1018
> https://issues.apache.org/jira/browse/AIRFLOW-1019
>
> I've also got a list of all open 1.8.1 JIRAs [1].
>
> Cheers,
> Chris
>
> [1] https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>
> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bd...@gmail.com>
>
> wrote:
>
>
> Hi Chris,
>
> I have a PR out for
>
> * Revert of 719, which makes 982 obsolete and removes 983 from the
> blockers list and just a new feature.
>
> See: https://github.com/apache/incubator-airflow/pull/2195 <
> https://github.com/apache/incubator-airflow/pull/2195>
>
> Cc: @alexvanboxel
>
> Bolke
>
> On 24 Mar 2017, at 10:21, Chris Riccomini <cr...@apache.org>
>
> wrote:
>
>
> Hey all,
>
> I've let this thread sit for a while. Here are a list of the issues
>
> that
>
> were raised:
>
> BLOCKERS:
> https://issues.apache.org/jira/browse/AIRFLOW-982
> https://issues.apache.org/jira/browse/AIRFLOW-983
> https://issues.apache.org/jira/browse/AIRFLOW-1019
> https://issues.apache.org/jira/browse/AIRFLOW-1017
>
> NICE TO HAVE:
> https://issues.apache.org/jira/browse/AIRFLOW-1015
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1004
> https://issues.apache.org/jira/browse/AIRFLOW-1003
> https://issues.apache.org/jira/browse/AIRFLOW-1001
>
> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>
> The rest remain open. I will wait on the release until the remaining
> blockers are finished. Dan/Daniel, can you comment on status?
>
> Ruslan, if you want to work on your nice to haves, and submit
>
> patches,
>
> that's great, otherwise I don't believe they'll get fixed as part of
>
> 1.8.1.
>
>
> Cheers,
> Chris
>
> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>
> dautkhanov@gmail.com>
>
> wrote:
>
> Thank you Sid!
>
>
> Best regards,
> Ruslan
>
> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>
> sanand@apache.org>
>
> wrote:
>
> Ruslan,
> Thanks for sharing this list. I can pick a few up. I agree we
>
> should
>
> aim
>
> to
>
> get some of them into 1.8.1.
>
> -s
>
> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>
> dautkhanov@gmail.com
>
>
> wrote:
>
> Some of the issues I ran into while testing 1.8rc5 :
>
> https://issues.apache.org/jira/browse/AIRFLOW-1015
>
> https://issues.apache.org/jira/browse/AIRFLOW-1013
> https://issues.apache.org/jira/browse/AIRFLOW-1004
> https://issues.apache.org/jira/browse/AIRFLOW-1003
> https://issues.apache.org/jira/browse/AIRFLOW-1001
> https://issues.apache.org/jira/browse/AIRFLOW-1015
>
>
>
> It would be great to have at least some of them fixed in 1.8.1.
>
> Thank you.
>
>
>
>
> --
> Ruslan Dautkhanov
>
> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
>
> dan.davydov@airbnb.com
>
> .
>
> invalid
>
> wrote:
>
>
> Here is my list for targeted 1.8.1 fixes:
> https://issues.apache.org/jira/browse/AIRFLOW-982
> https://issues.apache.org/jira/browse/AIRFLOW-983
> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
>
> general
>
> the
>
> slow
> startup time from this new logic of orphaned/reset task)
> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
>
> hopefully
>
> have a fix out for soon just finishing up tests)
>
> We are also hitting a new issue with subdags with rc5 that we
>
> weren't
>
> hitting with rc4 where subdags will occasionally just hang (had
>
> to
>
> roll
>
> back from rc5 to rc4), I'll try to spin up a JIRA for it soon
>
> which
>
> should
>
> be on the list too.
>
>
> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>
> criccomini@apache.org>
>
> wrote:
>
> Agreed. I'm looking for a list of checksums/JIRAs that we want
>
> in
>
> the
>
> bugfix release.
>
> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>
> bdbruin@gmail.com>
>
> wrote:
>
>
>
> On 21 Mar 2017, at 12:51, Bolke de Bruin <bd...@gmail.com>
>
> wrote:
>
>
> My suggestion, as we are using semantic versioning is:
>
> 1) no new features in the 1.8 branch
> 2) only bug fixes in the 1.8 branch
> 3) new features to land in 1.9
>
> This allows companies to
>
>
> Have a "known" version and can move to the new branch when they
>
> want
>
> to
>
> get new features. Obviously we only support N-1, so when 1.10
>
> comes
>
> out
>
> we
>
> stop supporting 1.8.X.
>
>
> Sent from my iPhone
>
> On 21 Mar 2017, at 11:22, Chris Riccomini <
>
> criccomini@apache.org>
>
> wrote:
>
>
> Hey all,
>
> I suggest that we start a 1.8.1 Airflow release now. The goal
>
> would
>
> be:
>
>
> 1) get a second release under our belt
> 2) patch known issues with the 1.8.0 release
>
> I'm happy to run it, but I saw Maxime mentioning that Airbnb
>
> might
>
> want
>
> to.
>
> @Max et al, can you comment?
>
> Also, can folks supply JIRAs for stuff that think needs to be
>
> in
>
> the
>
> 1.8.1
>
> bugfix release?
>
> Cheers,
> Chris
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Re: 1.8.1 release
Posted by Bolke de Bruin <bd...@gmail.com>.
I have a PR out for this (AIRFLOW-1033). https://github.com/apache/incubator-airflow/pull/2220
It seems large but that is due to activation of the ti_deps tests. See for the actual change: https://github.com/apache/incubator-airflow/pull/2220/commits/e16678103bf7502e98689cd32f3636cbb67e492f#diff-09c898c9cd54512de2fb3a05bae9724c <https://github.com/apache/incubator-airflow/pull/2220/commits/e16678103bf7502e98689cd32f3636cbb67e492f#diff-09c898c9cd54512de2fb3a05bae9724c>
Bolke
> On 5 Apr 2017, at 19:16, Chris Riccomini <cr...@apache.org> wrote:
>
> I've added it to 1.8.1 as a blocker.
>
> On Wed, Apr 5, 2017 at 10:11 AM, Ruslan Dautkhanov <da...@gmail.com>
> wrote:
>
>> https://issues.apache.org/jira/browse/AIRFLOW-1033
>>
>> Is also 1.8 and not 1.8.0 blocker ?
>>
>> Thanks.
>>
>>
>>
>> --
>> Ruslan Dautkhanov
>>
>> On Mon, Apr 3, 2017 at 3:20 PM, Bolke de Bruin <bd...@gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> Please add:
>>>
>>> https://issues.apache.org/jira/browse/AIRFLOW-1011 <
>>> https://issues.apache.org/jira/browse/AIRFLOW-1011> (PR:
>>> https://github.com/apache/incubator-airflow/pull/2179 <
>>> https://github.com/apache/incubator-airflow/pull/2179>)
>>>
>>> To the blocker list. It was missed due to the “Airflow 1.8” affected
>>> version tag instead of “1.8.0”. I fixed that for you.
>>>
>>> Bolke.
>>>
>>>> On 3 Apr 2017, at 22:06, Bolke de Bruin <bd...@gmail.com> wrote:
>>>>
>>>> I have a PR out for
>>>>
>>>> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213
>> <
>>> https://github.com/apache/incubator-airflow/pull/2213>
>>>> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212
>> <
>>> https://github.com/apache/incubator-airflow/pull/2212>
>>>>
>>>> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I
>>> don’t think it is a blocker: it happens only with @once dags that have a
>>> SLA, hardly very common. Nevertheless a fix would be nice obviously.
>>>>
>>>> Bolke
>>>>
>>>>> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <mailto:
>>> bdbruin@gmail.com>> wrote:
>>>>>
>>>>> Hey Chris,
>>>>>
>>>>> AIRFLOW-1000 has a PR out. You might want to discuss it on the list
>>> what the impact is though.
>>>>> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for
>>> the scheduler log files. I did downgrade from blocker to critical.
>>>>> AIRFLOW-719 Has a PR, including much needed increased test coverage,
>>> that I am pretty sure is working, but needs verification (plz @Alex)
>>>>>
>>>>> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
>>> around at the moment.
>>>>>
>>>>> Furthermore, do you have any timelines for getting to the release? Can
>>> I help in any way? You might want to chase a couple of times ;-).
>>>>>
>>>>> Bolke
>>>>>
>>>>>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org
>> <mailto:
>>> sanand@apache.org>> wrote:
>>>>>>
>>>>>> Chris,
>>>>>> I've submitted PRs for :
>>>>>>
>>>>>> - PR [AIRFLOW-1013] :
>>>>>> https://github.com/apache/incubator-airflow/pull/2203 <
>>> https://github.com/apache/incubator-airflow/pull/2203>
>>>>>> - PR [AIRFLOW-1054]:
>>>>>> https://github.com/apache/incubator-airflow/pull/2201 <
>>> https://github.com/apache/incubator-airflow/pull/2201>
>>>>>>
>>>>>> And filed a blocker for a new issue. Essentially, @once DAGs cannot
>> be
>>>>>> created if catchup=False :
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <
>>> https://issues.apache.org/jira/browse/AIRFLOW-1055>
>>>>>>
>>>>>> I have a PR that works for this, but will need to add unit tests for
>>> it as
>>>>>> well as for AIRFLOW-1013.
>>>>>>
>>>>>> -s
>>>>>>
>>>>>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org>
>>> wrote:
>>>>>>
>>>>>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013
>>> was a
>>>>>>> blocker. I will have a PR shortly.
>>>>>>> -s
>>>>>>>
>>>>>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <
>>> criccomini@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The following three JIRAs were not merged into the v1-8-test
>> branch,
>>> but
>>>>>>>> are listed as part of the 1.8.1 release:
>>>>>>>>
>>>>>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>>>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>>>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>>>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>>>>>
>>>>>>>> I'm going to merge them in now.
>>>>>>>>
>>>>>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <
>>> criccomini@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Bolke,
>>>>>>>>>
>>>>>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>>>>>>
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>>>>
>>>>>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>>>>>>
>>>>>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <
>> bdbruin@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Chris,
>>>>>>>>>>
>>>>>>>>>> I have a PR out for
>>>>>>>>>>
>>>>>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from
>> the
>>>>>>>>>> blockers list and just a new feature.
>>>>>>>>>>
>>>>>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>>>>>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>>>>>>>
>>>>>>>>>> Cc: @alexvanboxel
>>>>>>>>>>
>>>>>>>>>> Bolke
>>>>>>>>>>
>>>>>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <
>> criccomini@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hey all,
>>>>>>>>>>>
>>>>>>>>>>> I've let this thread sit for a while. Here are a list of the
>>> issues
>>>>>>>> that
>>>>>>>>>>> were raised:
>>>>>>>>>>>
>>>>>>>>>>> BLOCKERS:
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>>>>>>>>>>
>>>>>>>>>>> NICE TO HAVE:
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>>>
>>>>>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not
>> closed.
>>>>>>>>>>>
>>>>>>>>>>> The rest remain open. I will wait on the release until the
>>> remaining
>>>>>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
>>>>>>>>>>>
>>>>>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
>>>>>>>> patches,
>>>>>>>>>>> that's great, otherwise I don't believe they'll get fixed as
>> part
>>> of
>>>>>>>>>> 1.8.1.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Chris
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>>>>>>>>>> dautkhanov@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thank you Sid!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Ruslan
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>>>>>>>> sanand@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Ruslan,
>>>>>>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
>>>>>>>> should
>>>>>>>>>> aim
>>>>>>>>>>>> to
>>>>>>>>>>>>> get some of them into 1.8.1.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -s
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>>>>>>>>>> dautkhanov@gmail.com
>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It would be great to have at least some of them fixed in
>> 1.8.1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
>>>>>>>> dan.davydov@airbnb.com
>>>>>>>>>> .
>>>>>>>>>>>>>> invalid
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
>>>>>>>> general
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>> startup time from this new logic of orphaned/reset task)
>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I
>>> will
>>>>>>>>>>>>>> hopefully
>>>>>>>>>>>>>>> have a fix out for soon just finishing up tests)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that
>> we
>>>>>>>>>> weren't
>>>>>>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang
>>> (had
>>>>>>>> to
>>>>>>>>>>>> roll
>>>>>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it
>> soon
>>>>>>>> which
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>> be on the list too.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we
>>> want
>>>>>>>> in
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> bugfix release.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>>>>>>>>>>>> bdbruin@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <
>>> bdbruin@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1) no new features in the 1.8 branch
>>>>>>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
>>>>>>>>>>>>>>>>>> 3) new features to land in 1.9
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This allows companies to
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Have a "known" version and can move to the new branch when
>>> they
>>>>>>>>>>>>> want
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> get new features. Obviously we only support N-1, so when
>>> 1.10
>>>>>>>>>>>> comes
>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The
>>> goal
>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>> be:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) get a second release under our belt
>>>>>>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that
>>> Airbnb
>>>>>>>>>>>>> might
>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>>>>> @Max et al, can you comment?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs
>>> to be
>>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
>>
Re: 1.8.1 release
Posted by Chris Riccomini <cr...@apache.org>.
I've added it to 1.8.1 as a blocker.
On Wed, Apr 5, 2017 at 10:11 AM, Ruslan Dautkhanov <da...@gmail.com>
wrote:
> https://issues.apache.org/jira/browse/AIRFLOW-1033
>
> Is also 1.8 and not 1.8.0 blocker ?
>
> Thanks.
>
>
>
> --
> Ruslan Dautkhanov
>
> On Mon, Apr 3, 2017 at 3:20 PM, Bolke de Bruin <bd...@gmail.com> wrote:
>
> > Hi Chris,
> >
> > Please add:
> >
> > https://issues.apache.org/jira/browse/AIRFLOW-1011 <
> > https://issues.apache.org/jira/browse/AIRFLOW-1011> (PR:
> > https://github.com/apache/incubator-airflow/pull/2179 <
> > https://github.com/apache/incubator-airflow/pull/2179>)
> >
> > To the blocker list. It was missed due to the “Airflow 1.8” affected
> > version tag instead of “1.8.0”. I fixed that for you.
> >
> > Bolke.
> >
> > > On 3 Apr 2017, at 22:06, Bolke de Bruin <bd...@gmail.com> wrote:
> > >
> > > I have a PR out for
> > >
> > > * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213
> <
> > https://github.com/apache/incubator-airflow/pull/2213>
> > > * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212
> <
> > https://github.com/apache/incubator-airflow/pull/2212>
> > >
> > > Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I
> > don’t think it is a blocker: it happens only with @once dags that have a
> > SLA, hardly very common. Nevertheless a fix would be nice obviously.
> > >
> > > Bolke
> > >
> > >> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <mailto:
> > bdbruin@gmail.com>> wrote:
> > >>
> > >> Hey Chris,
> > >>
> > >> AIRFLOW-1000 has a PR out. You might want to discuss it on the list
> > what the impact is though.
> > >> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for
> > the scheduler log files. I did downgrade from blocker to critical.
> > >> AIRFLOW-719 Has a PR, including much needed increased test coverage,
> > that I am pretty sure is working, but needs verification (plz @Alex)
> > >>
> > >> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
> > around at the moment.
> > >>
> > >> Furthermore, do you have any timelines for getting to the release? Can
> > I help in any way? You might want to chase a couple of times ;-).
> > >>
> > >> Bolke
> > >>
> > >>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org
> <mailto:
> > sanand@apache.org>> wrote:
> > >>>
> > >>> Chris,
> > >>> I've submitted PRs for :
> > >>>
> > >>> - PR [AIRFLOW-1013] :
> > >>> https://github.com/apache/incubator-airflow/pull/2203 <
> > https://github.com/apache/incubator-airflow/pull/2203>
> > >>> - PR [AIRFLOW-1054]:
> > >>> https://github.com/apache/incubator-airflow/pull/2201 <
> > https://github.com/apache/incubator-airflow/pull/2201>
> > >>>
> > >>> And filed a blocker for a new issue. Essentially, @once DAGs cannot
> be
> > >>> created if catchup=False :
> > >>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <
> > https://issues.apache.org/jira/browse/AIRFLOW-1055>
> > >>>
> > >>> I have a PR that works for this, but will need to add unit tests for
> > it as
> > >>> well as for AIRFLOW-1013.
> > >>>
> > >>> -s
> > >>>
> > >>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org>
> > wrote:
> > >>>
> > >>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013
> > was a
> > >>>> blocker. I will have a PR shortly.
> > >>>> -s
> > >>>>
> > >>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <
> > criccomini@apache.org>
> > >>>> wrote:
> > >>>>
> > >>>>> The following three JIRAs were not merged into the v1-8-test
> branch,
> > but
> > >>>>> are listed as part of the 1.8.1 release:
> > >>>>>
> > >>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
> > >>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
> > >>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
> > >>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
> > >>>>>
> > >>>>> I'm going to merge them in now.
> > >>>>>
> > >>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <
> > criccomini@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hey Bolke,
> > >>>>>>
> > >>>>>> Great. Assuming your PR is committed, that leaves five blockers:
> > >>>>>>
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
> > >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
> > >>>>>>
> > >>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
> > >>>>>>
> > >>>>>> Cheers,
> > >>>>>> Chris
> > >>>>>>
> > >>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
> > >>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> > >>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
> > >>>>>>
> > >>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <
> bdbruin@gmail.com>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi Chris,
> > >>>>>>>
> > >>>>>>> I have a PR out for
> > >>>>>>>
> > >>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from
> the
> > >>>>>>> blockers list and just a new feature.
> > >>>>>>>
> > >>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
> > >>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
> > >>>>>>>
> > >>>>>>> Cc: @alexvanboxel
> > >>>>>>>
> > >>>>>>> Bolke
> > >>>>>>>
> > >>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <
> criccomini@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> Hey all,
> > >>>>>>>>
> > >>>>>>>> I've let this thread sit for a while. Here are a list of the
> > issues
> > >>>>> that
> > >>>>>>>> were raised:
> > >>>>>>>>
> > >>>>>>>> BLOCKERS:
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
> > >>>>>>>>
> > >>>>>>>> NICE TO HAVE:
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
> > >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
> > >>>>>>>>
> > >>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not
> closed.
> > >>>>>>>>
> > >>>>>>>> The rest remain open. I will wait on the release until the
> > remaining
> > >>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
> > >>>>>>>>
> > >>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
> > >>>>> patches,
> > >>>>>>>> that's great, otherwise I don't believe they'll get fixed as
> part
> > of
> > >>>>>>> 1.8.1.
> > >>>>>>>>
> > >>>>>>>> Cheers,
> > >>>>>>>> Chris
> > >>>>>>>>
> > >>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
> > >>>>>>> dautkhanov@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Thank you Sid!
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Best regards,
> > >>>>>>>>> Ruslan
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
> > >>>>> sanand@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Ruslan,
> > >>>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
> > >>>>> should
> > >>>>>>> aim
> > >>>>>>>>> to
> > >>>>>>>>>> get some of them into 1.8.1.
> > >>>>>>>>>>
> > >>>>>>>>>> -s
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
> > >>>>>>> dautkhanov@gmail.com
> > >>>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
> > >>>>>>>>>>>
> > >>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> It would be great to have at least some of them fixed in
> 1.8.1.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thank you.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Ruslan Dautkhanov
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
> > >>>>> dan.davydov@airbnb.com
> > >>>>>>> .
> > >>>>>>>>>>> invalid
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
> > >>>>> general
> > >>>>>>>>> the
> > >>>>>>>>>>>> slow
> > >>>>>>>>>>>> startup time from this new logic of orphaned/reset task)
> > >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I
> > will
> > >>>>>>>>>>> hopefully
> > >>>>>>>>>>>> have a fix out for soon just finishing up tests)
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that
> we
> > >>>>>>> weren't
> > >>>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang
> > (had
> > >>>>> to
> > >>>>>>>>> roll
> > >>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it
> soon
> > >>>>> which
> > >>>>>>>>>>> should
> > >>>>>>>>>>>> be on the list too.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
> > >>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we
> > want
> > >>>>> in
> > >>>>>>>>> the
> > >>>>>>>>>>>>> bugfix release.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
> > >>>>>>>>> bdbruin@gmail.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <
> > bdbruin@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 1) no new features in the 1.8 branch
> > >>>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
> > >>>>>>>>>>>>>>> 3) new features to land in 1.9
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> This allows companies to
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Have a "known" version and can move to the new branch when
> > they
> > >>>>>>>>>> want
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>>> get new features. Obviously we only support N-1, so when
> > 1.10
> > >>>>>>>>> comes
> > >>>>>>>>>>> out
> > >>>>>>>>>>>>> we
> > >>>>>>>>>>>>>> stop supporting 1.8.X.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Sent from my iPhone
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
> > >>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The
> > goal
> > >>>>>>>>>>> would
> > >>>>>>>>>>>>> be:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> 1) get a second release under our belt
> > >>>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that
> > Airbnb
> > >>>>>>>>>> might
> > >>>>>>>>>>>>> want
> > >>>>>>>>>>>>>> to.
> > >>>>>>>>>>>>>>>> @Max et al, can you comment?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs
> > to be
> > >>>>>>>>> in
> > >>>>>>>>>>> the
> > >>>>>>>>>>>>>> 1.8.1
> > >>>>>>>>>>>>>>>> bugfix release?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>
> > >
> >
> >
>
Re: 1.8.1 release
Posted by Ruslan Dautkhanov <da...@gmail.com>.
https://issues.apache.org/jira/browse/AIRFLOW-1033
Is also 1.8 and not 1.8.0 blocker ?
Thanks.
--
Ruslan Dautkhanov
On Mon, Apr 3, 2017 at 3:20 PM, Bolke de Bruin <bd...@gmail.com> wrote:
> Hi Chris,
>
> Please add:
>
> https://issues.apache.org/jira/browse/AIRFLOW-1011 <
> https://issues.apache.org/jira/browse/AIRFLOW-1011> (PR:
> https://github.com/apache/incubator-airflow/pull/2179 <
> https://github.com/apache/incubator-airflow/pull/2179>)
>
> To the blocker list. It was missed due to the “Airflow 1.8” affected
> version tag instead of “1.8.0”. I fixed that for you.
>
> Bolke.
>
> > On 3 Apr 2017, at 22:06, Bolke de Bruin <bd...@gmail.com> wrote:
> >
> > I have a PR out for
> >
> > * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 <
> https://github.com/apache/incubator-airflow/pull/2213>
> > * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 <
> https://github.com/apache/incubator-airflow/pull/2212>
> >
> > Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I
> don’t think it is a blocker: it happens only with @once dags that have a
> SLA, hardly very common. Nevertheless a fix would be nice obviously.
> >
> > Bolke
> >
> >> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <mailto:
> bdbruin@gmail.com>> wrote:
> >>
> >> Hey Chris,
> >>
> >> AIRFLOW-1000 has a PR out. You might want to discuss it on the list
> what the impact is though.
> >> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for
> the scheduler log files. I did downgrade from blocker to critical.
> >> AIRFLOW-719 Has a PR, including much needed increased test coverage,
> that I am pretty sure is working, but needs verification (plz @Alex)
> >>
> >> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not
> around at the moment.
> >>
> >> Furthermore, do you have any timelines for getting to the release? Can
> I help in any way? You might want to chase a couple of times ;-).
> >>
> >> Bolke
> >>
> >>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org <mailto:
> sanand@apache.org>> wrote:
> >>>
> >>> Chris,
> >>> I've submitted PRs for :
> >>>
> >>> - PR [AIRFLOW-1013] :
> >>> https://github.com/apache/incubator-airflow/pull/2203 <
> https://github.com/apache/incubator-airflow/pull/2203>
> >>> - PR [AIRFLOW-1054]:
> >>> https://github.com/apache/incubator-airflow/pull/2201 <
> https://github.com/apache/incubator-airflow/pull/2201>
> >>>
> >>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
> >>> created if catchup=False :
> >>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <
> https://issues.apache.org/jira/browse/AIRFLOW-1055>
> >>>
> >>> I have a PR that works for this, but will need to add unit tests for
> it as
> >>> well as for AIRFLOW-1013.
> >>>
> >>> -s
> >>>
> >>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org>
> wrote:
> >>>
> >>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013
> was a
> >>>> blocker. I will have a PR shortly.
> >>>> -s
> >>>>
> >>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <
> criccomini@apache.org>
> >>>> wrote:
> >>>>
> >>>>> The following three JIRAs were not merged into the v1-8-test branch,
> but
> >>>>> are listed as part of the 1.8.1 release:
> >>>>>
> >>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
> >>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
> >>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
> >>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
> >>>>>
> >>>>> I'm going to merge them in now.
> >>>>>
> >>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <
> criccomini@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> Hey Bolke,
> >>>>>>
> >>>>>> Great. Assuming your PR is committed, that leaves five blockers:
> >>>>>>
> >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
> >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
> >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
> >>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
> >>>>>>
> >>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Chris
> >>>>>>
> >>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
> >>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
> >>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
> >>>>>>
> >>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bd...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hi Chris,
> >>>>>>>
> >>>>>>> I have a PR out for
> >>>>>>>
> >>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
> >>>>>>> blockers list and just a new feature.
> >>>>>>>
> >>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
> >>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
> >>>>>>>
> >>>>>>> Cc: @alexvanboxel
> >>>>>>>
> >>>>>>> Bolke
> >>>>>>>
> >>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <cr...@apache.org>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hey all,
> >>>>>>>>
> >>>>>>>> I've let this thread sit for a while. Here are a list of the
> issues
> >>>>> that
> >>>>>>>> were raised:
> >>>>>>>>
> >>>>>>>> BLOCKERS:
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
> >>>>>>>>
> >>>>>>>> NICE TO HAVE:
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
> >>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
> >>>>>>>>
> >>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
> >>>>>>>>
> >>>>>>>> The rest remain open. I will wait on the release until the
> remaining
> >>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
> >>>>>>>>
> >>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
> >>>>> patches,
> >>>>>>>> that's great, otherwise I don't believe they'll get fixed as part
> of
> >>>>>>> 1.8.1.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Chris
> >>>>>>>>
> >>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
> >>>>>>> dautkhanov@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Thank you Sid!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Best regards,
> >>>>>>>>> Ruslan
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
> >>>>> sanand@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Ruslan,
> >>>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
> >>>>> should
> >>>>>>> aim
> >>>>>>>>> to
> >>>>>>>>>> get some of them into 1.8.1.
> >>>>>>>>>>
> >>>>>>>>>> -s
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
> >>>>>>> dautkhanov@gmail.com
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
> >>>>>>>>>>>
> >>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> It would be great to have at least some of them fixed in 1.8.1.
> >>>>>>>>>>>
> >>>>>>>>>>> Thank you.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Ruslan Dautkhanov
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
> >>>>> dan.davydov@airbnb.com
> >>>>>>> .
> >>>>>>>>>>> invalid
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
> >>>>> general
> >>>>>>>>> the
> >>>>>>>>>>>> slow
> >>>>>>>>>>>> startup time from this new logic of orphaned/reset task)
> >>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I
> will
> >>>>>>>>>>> hopefully
> >>>>>>>>>>>> have a fix out for soon just finishing up tests)
> >>>>>>>>>>>>
> >>>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that we
> >>>>>>> weren't
> >>>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang
> (had
> >>>>> to
> >>>>>>>>> roll
> >>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon
> >>>>> which
> >>>>>>>>>>> should
> >>>>>>>>>>>> be on the list too.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
> >>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we
> want
> >>>>> in
> >>>>>>>>> the
> >>>>>>>>>>>>> bugfix release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
> >>>>>>>>> bdbruin@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <
> bdbruin@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1) no new features in the 1.8 branch
> >>>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
> >>>>>>>>>>>>>>> 3) new features to land in 1.9
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> This allows companies to
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Have a "known" version and can move to the new branch when
> they
> >>>>>>>>>> want
> >>>>>>>>>>> to
> >>>>>>>>>>>>>> get new features. Obviously we only support N-1, so when
> 1.10
> >>>>>>>>> comes
> >>>>>>>>>>> out
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>>> stop supporting 1.8.X.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sent from my iPhone
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
> >>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The
> goal
> >>>>>>>>>>> would
> >>>>>>>>>>>>> be:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 1) get a second release under our belt
> >>>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that
> Airbnb
> >>>>>>>>>> might
> >>>>>>>>>>>>> want
> >>>>>>>>>>>>>> to.
> >>>>>>>>>>>>>>>> @Max et al, can you comment?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs
> to be
> >>>>>>>>> in
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> 1.8.1
> >>>>>>>>>>>>>>>> bugfix release?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>
> >
>
>
Re: 1.8.1 release
Posted by Bolke de Bruin <bd...@gmail.com>.
Hi Chris,
Please add:
https://issues.apache.org/jira/browse/AIRFLOW-1011 <https://issues.apache.org/jira/browse/AIRFLOW-1011> (PR: https://github.com/apache/incubator-airflow/pull/2179 <https://github.com/apache/incubator-airflow/pull/2179>)
To the blocker list. It was missed due to the “Airflow 1.8” affected version tag instead of “1.8.0”. I fixed that for you.
Bolke.
> On 3 Apr 2017, at 22:06, Bolke de Bruin <bd...@gmail.com> wrote:
>
> I have a PR out for
>
> * AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 <https://github.com/apache/incubator-airflow/pull/2213>
> * AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 <https://github.com/apache/incubator-airflow/pull/2212>
>
> Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t think it is a blocker: it happens only with @once dags that have a SLA, hardly very common. Nevertheless a fix would be nice obviously.
>
> Bolke
>
>> On 3 Apr 2017, at 11:05, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hey Chris,
>>
>> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the impact is though.
>> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the scheduler log files. I did downgrade from blocker to critical.
>> AIRFLOW-719 Has a PR, including much needed increased test coverage, that I am pretty sure is working, but needs verification (plz @Alex)
>>
>> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around at the moment.
>>
>> Furthermore, do you have any timelines for getting to the release? Can I help in any way? You might want to chase a couple of times ;-).
>>
>> Bolke
>>
>>> On 30 Mar 2017, at 19:48, siddharth anand <sanand@apache.org <ma...@apache.org>> wrote:
>>>
>>> Chris,
>>> I've submitted PRs for :
>>>
>>> - PR [AIRFLOW-1013] :
>>> https://github.com/apache/incubator-airflow/pull/2203 <https://github.com/apache/incubator-airflow/pull/2203>
>>> - PR [AIRFLOW-1054]:
>>> https://github.com/apache/incubator-airflow/pull/2201 <https://github.com/apache/incubator-airflow/pull/2201>
>>>
>>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
>>> created if catchup=False :
>>> https://issues.apache.org/jira/browse/AIRFLOW-1055 <https://issues.apache.org/jira/browse/AIRFLOW-1055>
>>>
>>> I have a PR that works for this, but will need to add unit tests for it as
>>> well as for AIRFLOW-1013.
>>>
>>> -s
>>>
>>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org> wrote:
>>>
>>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
>>>> blocker. I will have a PR shortly.
>>>> -s
>>>>
>>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <cr...@apache.org>
>>>> wrote:
>>>>
>>>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>>>> are listed as part of the 1.8.1 release:
>>>>>
>>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>>
>>>>> I'm going to merge them in now.
>>>>>
>>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <cr...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hey Bolke,
>>>>>>
>>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>
>>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>>>
>>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bd...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>>> Hi Chris,
>>>>>>>
>>>>>>> I have a PR out for
>>>>>>>
>>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>>>>>>> blockers list and just a new feature.
>>>>>>>
>>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>>>>
>>>>>>> Cc: @alexvanboxel
>>>>>>>
>>>>>>> Bolke
>>>>>>>
>>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <cr...@apache.org>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hey all,
>>>>>>>>
>>>>>>>> I've let this thread sit for a while. Here are a list of the issues
>>>>> that
>>>>>>>> were raised:
>>>>>>>>
>>>>>>>> BLOCKERS:
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>>>>>>>
>>>>>>>> NICE TO HAVE:
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>
>>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>>>>>>>>
>>>>>>>> The rest remain open. I will wait on the release until the remaining
>>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
>>>>>>>>
>>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
>>>>> patches,
>>>>>>>> that's great, otherwise I don't believe they'll get fixed as part of
>>>>>>> 1.8.1.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Chris
>>>>>>>>
>>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>>>>>>> dautkhanov@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thank you Sid!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Ruslan
>>>>>>>>>
>>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>>>>> sanand@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Ruslan,
>>>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
>>>>> should
>>>>>>> aim
>>>>>>>>> to
>>>>>>>>>> get some of them into 1.8.1.
>>>>>>>>>>
>>>>>>>>>> -s
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>>>>>>> dautkhanov@gmail.com
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
>>>>>>>>>>>
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It would be great to have at least some of them fixed in 1.8.1.
>>>>>>>>>>>
>>>>>>>>>>> Thank you.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
>>>>> dan.davydov@airbnb.com
>>>>>>> .
>>>>>>>>>>> invalid
>>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
>>>>> general
>>>>>>>>> the
>>>>>>>>>>>> slow
>>>>>>>>>>>> startup time from this new logic of orphaned/reset task)
>>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
>>>>>>>>>>> hopefully
>>>>>>>>>>>> have a fix out for soon just finishing up tests)
>>>>>>>>>>>>
>>>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that we
>>>>>>> weren't
>>>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang (had
>>>>> to
>>>>>>>>> roll
>>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon
>>>>> which
>>>>>>>>>>> should
>>>>>>>>>>>> be on the list too.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we want
>>>>> in
>>>>>>>>> the
>>>>>>>>>>>>> bugfix release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>>>>>>>>> bdbruin@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <bd...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) no new features in the 1.8 branch
>>>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
>>>>>>>>>>>>>>> 3) new features to land in 1.9
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This allows companies to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Have a "known" version and can move to the new branch when they
>>>>>>>>>> want
>>>>>>>>>>> to
>>>>>>>>>>>>>> get new features. Obviously we only support N-1, so when 1.10
>>>>>>>>> comes
>>>>>>>>>>> out
>>>>>>>>>>>>> we
>>>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The goal
>>>>>>>>>>> would
>>>>>>>>>>>>> be:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) get a second release under our belt
>>>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that Airbnb
>>>>>>>>>> might
>>>>>>>>>>>>> want
>>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>>> @Max et al, can you comment?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs to be
>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>
Re: 1.8.1 release
Posted by Bolke de Bruin <bd...@gmail.com>.
I have a PR out for
* AIRFLOW-1001: https://github.com/apache/incubator-airflow/pull/2213 <https://github.com/apache/incubator-airflow/pull/2213>
* AIRFLOW-1018: https://github.com/apache/incubator-airflow/pull/2212 <https://github.com/apache/incubator-airflow/pull/2212>
Furthermore, Sid, I checked AIRFLOW-1053 and while it is annoying I don’t think it is a blocker: it happens only with @once dags that have a SLA, hardly very common. Nevertheless a fix would be nice obviously.
Bolke
> On 3 Apr 2017, at 11:05, Bolke de Bruin <bd...@gmail.com> wrote:
>
> Hey Chris,
>
> AIRFLOW-1000 has a PR out. You might want to discuss it on the list what the impact is though.
> AIRFLOW-1018 I’ll have a PR out for shortly that allows “stdout” for the scheduler log files. I did downgrade from blocker to critical.
> AIRFLOW-719 Has a PR, including much needed increased test coverage, that I am pretty sure is working, but needs verification (plz @Alex)
>
> I would downgrade AIRFLOW-1019 to critical - especially as Dan is not around at the moment.
>
> Furthermore, do you have any timelines for getting to the release? Can I help in any way? You might want to chase a couple of times ;-).
>
> Bolke
>
>> On 30 Mar 2017, at 19:48, siddharth anand <sa...@apache.org> wrote:
>>
>> Chris,
>> I've submitted PRs for :
>>
>> - PR [AIRFLOW-1013] :
>> https://github.com/apache/incubator-airflow/pull/2203
>> - PR [AIRFLOW-1054]:
>> https://github.com/apache/incubator-airflow/pull/2201
>>
>> And filed a blocker for a new issue. Essentially, @once DAGs cannot be
>> created if catchup=False :
>> https://issues.apache.org/jira/browse/AIRFLOW-1055
>>
>> I have a PR that works for this, but will need to add unit tests for it as
>> well as for AIRFLOW-1013.
>>
>> -s
>>
>> On Wed, Mar 29, 2017 at 3:24 PM, siddharth anand <sa...@apache.org> wrote:
>>
>>> Didn't realize https://issues.apache.org/jira/browse/AIRFLOW-1013 was a
>>> blocker. I will have a PR shortly.
>>> -s
>>>
>>> On Wed, Mar 29, 2017 at 2:07 PM, Chris Riccomini <cr...@apache.org>
>>> wrote:
>>>
>>>> The following three JIRAs were not merged into the v1-8-test branch, but
>>>> are listed as part of the 1.8.1 release:
>>>>
>>>> AIRFLOW-1017 b2b9587cca9195229ab107394ad94b7702c70e37
>>>> AIRFLOW-906 bc47200711be4d2c0b36b772651dae4f5e01a204
>>>> AIRFLOW-858 94dc7fb0a6bb3c563d9df6566cd52a59bd0c4629
>>>> AIRFLOW-832 b0ae70d3a8e935dc9266b6853683ae5375a7390b
>>>>
>>>> I'm going to merge them in now.
>>>>
>>>> On Wed, Mar 29, 2017 at 1:53 PM, Chris Riccomini <cr...@apache.org>
>>>> wrote:
>>>>
>>>>> Hey Bolke,
>>>>>
>>>>> Great. Assuming your PR is committed, that leaves five blockers:
>>>>>
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1000
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1018
>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>
>>>>> I've also got a list of all open 1.8.1 JIRAs [1].
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>> [1] https://issues.apache.org/jira/issues/?jql=project%20%
>>>>> 3D%20AIRFLOW%20AND%20status%20in%20(Open%2C%20%22In%
>>>>> 20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%201.8.1
>>>>>
>>>>> On Mon, Mar 27, 2017 at 8:59 PM, Bolke de Bruin <bd...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> I have a PR out for
>>>>>>
>>>>>> * Revert of 719, which makes 982 obsolete and removes 983 from the
>>>>>> blockers list and just a new feature.
>>>>>>
>>>>>> See: https://github.com/apache/incubator-airflow/pull/2195 <
>>>>>> https://github.com/apache/incubator-airflow/pull/2195>
>>>>>>
>>>>>> Cc: @alexvanboxel
>>>>>>
>>>>>> Bolke
>>>>>>
>>>>>>> On 24 Mar 2017, at 10:21, Chris Riccomini <cr...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> Hey all,
>>>>>>>
>>>>>>> I've let this thread sit for a while. Here are a list of the issues
>>>> that
>>>>>>> were raised:
>>>>>>>
>>>>>>> BLOCKERS:
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017
>>>>>>>
>>>>>>> NICE TO HAVE:
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>
>>>>>>> It looks like AIRFLOW-1017 is done, though the JIRA is not closed.
>>>>>>>
>>>>>>> The rest remain open. I will wait on the release until the remaining
>>>>>>> blockers are finished. Dan/Daniel, can you comment on status?
>>>>>>>
>>>>>>> Ruslan, if you want to work on your nice to haves, and submit
>>>> patches,
>>>>>>> that's great, otherwise I don't believe they'll get fixed as part of
>>>>>> 1.8.1.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Chris
>>>>>>>
>>>>>>> On Wed, Mar 22, 2017 at 9:19 AM, Ruslan Dautkhanov <
>>>>>> dautkhanov@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thank you Sid!
>>>>>>>>
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Ruslan
>>>>>>>>
>>>>>>>> On Wed, Mar 22, 2017 at 12:01 AM, siddharth anand <
>>>> sanand@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Ruslan,
>>>>>>>>> Thanks for sharing this list. I can pick a few up. I agree we
>>>> should
>>>>>> aim
>>>>>>>> to
>>>>>>>>> get some of them into 1.8.1.
>>>>>>>>>
>>>>>>>>> -s
>>>>>>>>>
>>>>>>>>> On Tue, Mar 21, 2017 at 2:29 PM, Ruslan Dautkhanov <
>>>>>> dautkhanov@gmail.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Some of the issues I ran into while testing 1.8rc5 :
>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1013
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1004
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1003
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1001
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1015
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It would be great to have at least some of them fixed in 1.8.1.
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ruslan Dautkhanov
>>>>>>>>>>
>>>>>>>>>> On Tue, Mar 21, 2017 at 3:02 PM, Dan Davydov <
>>>> dan.davydov@airbnb.com
>>>>>> .
>>>>>>>>>> invalid
>>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Here is my list for targeted 1.8.1 fixes:
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-982
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-983
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1019 (and in
>>>> general
>>>>>>>> the
>>>>>>>>>>> slow
>>>>>>>>>>> startup time from this new logic of orphaned/reset task)
>>>>>>>>>>> https://issues.apache.org/jira/browse/AIRFLOW-1017 (which I will
>>>>>>>>>> hopefully
>>>>>>>>>>> have a fix out for soon just finishing up tests)
>>>>>>>>>>>
>>>>>>>>>>> We are also hitting a new issue with subdags with rc5 that we
>>>>>> weren't
>>>>>>>>>>> hitting with rc4 where subdags will occasionally just hang (had
>>>> to
>>>>>>>> roll
>>>>>>>>>>> back from rc5 to rc4), I'll try to spin up a JIRA for it soon
>>>> which
>>>>>>>>>> should
>>>>>>>>>>> be on the list too.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Mar 21, 2017 at 1:54 PM, Chris Riccomini <
>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Agreed. I'm looking for a list of checksums/JIRAs that we want
>>>> in
>>>>>>>> the
>>>>>>>>>>>> bugfix release.
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Mar 21, 2017 at 12:54 PM, Bolke de Bruin <
>>>>>>>> bdbruin@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 21 Mar 2017, at 12:51, Bolke de Bruin <bd...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My suggestion, as we are using semantic versioning is:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) no new features in the 1.8 branch
>>>>>>>>>>>>>> 2) only bug fixes in the 1.8 branch
>>>>>>>>>>>>>> 3) new features to land in 1.9
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This allows companies to
>>>>>>>>>>>>>
>>>>>>>>>>>>> Have a "known" version and can move to the new branch when they
>>>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>>>>> get new features. Obviously we only support N-1, so when 1.10
>>>>>>>> comes
>>>>>>>>>> out
>>>>>>>>>>>> we
>>>>>>>>>>>>> stop supporting 1.8.X.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 21 Mar 2017, at 11:22, Chris Riccomini <
>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I suggest that we start a 1.8.1 Airflow release now. The goal
>>>>>>>>>> would
>>>>>>>>>>>> be:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) get a second release under our belt
>>>>>>>>>>>>>>> 2) patch known issues with the 1.8.0 release
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm happy to run it, but I saw Maxime mentioning that Airbnb
>>>>>>>>> might
>>>>>>>>>>>> want
>>>>>>>>>>>>> to.
>>>>>>>>>>>>>>> @Max et al, can you comment?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, can folks supply JIRAs for stuff that think needs to be
>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>> 1.8.1
>>>>>>>>>>>>>>> bugfix release?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>