You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Bolke de Bruin <bd...@gmail.com> on 2017/02/09 15:26:15 UTC
[VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc2
Dear All,
I have made the SECOND RELEASE CANDIDATE of Airflow 1.8.0 available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/ <https://dist.apache.org/repos/dist/dev/incubator/airflow/> , public keys are available at https://dist.apache.org/repos/dist/release/incubator/airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/> . It is tagged with a local version “apache.incubating” so it allows upgrading from earlier releases.
Two issues have been fixed:
* Cgroups issue
* kwargs issue in PrestoOperator
As these issues are confined to a very specific area I would also like to raise a VOTE for releasing 1.8.0 based on release candidate 2, i.e. just renaming release candidate 2 to 1.8.0 release. This is taking into account that Alex has an open issue, but it cannot be confirmed by others.
Please respond to this email by:
+1,0,-1 with *binding* if you are a PMC member or *non-binding* if you are not.
Thanks!
Bolke
My VOTE: +1 (binding)
Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc2
Posted by Bolke de Bruin <bd...@gmail.com>.
Thanks Jeremiah, that is indeed the right location. I ended up fixing the issue differently, by cutting of the fractional seconds for an execution date as that might have operational consequences elsewhere I cannot oversee and is in line with earlier releases.
I have fixed the issue with trigger_dag in PR: https://github.com/apache/incubator-airflow/pull/2064 <https://github.com/apache/incubator-airflow/pull/2064> , please have a look and +1.
@Alex were you able to dive into your issue? I prefer to have it solved and we have a time window now. However, I would still like to have some more details to help you debugging.
Bolke
> On 10 Feb 2017, at 01:09, Jeremiah Lowin <jl...@apache.org> wrote:
>
> Bolke, I think you might be looking for
> https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L376 <https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L376>
>
> I am traveling and will not be able to deploy rc2 for testing before next
> week so I don't feel entirely comfortable offering a vote now, but (for
> what it's worth) I would have voted +1 based on my experience with rc1.
>
> On Thu, Feb 9, 2017 at 3:42 PM Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>> wrote:
>
>> You can trigger them (pun intended :) ) by issuing
>>
>> airflow trigger_dag -e 20160101T01:01:00 test_dag
>>
>> I have no clue why the first issue was never caught, but writing tests now
>> for it.
>>
>> If someone knows where the log directory for tasks is determined (the
>> dynamic part), I would be happy, cause I have trouble finding it…
>>
>>
>>> On 9 Feb 2017, at 22:27, Chris Riccomini <cr...@apache.org> wrote:
>>>
>>> :( Thanks for the update. I'm running RC2 right now, and not hitting the
>>> above issues (didn't check fractional execution dates).
>>>
>>> On Thu, Feb 9, 2017 at 1:25 PM, Bolke de Bruin <bdbruin@gmail.com
>> <mailto:bdbruin@gmail.com <ma...@gmail.com>>> wrote:
>>>
>>>> Replying to myself, changing to -1 (binding).
>>>>
>>>> Found three issues:
>>>> * in cli trigger_dag and the local client. Execution date will never be
>>>> set.
>>>> * Issue with log directories (when will we fix logging?): execution_date
>>>> can have fractions
>>>> * maybe related to the above, but a triggered_dag will not get its tasks
>>>> scheduled.
>>>>
>>>> Sorry it seems we will need an RC3, I consider the above to be blockers.
>>>>
>>>> - Bolke
>>>>
>>>>> On 9 Feb 2017, at 16:26, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>> wrote:
>>>>>
>>>>> Dear All,
>>>>>
>>>>> I have made the SECOND RELEASE CANDIDATE of Airflow 1.8.0 available at:
>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <https://dist.apache.org/repos/dist/dev/incubator/airflow/> <
>>>> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <https://dist.apache.org/repos/dist/dev/incubator/airflow/> <
>> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <https://dist.apache.org/repos/dist/dev/incubator/airflow/>>> , public keys
>>>> are available at https://dist.apache.org/repos/dist/release/incubator/ <https://dist.apache.org/repos/dist/release/incubator/>
>> <https://dist.apache.org/repos/dist/release/incubator/ <https://dist.apache.org/repos/dist/release/incubator/>>
>>>> airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/>
>> <https://dist.apache.org/repos/dist/release/incubator/airflow/>>
>>>> . It is tagged with a local version “apache.incubating” so it allows
>>>> upgrading from earlier releases.
>>>>>
>>>>> Two issues have been fixed:
>>>>>
>>>>> * Cgroups issue
>>>>> * kwargs issue in PrestoOperator
>>>>>
>>>>> As these issues are confined to a very specific area I would also like
>>>> to raise a VOTE for releasing 1.8.0 based on release candidate 2, i.e.
>> just
>>>> renaming release candidate 2 to 1.8.0 release. This is taking into
>> account
>>>> that Alex has an open issue, but it cannot be confirmed by others.
>>>>>
>>>>> Please respond to this email by:
>>>>>
>>>>> +1,0,-1 with *binding* if you are a PMC member or *non-binding* if you
>>>> are not.
>>>>>
>>>>> Thanks!
>>>>> Bolke
>>>>>
>>>>> My VOTE: +1 (binding)
Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc2
Posted by Jeremiah Lowin <jl...@apache.org>.
Bolke, I think you might be looking for
https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L376
I am traveling and will not be able to deploy rc2 for testing before next
week so I don't feel entirely comfortable offering a vote now, but (for
what it's worth) I would have voted +1 based on my experience with rc1.
On Thu, Feb 9, 2017 at 3:42 PM Bolke de Bruin <bd...@gmail.com> wrote:
> You can trigger them (pun intended :) ) by issuing
>
> airflow trigger_dag -e 20160101T01:01:00 test_dag
>
> I have no clue why the first issue was never caught, but writing tests now
> for it.
>
> If someone knows where the log directory for tasks is determined (the
> dynamic part), I would be happy, cause I have trouble finding it…
>
>
> > On 9 Feb 2017, at 22:27, Chris Riccomini <cr...@apache.org> wrote:
> >
> > :( Thanks for the update. I'm running RC2 right now, and not hitting the
> > above issues (didn't check fractional execution dates).
> >
> > On Thu, Feb 9, 2017 at 1:25 PM, Bolke de Bruin <bdbruin@gmail.com
> <ma...@gmail.com>> wrote:
> >
> >> Replying to myself, changing to -1 (binding).
> >>
> >> Found three issues:
> >> * in cli trigger_dag and the local client. Execution date will never be
> >> set.
> >> * Issue with log directories (when will we fix logging?): execution_date
> >> can have fractions
> >> * maybe related to the above, but a triggered_dag will not get its tasks
> >> scheduled.
> >>
> >> Sorry it seems we will need an RC3, I consider the above to be blockers.
> >>
> >> - Bolke
> >>
> >>> On 9 Feb 2017, at 16:26, Bolke de Bruin <bd...@gmail.com> wrote:
> >>>
> >>> Dear All,
> >>>
> >>> I have made the SECOND RELEASE CANDIDATE of Airflow 1.8.0 available at:
> >> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <
> >> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <
> https://dist.apache.org/repos/dist/dev/incubator/airflow/>> , public keys
> >> are available at https://dist.apache.org/repos/dist/release/incubator/
> <https://dist.apache.org/repos/dist/release/incubator/>
> >> airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/
> <https://dist.apache.org/repos/dist/release/incubator/airflow/>>
> >> . It is tagged with a local version “apache.incubating” so it allows
> >> upgrading from earlier releases.
> >>>
> >>> Two issues have been fixed:
> >>>
> >>> * Cgroups issue
> >>> * kwargs issue in PrestoOperator
> >>>
> >>> As these issues are confined to a very specific area I would also like
> >> to raise a VOTE for releasing 1.8.0 based on release candidate 2, i.e.
> just
> >> renaming release candidate 2 to 1.8.0 release. This is taking into
> account
> >> that Alex has an open issue, but it cannot be confirmed by others.
> >>>
> >>> Please respond to this email by:
> >>>
> >>> +1,0,-1 with *binding* if you are a PMC member or *non-binding* if you
> >> are not.
> >>>
> >>> Thanks!
> >>> Bolke
> >>>
> >>> My VOTE: +1 (binding)
>
>
Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc2
Posted by Bolke de Bruin <bd...@gmail.com>.
You can trigger them (pun intended :) ) by issuing
airflow trigger_dag -e 20160101T01:01:00 test_dag
I have no clue why the first issue was never caught, but writing tests now for it.
If someone knows where the log directory for tasks is determined (the dynamic part), I would be happy, cause I have trouble finding it…
> On 9 Feb 2017, at 22:27, Chris Riccomini <cr...@apache.org> wrote:
>
> :( Thanks for the update. I'm running RC2 right now, and not hitting the
> above issues (didn't check fractional execution dates).
>
> On Thu, Feb 9, 2017 at 1:25 PM, Bolke de Bruin <bdbruin@gmail.com <ma...@gmail.com>> wrote:
>
>> Replying to myself, changing to -1 (binding).
>>
>> Found three issues:
>> * in cli trigger_dag and the local client. Execution date will never be
>> set.
>> * Issue with log directories (when will we fix logging?): execution_date
>> can have fractions
>> * maybe related to the above, but a triggered_dag will not get its tasks
>> scheduled.
>>
>> Sorry it seems we will need an RC3, I consider the above to be blockers.
>>
>> - Bolke
>>
>>> On 9 Feb 2017, at 16:26, Bolke de Bruin <bd...@gmail.com> wrote:
>>>
>>> Dear All,
>>>
>>> I have made the SECOND RELEASE CANDIDATE of Airflow 1.8.0 available at:
>> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <
>> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <https://dist.apache.org/repos/dist/dev/incubator/airflow/>> , public keys
>> are available at https://dist.apache.org/repos/dist/release/incubator/ <https://dist.apache.org/repos/dist/release/incubator/>
>> airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/>>
>> . It is tagged with a local version “apache.incubating” so it allows
>> upgrading from earlier releases.
>>>
>>> Two issues have been fixed:
>>>
>>> * Cgroups issue
>>> * kwargs issue in PrestoOperator
>>>
>>> As these issues are confined to a very specific area I would also like
>> to raise a VOTE for releasing 1.8.0 based on release candidate 2, i.e. just
>> renaming release candidate 2 to 1.8.0 release. This is taking into account
>> that Alex has an open issue, but it cannot be confirmed by others.
>>>
>>> Please respond to this email by:
>>>
>>> +1,0,-1 with *binding* if you are a PMC member or *non-binding* if you
>> are not.
>>>
>>> Thanks!
>>> Bolke
>>>
>>> My VOTE: +1 (binding)
Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc2
Posted by Chris Riccomini <cr...@apache.org>.
:( Thanks for the update. I'm running RC2 right now, and not hitting the
above issues (didn't check fractional execution dates).
On Thu, Feb 9, 2017 at 1:25 PM, Bolke de Bruin <bd...@gmail.com> wrote:
> Replying to myself, changing to -1 (binding).
>
> Found three issues:
> * in cli trigger_dag and the local client. Execution date will never be
> set.
> * Issue with log directories (when will we fix logging?): execution_date
> can have fractions
> * maybe related to the above, but a triggered_dag will not get its tasks
> scheduled.
>
> Sorry it seems we will need an RC3, I consider the above to be blockers.
>
> - Bolke
>
> > On 9 Feb 2017, at 16:26, Bolke de Bruin <bd...@gmail.com> wrote:
> >
> > Dear All,
> >
> > I have made the SECOND RELEASE CANDIDATE of Airflow 1.8.0 available at:
> https://dist.apache.org/repos/dist/dev/incubator/airflow/ <
> https://dist.apache.org/repos/dist/dev/incubator/airflow/> , public keys
> are available at https://dist.apache.org/repos/dist/release/incubator/
> airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/>
> . It is tagged with a local version “apache.incubating” so it allows
> upgrading from earlier releases.
> >
> > Two issues have been fixed:
> >
> > * Cgroups issue
> > * kwargs issue in PrestoOperator
> >
> > As these issues are confined to a very specific area I would also like
> to raise a VOTE for releasing 1.8.0 based on release candidate 2, i.e. just
> renaming release candidate 2 to 1.8.0 release. This is taking into account
> that Alex has an open issue, but it cannot be confirmed by others.
> >
> > Please respond to this email by:
> >
> > +1,0,-1 with *binding* if you are a PMC member or *non-binding* if you
> are not.
> >
> > Thanks!
> > Bolke
> >
> > My VOTE: +1 (binding)
> >
> >
>
>
Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc2
Posted by Bolke de Bruin <bd...@gmail.com>.
Replying to myself, changing to -1 (binding).
Found three issues:
* in cli trigger_dag and the local client. Execution date will never be set.
* Issue with log directories (when will we fix logging?): execution_date can have fractions
* maybe related to the above, but a triggered_dag will not get its tasks scheduled.
Sorry it seems we will need an RC3, I consider the above to be blockers.
- Bolke
> On 9 Feb 2017, at 16:26, Bolke de Bruin <bd...@gmail.com> wrote:
>
> Dear All,
>
> I have made the SECOND RELEASE CANDIDATE of Airflow 1.8.0 available at: https://dist.apache.org/repos/dist/dev/incubator/airflow/ <https://dist.apache.org/repos/dist/dev/incubator/airflow/> , public keys are available at https://dist.apache.org/repos/dist/release/incubator/airflow/ <https://dist.apache.org/repos/dist/release/incubator/airflow/> . It is tagged with a local version “apache.incubating” so it allows upgrading from earlier releases.
>
> Two issues have been fixed:
>
> * Cgroups issue
> * kwargs issue in PrestoOperator
>
> As these issues are confined to a very specific area I would also like to raise a VOTE for releasing 1.8.0 based on release candidate 2, i.e. just renaming release candidate 2 to 1.8.0 release. This is taking into account that Alex has an open issue, but it cannot be confirmed by others.
>
> Please respond to this email by:
>
> +1,0,-1 with *binding* if you are a PMC member or *non-binding* if you are not.
>
> Thanks!
> Bolke
>
> My VOTE: +1 (binding)
>
>