You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Bolke de Bruin <bd...@gmail.com> on 2017/01/04 16:20:17 UTC
Airflow 1.8.0 alpha 2
Hi All,
I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~bolke/ <https://people.apache.org/~bolke/> .
Note: This still cannot be considered an Apache release. Working on this.
This build is signed (note it is served over https).
Changes are in the area of scheduler stability.
- Bolke
Re: Airflow 1.8.0 alpha 2
Posted by Jeremiah Lowin <jl...@gmail.com>.
Bolke -- I will see if I can solve it
On Fri, Jan 6, 2017 at 3:46 PM, Bolke de Bruin <bd...@gmail.com> wrote:
> For the XCOM issue: https://issues.apache.org/jira/browse/AIRFLOW-738 <
> https://issues.apache.org/jira/browse/AIRFLOW-738>
>
> With example dag that exposes the issue. Would be nice if someone with
> XCOM exposure could take a look.
>
> - Bolke
>
> > On 6 Jan 2017, at 15:31, Bolke de Bruin <bd...@gmail.com> wrote:
> >
> > State of to be Alpha 3:
> >
> > Blockers:
> >
> > * XCom throws an duplicate / locking error (new - confirmed)
> > * one_failed task not being run: now seems to pass suddenly (so fixed?)
> -> need to investigate why
> >
> > Fixed issues
> > * Regression in email
> > * LDAP case sensitivity
> >
> > Pending features:
> > * DAG.catchup : minor changes needed, documentation still required,
> integration tests seem to pass flawlessly
> > * Cgroups + impersonation: clean up of patches on going, more tests and
> more elaborate documentation required. Integration tests not executed yet
> > * Schedule all pending DAG runs in a single scheduler loop: no progress
> (**)
> > * Email attachments
> > * Add execution_date to trigger_dag
> >
> > If the pending features are merged within a reasonable time frame
> (except for **, as no progress currently) then I am planning to mark the
> tarball as Beta and only allow bug fixes and (very) minor features.
> Hopefully end of next week.
> >
> > Bolke.
> >
> >
> >
> >> On 5 Jan 2017, at 20:07, Chris Riccomini <cr...@apache.org> wrote:
> >>
> >> I have merged Robin's LDAP patch.
> >>
> >> On Thu, Jan 5, 2017 at 10:39 AM, Chris Riccomini <criccomini@apache.org
> >
> >> wrote:
> >>
> >>> I have found the email problem:
> >>>
> >>> https://issues.apache.org/jira/browse/AIRFLOW-734
> >>>
> >>> Working on a fix.
> >>>
> >>> On Thu, Jan 5, 2017 at 9:10 AM, Chris Riccomini <criccomini@apache.org
> >
> >>> wrote:
> >>>
> >>>> Also, I'm seeing a second issue:
> >>>>
> >>>> SMTP doesn't seem to work for us anymore:
> >>>>
> >>>> [2017-01-05 15:10:13,666] {models.py:1378} ERROR - SMTP AUTH
> extension not supported by server.
> >>>> Traceback (most recent call last):
> >>>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line
> 1374, in handle_failure
> >>>> self.email_alert(error, is_retry=False)
> >>>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line
> 1521, in email_alert
> >>>> send_email(task.email, title, body)
> >>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line
> 43, in send_email
> >>>> return backend(to, subject, html_content, files=files,
> dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
> >>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line
> 84, in send_email_smtp
> >>>> send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
> >>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line
> 100, in send_MIME_email
> >>>> s.login(SMTP_USER, SMTP_PASSWORD)
> >>>> File "/usr/lib64/python2.7/smtplib.py", line 584, in login
> >>>> raise SMTPException("SMTP AUTH extension not supported by server.")
> >>>> SMTPException: SMTP AUTH extension not supported by server.
> >>>>
> >>>> This was working on 1.7.1.2. Having a look now.
> >>>>
> >>>> On Thu, Jan 5, 2017 at 9:09 AM, Chris Riccomini <
> criccomini@apache.org>
> >>>> wrote:
> >>>>
> >>>>> Hey Robin,
> >>>>>
> >>>>> Awesome, thanks! I love open source. :) Let me try your patch out and
> >>>>> merge it.
> >>>>>
> >>>>> Cheers,
> >>>>> Chris
> >>>>>
> >>>>> On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <
> >>>>> Robin.Miller@affiliate.oliverwyman.com> wrote:
> >>>>>
> >>>>>> Hi Chris,
> >>>>>>
> >>>>>>
> >>>>>> I think I ran into this issue when setting up LDAP Auth in our
> >>>>>> environment (we're using very close to master as we needed some of
> the
> >>>>>> newer features/bugfixes). The problem turned out to be that the
> search was
> >>>>>> finding no results, so the line:
> >>>>>>
> >>>>>>
> >>>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
> >>>>>>
> >>>>>>
> >>>>>> would fail because it had no matching groups to return. This turned
> out
> >>>>>> to be because Windows Active Directory (the LDAP server we're using)
> >>>>>> returned capitals "CN=" where the code expected lowercase: regex =
> >>>>>> re.compile("cn=([^,]*).*")
> >>>>>>
> >>>>>>
> >>>>>> I haven't looked it up, but Windows Active Directory is case
> >>>>>> insensitive when it comes to usernames and groups, so I wouldn't be
> >>>>>> surprised if the protocol itself is case insensitive and both "cn="
> and
> >>>>>> "CN=" should be considered valid. As such I've a PR open for a
> simple fix
> >>>>>> to make this regex case insensitive: https://github.com/apache/incu
> >>>>>> bator-airflow/pull/1945
> >>>>>>
> >>>>>>
> >>>>>> Hopefully this helps,
> >>>>>>
> >>>>>> Robin Miller
> >>>>>> OLIVER WYMAN
> >>>>>> robin.miller@affiliate.oliverwyman.com<mailto:robin.miller@a
> >>>>>> ffiliate.oliverwyman.com>
> >>>>>> www.oliverwyman.com<http://www.oliverwyman.com/>
> >>>>>>
> >>>>>> ________________________________
> >>>>>> From: Chris Riccomini <cr...@apache.org>
> >>>>>> Sent: 05 January 2017 00:34:27
> >>>>>> To: dev@airflow.incubator.apache.org
> >>>>>> Subject: Re: Airflow 1.8.0 alpha 2
> >>>>>>
> >>>>>> I am now running 1.8.0a2 in our dev environment. It seems to be
> >>>>>> functioning
> >>>>>> well.
> >>>>>>
> >>>>>> One issue we've hit is that the LDAP auth plugin isn't working for
> us
> >>>>>> anymore:
> >>>>>>
> >>>>>> Traceback (most recent call last):
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988,
> in
> >>>>>> wsgi_app
> >>>>>> response = self.full_dispatch_request()
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641,
> in
> >>>>>> full_dispatch_request
> >>>>>> rv = self.handle_user_exception(e)
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544,
> in
> >>>>>> handle_user_exception
> >>>>>> reraise(exc_type, exc_value, tb)
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639,
> in
> >>>>>> full_dispatch_request
> >>>>>> rv = self.dispatch_request()
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625,
> in
> >>>>>> dispatch_request
> >>>>>> return self.view_functions[rule.endpoint](**req.view_args)
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
> >>>>>> 69,
> >>>>>> in inner
> >>>>>> return self._run_view(f, *args, **kwargs)
> >>>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
> >>>>>> 368,
> >>>>>> in _run_view
> >>>>>> return fn(self, *args, **kwargs)
> >>>>>> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line
> >>>>>> 657,
> >>>>>> in login
> >>>>>> return airflow.login.login(self, request)
> >>>>>> File
> >>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
> >>>>>> nds/ldap_auth.py",
> >>>>>> line 276, in login
> >>>>>> flask_login.login_user(LdapUser(user))
> >>>>>> File "<string>", line 4, in __init__
> >>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
> >>>>>> line
> >>>>>> 306, in _initialize_instance
> >>>>>> manager.dispatch.init_failure(self, args, kwargs)
> >>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelp
> >>>>>> ers.py",
> >>>>>> line 60, in __exit__
> >>>>>> compat.reraise(exc_type, exc_value, exc_tb)
> >>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
> >>>>>> line
> >>>>>> 303, in _initialize_instance
> >>>>>> return manager.original_init(*mixed[1:], **kwargs)
> >>>>>> File
> >>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
> >>>>>> nds/ldap_auth.py",
> >>>>>> line 148, in __init__
> >>>>>> user.username)
> >>>>>> File
> >>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
> >>>>>> nds/ldap_auth.py",
> >>>>>> line 106, in groups_user
> >>>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
> >>>>>> AttributeError: 'NoneType' object has no attribute 'group'
> >>>>>>
> >>>>>> I believe it's from this patch:
> >>>>>>
> >>>>>> https://github.com/apache/incubator-airflow/commit/d6d3f5367
> >>>>>> 3ba3736d7a858531823933cfef2bb4e
> >>>>>>
> >>>>>> I haven't dug into it yet. We'll need to fix it, though. In the
> >>>>>> meantime, I
> >>>>>> just commented out the call to `groups_user`. More to come.
> >>>>>>
> >>>>>> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> I have another fix that certainly need to be in the final release,
> >>>>>> but not
> >>>>>>> ready to merge due to failed tests:
> >>>>>>>
> >>>>>>> https://github.com/apache/incubator-airflow/pull/1961
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <
> criccomini@apache.org
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Great, I will work toward deploying this in our dev cluster today.
> >>>>>> :D
> >>>>>>>>
> >>>>>>>> On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <
> bdbruin@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Some issues remain:
> >>>>>>>>>
> >>>>>>>>> * one_failed not executed as dag run is marked failed seemingly
> >>>>>>>>> prematurely (@chris yes you should see this, see below for an
> >>>>>> example
> >>>>>>>> that
> >>>>>>>>> is not working properly), confirmed regression
> >>>>>>>>> * celery instability Alex
> >>>>>>>>> * Wrong DAG state after failure inside branch
> >>>>>>>>>
> >>>>>>>>> So Alpha 2 is definitely not ready for production, but please do
> >>>>>> put in
> >>>>>>>>> your canary dags and let them run. I am still quite concerned
> >>>>>> about the
> >>>>>>>>> scheduler integrity and stability.
> >>>>>>>>>
> >>>>>>>>> - Bolke
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> one_failed_not_executed.py
> >>>>>>>>> ====
> >>>>>>>>> from airflow import DAG
> >>>>>>>>> from airflow.operators.bash_operator import BashOperator
> >>>>>>>>> from airflow.operators.dummy_operator import DummyOperator
> >>>>>>>>> from datetime import datetime, timedelta
> >>>>>>>>>
> >>>>>>>>> default_args = {
> >>>>>>>>> 'owner': 'airflow',
> >>>>>>>>> 'depends_on_past': False,
> >>>>>>>>> 'start_date': datetime(2016,10,5,19),
> >>>>>>>>> 'email': ['airflow@airflow.com'],
> >>>>>>>>> 'email_on_failure': False,
> >>>>>>>>> 'email_on_retry': False,
> >>>>>>>>> 'retries': 1,
> >>>>>>>>> 'retry_delay': timedelta(seconds=1),
> >>>>>>>>> }
> >>>>>>>>>
> >>>>>>>>> dag = DAG('tutorial', default_args=default_args,
> >>>>>>>> schedule_interval='@once')
> >>>>>>>>>
> >>>>>>>>> task1 = BashOperator(
> >>>>>>>>> task_id='first_one',
> >>>>>>>>> bash_command='date',
> >>>>>>>>> dag=dag)
> >>>>>>>>>
> >>>>>>>>> task2 = BashOperator(
> >>>>>>>>> task_id='second_one',
> >>>>>>>>> bash_command='this_should_not_work',
> >>>>>>>>> dag=dag)
> >>>>>>>>>
> >>>>>>>>> task2.set_upstream(task1)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> task3 = BashOperator(
> >>>>>>>>> task_id='third_one',
> >>>>>>>>> bash_command='random_command_third',
> >>>>>>>>> dag=dag)
> >>>>>>>>>
> >>>>>>>>> task3.set_upstream(task2)
> >>>>>>>>>
> >>>>>>>>> fail_task = DummyOperator(
> >>>>>>>>> task_id='one_failed',
> >>>>>>>>> trigger_rule='one_failed',
> >>>>>>>>> dag=dag)
> >>>>>>>>>
> >>>>>>>>> fail_task.set_upstream([task1,task2,task3])
> >>>>>>>>>
> >>>>>>>>>> On 4 Jan 2017, at 20:35, Chris Riccomini <criccomini@apache.org
> >>>>>>>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Bolke, can you describe the current state of the alpha 2
> >>>>>> release? I
> >>>>>>> saw
> >>>>>>>>>> some comments from Alex yesterday about celery instability. If
> >>>>>> I'm
> >>>>>>>>> running
> >>>>>>>>>> on LocalExecutor, should I be seeing any issues?
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <
> >>>>>> bdbruin@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi All,
> >>>>>>>>>>>
> >>>>>>>>>>> I have put up Airflow 1.8.0 alpha 2 in
> >>>>>> https://people.apache.org/~
> >>>>>>>>> bolke/ <
> >>>>>>>>>>> https://people.apache.org/~bolke/> .
> >>>>>>>>>>>
> >>>>>>>>>>> Note: This still cannot be considered an Apache release.
> >>>>>> Working on
> >>>>>>>>> this.
> >>>>>>>>>>>
> >>>>>>>>>>> This build is signed (note it is served over https).
> >>>>>>>>>>>
> >>>>>>>>>>> Changes are in the area of scheduler stability.
> >>>>>>>>>>>
> >>>>>>>>>>> - Bolke
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>> --
> >>>>>>> _/
> >>>>>>> _/ Alex Van Boxel
> >>>>>>>
> >>>>>>
> >>>>>> ________________________________
> >>>>>> This e-mail and any attachments may be confidential or legally
> >>>>>> privileged. If you received this message in error or are not the
> intended
> >>>>>> recipient, you should destroy the e-mail message and any
> attachments or
> >>>>>> copies, and you are prohibited from retaining, distributing,
> disclosing or
> >>>>>> using any information contained herein. Please inform us of the
> erroneous
> >>>>>> delivery by return e-mail. Thank you for your cooperation.
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >
>
>
Re: Airflow 1.8.0 alpha 2
Posted by Bolke de Bruin <bd...@gmail.com>.
For the XCOM issue: https://issues.apache.org/jira/browse/AIRFLOW-738 <https://issues.apache.org/jira/browse/AIRFLOW-738>
With example dag that exposes the issue. Would be nice if someone with XCOM exposure could take a look.
- Bolke
> On 6 Jan 2017, at 15:31, Bolke de Bruin <bd...@gmail.com> wrote:
>
> State of to be Alpha 3:
>
> Blockers:
>
> * XCom throws an duplicate / locking error (new - confirmed)
> * one_failed task not being run: now seems to pass suddenly (so fixed?) -> need to investigate why
>
> Fixed issues
> * Regression in email
> * LDAP case sensitivity
>
> Pending features:
> * DAG.catchup : minor changes needed, documentation still required, integration tests seem to pass flawlessly
> * Cgroups + impersonation: clean up of patches on going, more tests and more elaborate documentation required. Integration tests not executed yet
> * Schedule all pending DAG runs in a single scheduler loop: no progress (**)
> * Email attachments
> * Add execution_date to trigger_dag
>
> If the pending features are merged within a reasonable time frame (except for **, as no progress currently) then I am planning to mark the tarball as Beta and only allow bug fixes and (very) minor features. Hopefully end of next week.
>
> Bolke.
>
>
>
>> On 5 Jan 2017, at 20:07, Chris Riccomini <cr...@apache.org> wrote:
>>
>> I have merged Robin's LDAP patch.
>>
>> On Thu, Jan 5, 2017 at 10:39 AM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>>> I have found the email problem:
>>>
>>> https://issues.apache.org/jira/browse/AIRFLOW-734
>>>
>>> Working on a fix.
>>>
>>> On Thu, Jan 5, 2017 at 9:10 AM, Chris Riccomini <cr...@apache.org>
>>> wrote:
>>>
>>>> Also, I'm seeing a second issue:
>>>>
>>>> SMTP doesn't seem to work for us anymore:
>>>>
>>>> [2017-01-05 15:10:13,666] {models.py:1378} ERROR - SMTP AUTH extension not supported by server.
>>>> Traceback (most recent call last):
>>>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1374, in handle_failure
>>>> self.email_alert(error, is_retry=False)
>>>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1521, in email_alert
>>>> send_email(task.email, title, body)
>>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 43, in send_email
>>>> return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
>>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 84, in send_email_smtp
>>>> send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
>>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 100, in send_MIME_email
>>>> s.login(SMTP_USER, SMTP_PASSWORD)
>>>> File "/usr/lib64/python2.7/smtplib.py", line 584, in login
>>>> raise SMTPException("SMTP AUTH extension not supported by server.")
>>>> SMTPException: SMTP AUTH extension not supported by server.
>>>>
>>>> This was working on 1.7.1.2. Having a look now.
>>>>
>>>> On Thu, Jan 5, 2017 at 9:09 AM, Chris Riccomini <cr...@apache.org>
>>>> wrote:
>>>>
>>>>> Hey Robin,
>>>>>
>>>>> Awesome, thanks! I love open source. :) Let me try your patch out and
>>>>> merge it.
>>>>>
>>>>> Cheers,
>>>>> Chris
>>>>>
>>>>> On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <
>>>>> Robin.Miller@affiliate.oliverwyman.com> wrote:
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>>
>>>>>> I think I ran into this issue when setting up LDAP Auth in our
>>>>>> environment (we're using very close to master as we needed some of the
>>>>>> newer features/bugfixes). The problem turned out to be that the search was
>>>>>> finding no results, so the line:
>>>>>>
>>>>>>
>>>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>>>>
>>>>>>
>>>>>> would fail because it had no matching groups to return. This turned out
>>>>>> to be because Windows Active Directory (the LDAP server we're using)
>>>>>> returned capitals "CN=" where the code expected lowercase: regex =
>>>>>> re.compile("cn=([^,]*).*")
>>>>>>
>>>>>>
>>>>>> I haven't looked it up, but Windows Active Directory is case
>>>>>> insensitive when it comes to usernames and groups, so I wouldn't be
>>>>>> surprised if the protocol itself is case insensitive and both "cn=" and
>>>>>> "CN=" should be considered valid. As such I've a PR open for a simple fix
>>>>>> to make this regex case insensitive: https://github.com/apache/incu
>>>>>> bator-airflow/pull/1945
>>>>>>
>>>>>>
>>>>>> Hopefully this helps,
>>>>>>
>>>>>> Robin Miller
>>>>>> OLIVER WYMAN
>>>>>> robin.miller@affiliate.oliverwyman.com<mailto:robin.miller@a
>>>>>> ffiliate.oliverwyman.com>
>>>>>> www.oliverwyman.com<http://www.oliverwyman.com/>
>>>>>>
>>>>>> ________________________________
>>>>>> From: Chris Riccomini <cr...@apache.org>
>>>>>> Sent: 05 January 2017 00:34:27
>>>>>> To: dev@airflow.incubator.apache.org
>>>>>> Subject: Re: Airflow 1.8.0 alpha 2
>>>>>>
>>>>>> I am now running 1.8.0a2 in our dev environment. It seems to be
>>>>>> functioning
>>>>>> well.
>>>>>>
>>>>>> One issue we've hit is that the LDAP auth plugin isn't working for us
>>>>>> anymore:
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
>>>>>> wsgi_app
>>>>>> response = self.full_dispatch_request()
>>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
>>>>>> full_dispatch_request
>>>>>> rv = self.handle_user_exception(e)
>>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
>>>>>> handle_user_exception
>>>>>> reraise(exc_type, exc_value, tb)
>>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
>>>>>> full_dispatch_request
>>>>>> rv = self.dispatch_request()
>>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
>>>>>> dispatch_request
>>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>>>>> 69,
>>>>>> in inner
>>>>>> return self._run_view(f, *args, **kwargs)
>>>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>>>>> 368,
>>>>>> in _run_view
>>>>>> return fn(self, *args, **kwargs)
>>>>>> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line
>>>>>> 657,
>>>>>> in login
>>>>>> return airflow.login.login(self, request)
>>>>>> File
>>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>>>> nds/ldap_auth.py",
>>>>>> line 276, in login
>>>>>> flask_login.login_user(LdapUser(user))
>>>>>> File "<string>", line 4, in __init__
>>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
>>>>>> line
>>>>>> 306, in _initialize_instance
>>>>>> manager.dispatch.init_failure(self, args, kwargs)
>>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelp
>>>>>> ers.py",
>>>>>> line 60, in __exit__
>>>>>> compat.reraise(exc_type, exc_value, exc_tb)
>>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
>>>>>> line
>>>>>> 303, in _initialize_instance
>>>>>> return manager.original_init(*mixed[1:], **kwargs)
>>>>>> File
>>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>>>> nds/ldap_auth.py",
>>>>>> line 148, in __init__
>>>>>> user.username)
>>>>>> File
>>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>>>> nds/ldap_auth.py",
>>>>>> line 106, in groups_user
>>>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>>>> AttributeError: 'NoneType' object has no attribute 'group'
>>>>>>
>>>>>> I believe it's from this patch:
>>>>>>
>>>>>> https://github.com/apache/incubator-airflow/commit/d6d3f5367
>>>>>> 3ba3736d7a858531823933cfef2bb4e
>>>>>>
>>>>>> I haven't dug into it yet. We'll need to fix it, though. In the
>>>>>> meantime, I
>>>>>> just commented out the call to `groups_user`. More to come.
>>>>>>
>>>>>> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be>
>>>>>> wrote:
>>>>>>
>>>>>>> I have another fix that certainly need to be in the final release,
>>>>>> but not
>>>>>>> ready to merge due to failed tests:
>>>>>>>
>>>>>>> https://github.com/apache/incubator-airflow/pull/1961
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <criccomini@apache.org
>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Great, I will work toward deploying this in our dev cluster today.
>>>>>> :D
>>>>>>>>
>>>>>>>> On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Some issues remain:
>>>>>>>>>
>>>>>>>>> * one_failed not executed as dag run is marked failed seemingly
>>>>>>>>> prematurely (@chris yes you should see this, see below for an
>>>>>> example
>>>>>>>> that
>>>>>>>>> is not working properly), confirmed regression
>>>>>>>>> * celery instability Alex
>>>>>>>>> * Wrong DAG state after failure inside branch
>>>>>>>>>
>>>>>>>>> So Alpha 2 is definitely not ready for production, but please do
>>>>>> put in
>>>>>>>>> your canary dags and let them run. I am still quite concerned
>>>>>> about the
>>>>>>>>> scheduler integrity and stability.
>>>>>>>>>
>>>>>>>>> - Bolke
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> one_failed_not_executed.py
>>>>>>>>> ====
>>>>>>>>> from airflow import DAG
>>>>>>>>> from airflow.operators.bash_operator import BashOperator
>>>>>>>>> from airflow.operators.dummy_operator import DummyOperator
>>>>>>>>> from datetime import datetime, timedelta
>>>>>>>>>
>>>>>>>>> default_args = {
>>>>>>>>> 'owner': 'airflow',
>>>>>>>>> 'depends_on_past': False,
>>>>>>>>> 'start_date': datetime(2016,10,5,19),
>>>>>>>>> 'email': ['airflow@airflow.com'],
>>>>>>>>> 'email_on_failure': False,
>>>>>>>>> 'email_on_retry': False,
>>>>>>>>> 'retries': 1,
>>>>>>>>> 'retry_delay': timedelta(seconds=1),
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> dag = DAG('tutorial', default_args=default_args,
>>>>>>>> schedule_interval='@once')
>>>>>>>>>
>>>>>>>>> task1 = BashOperator(
>>>>>>>>> task_id='first_one',
>>>>>>>>> bash_command='date',
>>>>>>>>> dag=dag)
>>>>>>>>>
>>>>>>>>> task2 = BashOperator(
>>>>>>>>> task_id='second_one',
>>>>>>>>> bash_command='this_should_not_work',
>>>>>>>>> dag=dag)
>>>>>>>>>
>>>>>>>>> task2.set_upstream(task1)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> task3 = BashOperator(
>>>>>>>>> task_id='third_one',
>>>>>>>>> bash_command='random_command_third',
>>>>>>>>> dag=dag)
>>>>>>>>>
>>>>>>>>> task3.set_upstream(task2)
>>>>>>>>>
>>>>>>>>> fail_task = DummyOperator(
>>>>>>>>> task_id='one_failed',
>>>>>>>>> trigger_rule='one_failed',
>>>>>>>>> dag=dag)
>>>>>>>>>
>>>>>>>>> fail_task.set_upstream([task1,task2,task3])
>>>>>>>>>
>>>>>>>>>> On 4 Jan 2017, at 20:35, Chris Riccomini <criccomini@apache.org
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Bolke, can you describe the current state of the alpha 2
>>>>>> release? I
>>>>>>> saw
>>>>>>>>>> some comments from Alex yesterday about celery instability. If
>>>>>> I'm
>>>>>>>>> running
>>>>>>>>>> on LocalExecutor, should I be seeing any issues?
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <
>>>>>> bdbruin@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi All,
>>>>>>>>>>>
>>>>>>>>>>> I have put up Airflow 1.8.0 alpha 2 in
>>>>>> https://people.apache.org/~
>>>>>>>>> bolke/ <
>>>>>>>>>>> https://people.apache.org/~bolke/> .
>>>>>>>>>>>
>>>>>>>>>>> Note: This still cannot be considered an Apache release.
>>>>>> Working on
>>>>>>>>> this.
>>>>>>>>>>>
>>>>>>>>>>> This build is signed (note it is served over https).
>>>>>>>>>>>
>>>>>>>>>>> Changes are in the area of scheduler stability.
>>>>>>>>>>>
>>>>>>>>>>> - Bolke
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> _/
>>>>>>> _/ Alex Van Boxel
>>>>>>>
>>>>>>
>>>>>> ________________________________
>>>>>> This e-mail and any attachments may be confidential or legally
>>>>>> privileged. If you received this message in error or are not the intended
>>>>>> recipient, you should destroy the e-mail message and any attachments or
>>>>>> copies, and you are prohibited from retaining, distributing, disclosing or
>>>>>> using any information contained herein. Please inform us of the erroneous
>>>>>> delivery by return e-mail. Thank you for your cooperation.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>
Re: Airflow 1.8.0 alpha 2
Posted by Bolke de Bruin <bd...@gmail.com>.
State of to be Alpha 3:
Blockers:
* XCom throws an duplicate / locking error (new - confirmed)
* one_failed task not being run: now seems to pass suddenly (so fixed?) -> need to investigate why
Fixed issues
* Regression in email
* LDAP case sensitivity
Pending features:
* DAG.catchup : minor changes needed, documentation still required, integration tests seem to pass flawlessly
* Cgroups + impersonation: clean up of patches on going, more tests and more elaborate documentation required. Integration tests not executed yet
* Schedule all pending DAG runs in a single scheduler loop: no progress (**)
* Email attachments
* Add execution_date to trigger_dag
If the pending features are merged within a reasonable time frame (except for **, as no progress currently) then I am planning to mark the tarball as Beta and only allow bug fixes and (very) minor features. Hopefully end of next week.
Bolke.
> On 5 Jan 2017, at 20:07, Chris Riccomini <cr...@apache.org> wrote:
>
> I have merged Robin's LDAP patch.
>
> On Thu, Jan 5, 2017 at 10:39 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> I have found the email problem:
>>
>> https://issues.apache.org/jira/browse/AIRFLOW-734
>>
>> Working on a fix.
>>
>> On Thu, Jan 5, 2017 at 9:10 AM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>>> Also, I'm seeing a second issue:
>>>
>>> SMTP doesn't seem to work for us anymore:
>>>
>>> [2017-01-05 15:10:13,666] {models.py:1378} ERROR - SMTP AUTH extension not supported by server.
>>> Traceback (most recent call last):
>>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1374, in handle_failure
>>> self.email_alert(error, is_retry=False)
>>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1521, in email_alert
>>> send_email(task.email, title, body)
>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 43, in send_email
>>> return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 84, in send_email_smtp
>>> send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
>>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 100, in send_MIME_email
>>> s.login(SMTP_USER, SMTP_PASSWORD)
>>> File "/usr/lib64/python2.7/smtplib.py", line 584, in login
>>> raise SMTPException("SMTP AUTH extension not supported by server.")
>>> SMTPException: SMTP AUTH extension not supported by server.
>>>
>>> This was working on 1.7.1.2. Having a look now.
>>>
>>> On Thu, Jan 5, 2017 at 9:09 AM, Chris Riccomini <cr...@apache.org>
>>> wrote:
>>>
>>>> Hey Robin,
>>>>
>>>> Awesome, thanks! I love open source. :) Let me try your patch out and
>>>> merge it.
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <
>>>> Robin.Miller@affiliate.oliverwyman.com> wrote:
>>>>
>>>>> Hi Chris,
>>>>>
>>>>>
>>>>> I think I ran into this issue when setting up LDAP Auth in our
>>>>> environment (we're using very close to master as we needed some of the
>>>>> newer features/bugfixes). The problem turned out to be that the search was
>>>>> finding no results, so the line:
>>>>>
>>>>>
>>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>>>
>>>>>
>>>>> would fail because it had no matching groups to return. This turned out
>>>>> to be because Windows Active Directory (the LDAP server we're using)
>>>>> returned capitals "CN=" where the code expected lowercase: regex =
>>>>> re.compile("cn=([^,]*).*")
>>>>>
>>>>>
>>>>> I haven't looked it up, but Windows Active Directory is case
>>>>> insensitive when it comes to usernames and groups, so I wouldn't be
>>>>> surprised if the protocol itself is case insensitive and both "cn=" and
>>>>> "CN=" should be considered valid. As such I've a PR open for a simple fix
>>>>> to make this regex case insensitive: https://github.com/apache/incu
>>>>> bator-airflow/pull/1945
>>>>>
>>>>>
>>>>> Hopefully this helps,
>>>>>
>>>>> Robin Miller
>>>>> OLIVER WYMAN
>>>>> robin.miller@affiliate.oliverwyman.com<mailto:robin.miller@a
>>>>> ffiliate.oliverwyman.com>
>>>>> www.oliverwyman.com<http://www.oliverwyman.com/>
>>>>>
>>>>> ________________________________
>>>>> From: Chris Riccomini <cr...@apache.org>
>>>>> Sent: 05 January 2017 00:34:27
>>>>> To: dev@airflow.incubator.apache.org
>>>>> Subject: Re: Airflow 1.8.0 alpha 2
>>>>>
>>>>> I am now running 1.8.0a2 in our dev environment. It seems to be
>>>>> functioning
>>>>> well.
>>>>>
>>>>> One issue we've hit is that the LDAP auth plugin isn't working for us
>>>>> anymore:
>>>>>
>>>>> Traceback (most recent call last):
>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
>>>>> wsgi_app
>>>>> response = self.full_dispatch_request()
>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
>>>>> full_dispatch_request
>>>>> rv = self.handle_user_exception(e)
>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
>>>>> handle_user_exception
>>>>> reraise(exc_type, exc_value, tb)
>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
>>>>> full_dispatch_request
>>>>> rv = self.dispatch_request()
>>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
>>>>> dispatch_request
>>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>>>> 69,
>>>>> in inner
>>>>> return self._run_view(f, *args, **kwargs)
>>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>>>> 368,
>>>>> in _run_view
>>>>> return fn(self, *args, **kwargs)
>>>>> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line
>>>>> 657,
>>>>> in login
>>>>> return airflow.login.login(self, request)
>>>>> File
>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>>> nds/ldap_auth.py",
>>>>> line 276, in login
>>>>> flask_login.login_user(LdapUser(user))
>>>>> File "<string>", line 4, in __init__
>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
>>>>> line
>>>>> 306, in _initialize_instance
>>>>> manager.dispatch.init_failure(self, args, kwargs)
>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelp
>>>>> ers.py",
>>>>> line 60, in __exit__
>>>>> compat.reraise(exc_type, exc_value, exc_tb)
>>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
>>>>> line
>>>>> 303, in _initialize_instance
>>>>> return manager.original_init(*mixed[1:], **kwargs)
>>>>> File
>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>>> nds/ldap_auth.py",
>>>>> line 148, in __init__
>>>>> user.username)
>>>>> File
>>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>>> nds/ldap_auth.py",
>>>>> line 106, in groups_user
>>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>>> AttributeError: 'NoneType' object has no attribute 'group'
>>>>>
>>>>> I believe it's from this patch:
>>>>>
>>>>> https://github.com/apache/incubator-airflow/commit/d6d3f5367
>>>>> 3ba3736d7a858531823933cfef2bb4e
>>>>>
>>>>> I haven't dug into it yet. We'll need to fix it, though. In the
>>>>> meantime, I
>>>>> just commented out the call to `groups_user`. More to come.
>>>>>
>>>>> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be>
>>>>> wrote:
>>>>>
>>>>>> I have another fix that certainly need to be in the final release,
>>>>> but not
>>>>>> ready to merge due to failed tests:
>>>>>>
>>>>>> https://github.com/apache/incubator-airflow/pull/1961
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <criccomini@apache.org
>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Great, I will work toward deploying this in our dev cluster today.
>>>>> :D
>>>>>>>
>>>>>>> On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Some issues remain:
>>>>>>>>
>>>>>>>> * one_failed not executed as dag run is marked failed seemingly
>>>>>>>> prematurely (@chris yes you should see this, see below for an
>>>>> example
>>>>>>> that
>>>>>>>> is not working properly), confirmed regression
>>>>>>>> * celery instability Alex
>>>>>>>> * Wrong DAG state after failure inside branch
>>>>>>>>
>>>>>>>> So Alpha 2 is definitely not ready for production, but please do
>>>>> put in
>>>>>>>> your canary dags and let them run. I am still quite concerned
>>>>> about the
>>>>>>>> scheduler integrity and stability.
>>>>>>>>
>>>>>>>> - Bolke
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> one_failed_not_executed.py
>>>>>>>> ====
>>>>>>>> from airflow import DAG
>>>>>>>> from airflow.operators.bash_operator import BashOperator
>>>>>>>> from airflow.operators.dummy_operator import DummyOperator
>>>>>>>> from datetime import datetime, timedelta
>>>>>>>>
>>>>>>>> default_args = {
>>>>>>>> 'owner': 'airflow',
>>>>>>>> 'depends_on_past': False,
>>>>>>>> 'start_date': datetime(2016,10,5,19),
>>>>>>>> 'email': ['airflow@airflow.com'],
>>>>>>>> 'email_on_failure': False,
>>>>>>>> 'email_on_retry': False,
>>>>>>>> 'retries': 1,
>>>>>>>> 'retry_delay': timedelta(seconds=1),
>>>>>>>> }
>>>>>>>>
>>>>>>>> dag = DAG('tutorial', default_args=default_args,
>>>>>>> schedule_interval='@once')
>>>>>>>>
>>>>>>>> task1 = BashOperator(
>>>>>>>> task_id='first_one',
>>>>>>>> bash_command='date',
>>>>>>>> dag=dag)
>>>>>>>>
>>>>>>>> task2 = BashOperator(
>>>>>>>> task_id='second_one',
>>>>>>>> bash_command='this_should_not_work',
>>>>>>>> dag=dag)
>>>>>>>>
>>>>>>>> task2.set_upstream(task1)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> task3 = BashOperator(
>>>>>>>> task_id='third_one',
>>>>>>>> bash_command='random_command_third',
>>>>>>>> dag=dag)
>>>>>>>>
>>>>>>>> task3.set_upstream(task2)
>>>>>>>>
>>>>>>>> fail_task = DummyOperator(
>>>>>>>> task_id='one_failed',
>>>>>>>> trigger_rule='one_failed',
>>>>>>>> dag=dag)
>>>>>>>>
>>>>>>>> fail_task.set_upstream([task1,task2,task3])
>>>>>>>>
>>>>>>>>> On 4 Jan 2017, at 20:35, Chris Riccomini <criccomini@apache.org
>>>>>>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Bolke, can you describe the current state of the alpha 2
>>>>> release? I
>>>>>> saw
>>>>>>>>> some comments from Alex yesterday about celery instability. If
>>>>> I'm
>>>>>>>> running
>>>>>>>>> on LocalExecutor, should I be seeing any issues?
>>>>>>>>>
>>>>>>>>> On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <
>>>>> bdbruin@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi All,
>>>>>>>>>>
>>>>>>>>>> I have put up Airflow 1.8.0 alpha 2 in
>>>>> https://people.apache.org/~
>>>>>>>> bolke/ <
>>>>>>>>>> https://people.apache.org/~bolke/> .
>>>>>>>>>>
>>>>>>>>>> Note: This still cannot be considered an Apache release.
>>>>> Working on
>>>>>>>> this.
>>>>>>>>>>
>>>>>>>>>> This build is signed (note it is served over https).
>>>>>>>>>>
>>>>>>>>>> Changes are in the area of scheduler stability.
>>>>>>>>>>
>>>>>>>>>> - Bolke
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> _/
>>>>>> _/ Alex Van Boxel
>>>>>>
>>>>>
>>>>> ________________________________
>>>>> This e-mail and any attachments may be confidential or legally
>>>>> privileged. If you received this message in error or are not the intended
>>>>> recipient, you should destroy the e-mail message and any attachments or
>>>>> copies, and you are prohibited from retaining, distributing, disclosing or
>>>>> using any information contained herein. Please inform us of the erroneous
>>>>> delivery by return e-mail. Thank you for your cooperation.
>>>>>
>>>>
>>>>
>>>
>>
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
I have merged Robin's LDAP patch.
On Thu, Jan 5, 2017 at 10:39 AM, Chris Riccomini <cr...@apache.org>
wrote:
> I have found the email problem:
>
> https://issues.apache.org/jira/browse/AIRFLOW-734
>
> Working on a fix.
>
> On Thu, Jan 5, 2017 at 9:10 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> Also, I'm seeing a second issue:
>>
>> SMTP doesn't seem to work for us anymore:
>>
>> [2017-01-05 15:10:13,666] {models.py:1378} ERROR - SMTP AUTH extension not supported by server.
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1374, in handle_failure
>> self.email_alert(error, is_retry=False)
>> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1521, in email_alert
>> send_email(task.email, title, body)
>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 43, in send_email
>> return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 84, in send_email_smtp
>> send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
>> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 100, in send_MIME_email
>> s.login(SMTP_USER, SMTP_PASSWORD)
>> File "/usr/lib64/python2.7/smtplib.py", line 584, in login
>> raise SMTPException("SMTP AUTH extension not supported by server.")
>> SMTPException: SMTP AUTH extension not supported by server.
>>
>> This was working on 1.7.1.2. Having a look now.
>>
>> On Thu, Jan 5, 2017 at 9:09 AM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>>> Hey Robin,
>>>
>>> Awesome, thanks! I love open source. :) Let me try your patch out and
>>> merge it.
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <
>>> Robin.Miller@affiliate.oliverwyman.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>>
>>>> I think I ran into this issue when setting up LDAP Auth in our
>>>> environment (we're using very close to master as we needed some of the
>>>> newer features/bugfixes). The problem turned out to be that the search was
>>>> finding no results, so the line:
>>>>
>>>>
>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>>
>>>>
>>>> would fail because it had no matching groups to return. This turned out
>>>> to be because Windows Active Directory (the LDAP server we're using)
>>>> returned capitals "CN=" where the code expected lowercase: regex =
>>>> re.compile("cn=([^,]*).*")
>>>>
>>>>
>>>> I haven't looked it up, but Windows Active Directory is case
>>>> insensitive when it comes to usernames and groups, so I wouldn't be
>>>> surprised if the protocol itself is case insensitive and both "cn=" and
>>>> "CN=" should be considered valid. As such I've a PR open for a simple fix
>>>> to make this regex case insensitive: https://github.com/apache/incu
>>>> bator-airflow/pull/1945
>>>>
>>>>
>>>> Hopefully this helps,
>>>>
>>>> Robin Miller
>>>> OLIVER WYMAN
>>>> robin.miller@affiliate.oliverwyman.com<mailto:robin.miller@a
>>>> ffiliate.oliverwyman.com>
>>>> www.oliverwyman.com<http://www.oliverwyman.com/>
>>>>
>>>> ________________________________
>>>> From: Chris Riccomini <cr...@apache.org>
>>>> Sent: 05 January 2017 00:34:27
>>>> To: dev@airflow.incubator.apache.org
>>>> Subject: Re: Airflow 1.8.0 alpha 2
>>>>
>>>> I am now running 1.8.0a2 in our dev environment. It seems to be
>>>> functioning
>>>> well.
>>>>
>>>> One issue we've hit is that the LDAP auth plugin isn't working for us
>>>> anymore:
>>>>
>>>> Traceback (most recent call last):
>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
>>>> wsgi_app
>>>> response = self.full_dispatch_request()
>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
>>>> full_dispatch_request
>>>> rv = self.handle_user_exception(e)
>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
>>>> handle_user_exception
>>>> reraise(exc_type, exc_value, tb)
>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
>>>> full_dispatch_request
>>>> rv = self.dispatch_request()
>>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
>>>> dispatch_request
>>>> return self.view_functions[rule.endpoint](**req.view_args)
>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>>> 69,
>>>> in inner
>>>> return self._run_view(f, *args, **kwargs)
>>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>>> 368,
>>>> in _run_view
>>>> return fn(self, *args, **kwargs)
>>>> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line
>>>> 657,
>>>> in login
>>>> return airflow.login.login(self, request)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>> nds/ldap_auth.py",
>>>> line 276, in login
>>>> flask_login.login_user(LdapUser(user))
>>>> File "<string>", line 4, in __init__
>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
>>>> line
>>>> 306, in _initialize_instance
>>>> manager.dispatch.init_failure(self, args, kwargs)
>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelp
>>>> ers.py",
>>>> line 60, in __exit__
>>>> compat.reraise(exc_type, exc_value, exc_tb)
>>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py",
>>>> line
>>>> 303, in _initialize_instance
>>>> return manager.original_init(*mixed[1:], **kwargs)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>> nds/ldap_auth.py",
>>>> line 148, in __init__
>>>> user.username)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>>> nds/ldap_auth.py",
>>>> line 106, in groups_user
>>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>> AttributeError: 'NoneType' object has no attribute 'group'
>>>>
>>>> I believe it's from this patch:
>>>>
>>>> https://github.com/apache/incubator-airflow/commit/d6d3f5367
>>>> 3ba3736d7a858531823933cfef2bb4e
>>>>
>>>> I haven't dug into it yet. We'll need to fix it, though. In the
>>>> meantime, I
>>>> just commented out the call to `groups_user`. More to come.
>>>>
>>>> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be>
>>>> wrote:
>>>>
>>>> > I have another fix that certainly need to be in the final release,
>>>> but not
>>>> > ready to merge due to failed tests:
>>>> >
>>>> > https://github.com/apache/incubator-airflow/pull/1961
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <criccomini@apache.org
>>>> >
>>>> > wrote:
>>>> >
>>>> > > Great, I will work toward deploying this in our dev cluster today.
>>>> :D
>>>> > >
>>>> > > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
>>>> > wrote:
>>>> > >
>>>> > > > Some issues remain:
>>>> > > >
>>>> > > > * one_failed not executed as dag run is marked failed seemingly
>>>> > > > prematurely (@chris yes you should see this, see below for an
>>>> example
>>>> > > that
>>>> > > > is not working properly), confirmed regression
>>>> > > > * celery instability Alex
>>>> > > > * Wrong DAG state after failure inside branch
>>>> > > >
>>>> > > > So Alpha 2 is definitely not ready for production, but please do
>>>> put in
>>>> > > > your canary dags and let them run. I am still quite concerned
>>>> about the
>>>> > > > scheduler integrity and stability.
>>>> > > >
>>>> > > > - Bolke
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > one_failed_not_executed.py
>>>> > > > ====
>>>> > > > from airflow import DAG
>>>> > > > from airflow.operators.bash_operator import BashOperator
>>>> > > > from airflow.operators.dummy_operator import DummyOperator
>>>> > > > from datetime import datetime, timedelta
>>>> > > >
>>>> > > > default_args = {
>>>> > > > 'owner': 'airflow',
>>>> > > > 'depends_on_past': False,
>>>> > > > 'start_date': datetime(2016,10,5,19),
>>>> > > > 'email': ['airflow@airflow.com'],
>>>> > > > 'email_on_failure': False,
>>>> > > > 'email_on_retry': False,
>>>> > > > 'retries': 1,
>>>> > > > 'retry_delay': timedelta(seconds=1),
>>>> > > > }
>>>> > > >
>>>> > > > dag = DAG('tutorial', default_args=default_args,
>>>> > > schedule_interval='@once')
>>>> > > >
>>>> > > > task1 = BashOperator(
>>>> > > > task_id='first_one',
>>>> > > > bash_command='date',
>>>> > > > dag=dag)
>>>> > > >
>>>> > > > task2 = BashOperator(
>>>> > > > task_id='second_one',
>>>> > > > bash_command='this_should_not_work',
>>>> > > > dag=dag)
>>>> > > >
>>>> > > > task2.set_upstream(task1)
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > task3 = BashOperator(
>>>> > > > task_id='third_one',
>>>> > > > bash_command='random_command_third',
>>>> > > > dag=dag)
>>>> > > >
>>>> > > > task3.set_upstream(task2)
>>>> > > >
>>>> > > > fail_task = DummyOperator(
>>>> > > > task_id='one_failed',
>>>> > > > trigger_rule='one_failed',
>>>> > > > dag=dag)
>>>> > > >
>>>> > > > fail_task.set_upstream([task1,task2,task3])
>>>> > > >
>>>> > > > > On 4 Jan 2017, at 20:35, Chris Riccomini <criccomini@apache.org
>>>> >
>>>> > > wrote:
>>>> > > > >
>>>> > > > > Bolke, can you describe the current state of the alpha 2
>>>> release? I
>>>> > saw
>>>> > > > > some comments from Alex yesterday about celery instability. If
>>>> I'm
>>>> > > > running
>>>> > > > > on LocalExecutor, should I be seeing any issues?
>>>> > > > >
>>>> > > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <
>>>> bdbruin@gmail.com>
>>>> > > > wrote:
>>>> > > > >
>>>> > > > >> Hi All,
>>>> > > > >>
>>>> > > > >> I have put up Airflow 1.8.0 alpha 2 in
>>>> https://people.apache.org/~
>>>> > > > bolke/ <
>>>> > > > >> https://people.apache.org/~bolke/> .
>>>> > > > >>
>>>> > > > >> Note: This still cannot be considered an Apache release.
>>>> Working on
>>>> > > > this.
>>>> > > > >>
>>>> > > > >> This build is signed (note it is served over https).
>>>> > > > >>
>>>> > > > >> Changes are in the area of scheduler stability.
>>>> > > > >>
>>>> > > > >> - Bolke
>>>> > > >
>>>> > > >
>>>> > >
>>>> > --
>>>> > _/
>>>> > _/ Alex Van Boxel
>>>> >
>>>>
>>>> ________________________________
>>>> This e-mail and any attachments may be confidential or legally
>>>> privileged. If you received this message in error or are not the intended
>>>> recipient, you should destroy the e-mail message and any attachments or
>>>> copies, and you are prohibited from retaining, distributing, disclosing or
>>>> using any information contained herein. Please inform us of the erroneous
>>>> delivery by return e-mail. Thank you for your cooperation.
>>>>
>>>
>>>
>>
>
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
I have found the email problem:
https://issues.apache.org/jira/browse/AIRFLOW-734
Working on a fix.
On Thu, Jan 5, 2017 at 9:10 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Also, I'm seeing a second issue:
>
> SMTP doesn't seem to work for us anymore:
>
> [2017-01-05 15:10:13,666] {models.py:1378} ERROR - SMTP AUTH extension not supported by server.
> Traceback (most recent call last):
> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1374, in handle_failure
> self.email_alert(error, is_retry=False)
> File "/usr/lib/python2.7/site-packages/airflow/models.py", line 1521, in email_alert
> send_email(task.email, title, body)
> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 43, in send_email
> return backend(to, subject, html_content, files=files, dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 84, in send_email_smtp
> send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
> File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line 100, in send_MIME_email
> s.login(SMTP_USER, SMTP_PASSWORD)
> File "/usr/lib64/python2.7/smtplib.py", line 584, in login
> raise SMTPException("SMTP AUTH extension not supported by server.")
> SMTPException: SMTP AUTH extension not supported by server.
>
> This was working on 1.7.1.2. Having a look now.
>
> On Thu, Jan 5, 2017 at 9:09 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> Hey Robin,
>>
>> Awesome, thanks! I love open source. :) Let me try your patch out and
>> merge it.
>>
>> Cheers,
>> Chris
>>
>> On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <
>> Robin.Miller@affiliate.oliverwyman.com> wrote:
>>
>>> Hi Chris,
>>>
>>>
>>> I think I ran into this issue when setting up LDAP Auth in our
>>> environment (we're using very close to master as we needed some of the
>>> newer features/bugfixes). The problem turned out to be that the search was
>>> finding no results, so the line:
>>>
>>>
>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>>
>>>
>>> would fail because it had no matching groups to return. This turned out
>>> to be because Windows Active Directory (the LDAP server we're using)
>>> returned capitals "CN=" where the code expected lowercase: regex =
>>> re.compile("cn=([^,]*).*")
>>>
>>>
>>> I haven't looked it up, but Windows Active Directory is case insensitive
>>> when it comes to usernames and groups, so I wouldn't be surprised if the
>>> protocol itself is case insensitive and both "cn=" and "CN=" should be
>>> considered valid. As such I've a PR open for a simple fix to make this
>>> regex case insensitive: https://github.com/apache/incu
>>> bator-airflow/pull/1945
>>>
>>>
>>> Hopefully this helps,
>>>
>>> Robin Miller
>>> OLIVER WYMAN
>>> robin.miller@affiliate.oliverwyman.com<mailto:robin.miller@a
>>> ffiliate.oliverwyman.com>
>>> www.oliverwyman.com<http://www.oliverwyman.com/>
>>>
>>> ________________________________
>>> From: Chris Riccomini <cr...@apache.org>
>>> Sent: 05 January 2017 00:34:27
>>> To: dev@airflow.incubator.apache.org
>>> Subject: Re: Airflow 1.8.0 alpha 2
>>>
>>> I am now running 1.8.0a2 in our dev environment. It seems to be
>>> functioning
>>> well.
>>>
>>> One issue we've hit is that the LDAP auth plugin isn't working for us
>>> anymore:
>>>
>>> Traceback (most recent call last):
>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
>>> wsgi_app
>>> response = self.full_dispatch_request()
>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
>>> full_dispatch_request
>>> rv = self.handle_user_exception(e)
>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
>>> handle_user_exception
>>> reraise(exc_type, exc_value, tb)
>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
>>> full_dispatch_request
>>> rv = self.dispatch_request()
>>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
>>> dispatch_request
>>> return self.view_functions[rule.endpoint](**req.view_args)
>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 69,
>>> in inner
>>> return self._run_view(f, *args, **kwargs)
>>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line
>>> 368,
>>> in _run_view
>>> return fn(self, *args, **kwargs)
>>> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line 657,
>>> in login
>>> return airflow.login.login(self, request)
>>> File
>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>> nds/ldap_auth.py",
>>> line 276, in login
>>> flask_login.login_user(LdapUser(user))
>>> File "<string>", line 4, in __init__
>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
>>> 306, in _initialize_instance
>>> manager.dispatch.init_failure(self, args, kwargs)
>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelp
>>> ers.py",
>>> line 60, in __exit__
>>> compat.reraise(exc_type, exc_value, exc_tb)
>>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
>>> 303, in _initialize_instance
>>> return manager.original_init(*mixed[1:], **kwargs)
>>> File
>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>> nds/ldap_auth.py",
>>> line 148, in __init__
>>> user.username)
>>> File
>>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>>> nds/ldap_auth.py",
>>> line 106, in groups_user
>>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>> AttributeError: 'NoneType' object has no attribute 'group'
>>>
>>> I believe it's from this patch:
>>>
>>> https://github.com/apache/incubator-airflow/commit/d6d3f5367
>>> 3ba3736d7a858531823933cfef2bb4e
>>>
>>> I haven't dug into it yet. We'll need to fix it, though. In the
>>> meantime, I
>>> just commented out the call to `groups_user`. More to come.
>>>
>>> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be>
>>> wrote:
>>>
>>> > I have another fix that certainly need to be in the final release, but
>>> not
>>> > ready to merge due to failed tests:
>>> >
>>> > https://github.com/apache/incubator-airflow/pull/1961
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <cr...@apache.org>
>>> > wrote:
>>> >
>>> > > Great, I will work toward deploying this in our dev cluster today. :D
>>> > >
>>> > > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
>>> > wrote:
>>> > >
>>> > > > Some issues remain:
>>> > > >
>>> > > > * one_failed not executed as dag run is marked failed seemingly
>>> > > > prematurely (@chris yes you should see this, see below for an
>>> example
>>> > > that
>>> > > > is not working properly), confirmed regression
>>> > > > * celery instability Alex
>>> > > > * Wrong DAG state after failure inside branch
>>> > > >
>>> > > > So Alpha 2 is definitely not ready for production, but please do
>>> put in
>>> > > > your canary dags and let them run. I am still quite concerned
>>> about the
>>> > > > scheduler integrity and stability.
>>> > > >
>>> > > > - Bolke
>>> > > >
>>> > > >
>>> > > >
>>> > > > one_failed_not_executed.py
>>> > > > ====
>>> > > > from airflow import DAG
>>> > > > from airflow.operators.bash_operator import BashOperator
>>> > > > from airflow.operators.dummy_operator import DummyOperator
>>> > > > from datetime import datetime, timedelta
>>> > > >
>>> > > > default_args = {
>>> > > > 'owner': 'airflow',
>>> > > > 'depends_on_past': False,
>>> > > > 'start_date': datetime(2016,10,5,19),
>>> > > > 'email': ['airflow@airflow.com'],
>>> > > > 'email_on_failure': False,
>>> > > > 'email_on_retry': False,
>>> > > > 'retries': 1,
>>> > > > 'retry_delay': timedelta(seconds=1),
>>> > > > }
>>> > > >
>>> > > > dag = DAG('tutorial', default_args=default_args,
>>> > > schedule_interval='@once')
>>> > > >
>>> > > > task1 = BashOperator(
>>> > > > task_id='first_one',
>>> > > > bash_command='date',
>>> > > > dag=dag)
>>> > > >
>>> > > > task2 = BashOperator(
>>> > > > task_id='second_one',
>>> > > > bash_command='this_should_not_work',
>>> > > > dag=dag)
>>> > > >
>>> > > > task2.set_upstream(task1)
>>> > > >
>>> > > >
>>> > > >
>>> > > > task3 = BashOperator(
>>> > > > task_id='third_one',
>>> > > > bash_command='random_command_third',
>>> > > > dag=dag)
>>> > > >
>>> > > > task3.set_upstream(task2)
>>> > > >
>>> > > > fail_task = DummyOperator(
>>> > > > task_id='one_failed',
>>> > > > trigger_rule='one_failed',
>>> > > > dag=dag)
>>> > > >
>>> > > > fail_task.set_upstream([task1,task2,task3])
>>> > > >
>>> > > > > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org>
>>> > > wrote:
>>> > > > >
>>> > > > > Bolke, can you describe the current state of the alpha 2
>>> release? I
>>> > saw
>>> > > > > some comments from Alex yesterday about celery instability. If
>>> I'm
>>> > > > running
>>> > > > > on LocalExecutor, should I be seeing any issues?
>>> > > > >
>>> > > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <
>>> bdbruin@gmail.com>
>>> > > > wrote:
>>> > > > >
>>> > > > >> Hi All,
>>> > > > >>
>>> > > > >> I have put up Airflow 1.8.0 alpha 2 in
>>> https://people.apache.org/~
>>> > > > bolke/ <
>>> > > > >> https://people.apache.org/~bolke/> .
>>> > > > >>
>>> > > > >> Note: This still cannot be considered an Apache release.
>>> Working on
>>> > > > this.
>>> > > > >>
>>> > > > >> This build is signed (note it is served over https).
>>> > > > >>
>>> > > > >> Changes are in the area of scheduler stability.
>>> > > > >>
>>> > > > >> - Bolke
>>> > > >
>>> > > >
>>> > >
>>> > --
>>> > _/
>>> > _/ Alex Van Boxel
>>> >
>>>
>>> ________________________________
>>> This e-mail and any attachments may be confidential or legally
>>> privileged. If you received this message in error or are not the intended
>>> recipient, you should destroy the e-mail message and any attachments or
>>> copies, and you are prohibited from retaining, distributing, disclosing or
>>> using any information contained herein. Please inform us of the erroneous
>>> delivery by return e-mail. Thank you for your cooperation.
>>>
>>
>>
>
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
Also, I'm seeing a second issue:
SMTP doesn't seem to work for us anymore:
[2017-01-05 15:10:13,666] {models.py:1378} ERROR - SMTP AUTH extension
not supported by server.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/airflow/models.py", line
1374, in handle_failure
self.email_alert(error, is_retry=False)
File "/usr/lib/python2.7/site-packages/airflow/models.py", line
1521, in email_alert
send_email(task.email, title, body)
File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line
43, in send_email
return backend(to, subject, html_content, files=files,
dryrun=dryrun, cc=cc, bcc=bcc, mime_subtype=mime_subtype)
File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line
84, in send_email_smtp
send_MIME_email(SMTP_MAIL_FROM, recipients, msg, dryrun)
File "/usr/lib/python2.7/site-packages/airflow/utils/email.py", line
100, in send_MIME_email
s.login(SMTP_USER, SMTP_PASSWORD)
File "/usr/lib64/python2.7/smtplib.py", line 584, in login
raise SMTPException("SMTP AUTH extension not supported by server.")
SMTPException: SMTP AUTH extension not supported by server.
This was working on 1.7.1.2. Having a look now.
On Thu, Jan 5, 2017 at 9:09 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey Robin,
>
> Awesome, thanks! I love open source. :) Let me try your patch out and
> merge it.
>
> Cheers,
> Chris
>
> On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <Robin.Miller@affiliate.
> oliverwyman.com> wrote:
>
>> Hi Chris,
>>
>>
>> I think I ran into this issue when setting up LDAP Auth in our
>> environment (we're using very close to master as we needed some of the
>> newer features/bugfixes). The problem turned out to be that the search was
>> finding no results, so the line:
>>
>>
>> groups_list = [regex.search(i).group(1) for i in user_groups]
>>
>>
>> would fail because it had no matching groups to return. This turned out
>> to be because Windows Active Directory (the LDAP server we're using)
>> returned capitals "CN=" where the code expected lowercase: regex =
>> re.compile("cn=([^,]*).*")
>>
>>
>> I haven't looked it up, but Windows Active Directory is case insensitive
>> when it comes to usernames and groups, so I wouldn't be surprised if the
>> protocol itself is case insensitive and both "cn=" and "CN=" should be
>> considered valid. As such I've a PR open for a simple fix to make this
>> regex case insensitive: https://github.com/apache/incu
>> bator-airflow/pull/1945
>>
>>
>> Hopefully this helps,
>>
>> Robin Miller
>> OLIVER WYMAN
>> robin.miller@affiliate.oliverwyman.com<mailto:robin.miller@
>> affiliate.oliverwyman.com>
>> www.oliverwyman.com<http://www.oliverwyman.com/>
>>
>> ________________________________
>> From: Chris Riccomini <cr...@apache.org>
>> Sent: 05 January 2017 00:34:27
>> To: dev@airflow.incubator.apache.org
>> Subject: Re: Airflow 1.8.0 alpha 2
>>
>> I am now running 1.8.0a2 in our dev environment. It seems to be
>> functioning
>> well.
>>
>> One issue we've hit is that the LDAP auth plugin isn't working for us
>> anymore:
>>
>> Traceback (most recent call last):
>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
>> wsgi_app
>> response = self.full_dispatch_request()
>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
>> full_dispatch_request
>> rv = self.handle_user_exception(e)
>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
>> handle_user_exception
>> reraise(exc_type, exc_value, tb)
>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
>> full_dispatch_request
>> rv = self.dispatch_request()
>> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
>> dispatch_request
>> return self.view_functions[rule.endpoint](**req.view_args)
>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 69,
>> in inner
>> return self._run_view(f, *args, **kwargs)
>> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 368,
>> in _run_view
>> return fn(self, *args, **kwargs)
>> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line 657,
>> in login
>> return airflow.login.login(self, request)
>> File
>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>> nds/ldap_auth.py",
>> line 276, in login
>> flask_login.login_user(LdapUser(user))
>> File "<string>", line 4, in __init__
>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
>> 306, in _initialize_instance
>> manager.dispatch.init_failure(self, args, kwargs)
>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelp
>> ers.py",
>> line 60, in __exit__
>> compat.reraise(exc_type, exc_value, exc_tb)
>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
>> 303, in _initialize_instance
>> return manager.original_init(*mixed[1:], **kwargs)
>> File
>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>> nds/ldap_auth.py",
>> line 148, in __init__
>> user.username)
>> File
>> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/backe
>> nds/ldap_auth.py",
>> line 106, in groups_user
>> groups_list = [regex.search(i).group(1) for i in user_groups]
>> AttributeError: 'NoneType' object has no attribute 'group'
>>
>> I believe it's from this patch:
>>
>> https://github.com/apache/incubator-airflow/commit/d6d3f5367
>> 3ba3736d7a858531823933cfef2bb4e
>>
>> I haven't dug into it yet. We'll need to fix it, though. In the meantime,
>> I
>> just commented out the call to `groups_user`. More to come.
>>
>> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be> wrote:
>>
>> > I have another fix that certainly need to be in the final release, but
>> not
>> > ready to merge due to failed tests:
>> >
>> > https://github.com/apache/incubator-airflow/pull/1961
>> >
>> >
>> >
>> >
>> > On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <cr...@apache.org>
>> > wrote:
>> >
>> > > Great, I will work toward deploying this in our dev cluster today. :D
>> > >
>> > > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
>> > wrote:
>> > >
>> > > > Some issues remain:
>> > > >
>> > > > * one_failed not executed as dag run is marked failed seemingly
>> > > > prematurely (@chris yes you should see this, see below for an
>> example
>> > > that
>> > > > is not working properly), confirmed regression
>> > > > * celery instability Alex
>> > > > * Wrong DAG state after failure inside branch
>> > > >
>> > > > So Alpha 2 is definitely not ready for production, but please do
>> put in
>> > > > your canary dags and let them run. I am still quite concerned about
>> the
>> > > > scheduler integrity and stability.
>> > > >
>> > > > - Bolke
>> > > >
>> > > >
>> > > >
>> > > > one_failed_not_executed.py
>> > > > ====
>> > > > from airflow import DAG
>> > > > from airflow.operators.bash_operator import BashOperator
>> > > > from airflow.operators.dummy_operator import DummyOperator
>> > > > from datetime import datetime, timedelta
>> > > >
>> > > > default_args = {
>> > > > 'owner': 'airflow',
>> > > > 'depends_on_past': False,
>> > > > 'start_date': datetime(2016,10,5,19),
>> > > > 'email': ['airflow@airflow.com'],
>> > > > 'email_on_failure': False,
>> > > > 'email_on_retry': False,
>> > > > 'retries': 1,
>> > > > 'retry_delay': timedelta(seconds=1),
>> > > > }
>> > > >
>> > > > dag = DAG('tutorial', default_args=default_args,
>> > > schedule_interval='@once')
>> > > >
>> > > > task1 = BashOperator(
>> > > > task_id='first_one',
>> > > > bash_command='date',
>> > > > dag=dag)
>> > > >
>> > > > task2 = BashOperator(
>> > > > task_id='second_one',
>> > > > bash_command='this_should_not_work',
>> > > > dag=dag)
>> > > >
>> > > > task2.set_upstream(task1)
>> > > >
>> > > >
>> > > >
>> > > > task3 = BashOperator(
>> > > > task_id='third_one',
>> > > > bash_command='random_command_third',
>> > > > dag=dag)
>> > > >
>> > > > task3.set_upstream(task2)
>> > > >
>> > > > fail_task = DummyOperator(
>> > > > task_id='one_failed',
>> > > > trigger_rule='one_failed',
>> > > > dag=dag)
>> > > >
>> > > > fail_task.set_upstream([task1,task2,task3])
>> > > >
>> > > > > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org>
>> > > wrote:
>> > > > >
>> > > > > Bolke, can you describe the current state of the alpha 2 release?
>> I
>> > saw
>> > > > > some comments from Alex yesterday about celery instability. If I'm
>> > > > running
>> > > > > on LocalExecutor, should I be seeing any issues?
>> > > > >
>> > > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bdbruin@gmail.com
>> >
>> > > > wrote:
>> > > > >
>> > > > >> Hi All,
>> > > > >>
>> > > > >> I have put up Airflow 1.8.0 alpha 2 in
>> https://people.apache.org/~
>> > > > bolke/ <
>> > > > >> https://people.apache.org/~bolke/> .
>> > > > >>
>> > > > >> Note: This still cannot be considered an Apache release. Working
>> on
>> > > > this.
>> > > > >>
>> > > > >> This build is signed (note it is served over https).
>> > > > >>
>> > > > >> Changes are in the area of scheduler stability.
>> > > > >>
>> > > > >> - Bolke
>> > > >
>> > > >
>> > >
>> > --
>> > _/
>> > _/ Alex Van Boxel
>> >
>>
>> ________________________________
>> This e-mail and any attachments may be confidential or legally
>> privileged. If you received this message in error or are not the intended
>> recipient, you should destroy the e-mail message and any attachments or
>> copies, and you are prohibited from retaining, distributing, disclosing or
>> using any information contained herein. Please inform us of the erroneous
>> delivery by return e-mail. Thank you for your cooperation.
>>
>
>
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
Hey Robin,
Awesome, thanks! I love open source. :) Let me try your patch out and merge
it.
Cheers,
Chris
On Thu, Jan 5, 2017 at 1:34 AM, Miller, Robin <
Robin.Miller@affiliate.oliverwyman.com> wrote:
> Hi Chris,
>
>
> I think I ran into this issue when setting up LDAP Auth in our environment
> (we're using very close to master as we needed some of the newer
> features/bugfixes). The problem turned out to be that the search was
> finding no results, so the line:
>
>
> groups_list = [regex.search(i).group(1) for i in user_groups]
>
>
> would fail because it had no matching groups to return. This turned out to
> be because Windows Active Directory (the LDAP server we're using) returned
> capitals "CN=" where the code expected lowercase: regex =
> re.compile("cn=([^,]*).*")
>
>
> I haven't looked it up, but Windows Active Directory is case insensitive
> when it comes to usernames and groups, so I wouldn't be surprised if the
> protocol itself is case insensitive and both "cn=" and "CN=" should be
> considered valid. As such I've a PR open for a simple fix to make this
> regex case insensitive: https://github.com/apache/
> incubator-airflow/pull/1945
>
>
> Hopefully this helps,
>
> Robin Miller
> OLIVER WYMAN
> robin.miller@affiliate.oliverwyman.com<mailto:robin.
> miller@affiliate.oliverwyman.com>
> www.oliverwyman.com<http://www.oliverwyman.com/>
>
> ________________________________
> From: Chris Riccomini <cr...@apache.org>
> Sent: 05 January 2017 00:34:27
> To: dev@airflow.incubator.apache.org
> Subject: Re: Airflow 1.8.0 alpha 2
>
> I am now running 1.8.0a2 in our dev environment. It seems to be functioning
> well.
>
> One issue we've hit is that the LDAP auth plugin isn't working for us
> anymore:
>
> Traceback (most recent call last):
> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
> wsgi_app
> response = self.full_dispatch_request()
> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
> full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
> handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
> full_dispatch_request
> rv = self.dispatch_request()
> File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
> dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 69,
> in inner
> return self._run_view(f, *args, **kwargs)
> File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 368,
> in _run_view
> return fn(self, *args, **kwargs)
> File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line 657,
> in login
> return airflow.login.login(self, request)
> File
> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/
> backends/ldap_auth.py",
> line 276, in login
> flask_login.login_user(LdapUser(user))
> File "<string>", line 4, in __init__
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
> 306, in _initialize_instance
> manager.dispatch.init_failure(self, args, kwargs)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
> line 60, in __exit__
> compat.reraise(exc_type, exc_value, exc_tb)
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
> 303, in _initialize_instance
> return manager.original_init(*mixed[1:], **kwargs)
> File
> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/
> backends/ldap_auth.py",
> line 148, in __init__
> user.username)
> File
> "/usr/lib/python2.7/site-packages/airflow/contrib/auth/
> backends/ldap_auth.py",
> line 106, in groups_user
> groups_list = [regex.search(i).group(1) for i in user_groups]
> AttributeError: 'NoneType' object has no attribute 'group'
>
> I believe it's from this patch:
>
> https://github.com/apache/incubator-airflow/commit/
> d6d3f53673ba3736d7a858531823933cfef2bb4e
>
> I haven't dug into it yet. We'll need to fix it, though. In the meantime, I
> just commented out the call to `groups_user`. More to come.
>
> On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be> wrote:
>
> > I have another fix that certainly need to be in the final release, but
> not
> > ready to merge due to failed tests:
> >
> > https://github.com/apache/incubator-airflow/pull/1961
> >
> >
> >
> >
> > On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <cr...@apache.org>
> > wrote:
> >
> > > Great, I will work toward deploying this in our dev cluster today. :D
> > >
> > > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
> > wrote:
> > >
> > > > Some issues remain:
> > > >
> > > > * one_failed not executed as dag run is marked failed seemingly
> > > > prematurely (@chris yes you should see this, see below for an example
> > > that
> > > > is not working properly), confirmed regression
> > > > * celery instability Alex
> > > > * Wrong DAG state after failure inside branch
> > > >
> > > > So Alpha 2 is definitely not ready for production, but please do put
> in
> > > > your canary dags and let them run. I am still quite concerned about
> the
> > > > scheduler integrity and stability.
> > > >
> > > > - Bolke
> > > >
> > > >
> > > >
> > > > one_failed_not_executed.py
> > > > ====
> > > > from airflow import DAG
> > > > from airflow.operators.bash_operator import BashOperator
> > > > from airflow.operators.dummy_operator import DummyOperator
> > > > from datetime import datetime, timedelta
> > > >
> > > > default_args = {
> > > > 'owner': 'airflow',
> > > > 'depends_on_past': False,
> > > > 'start_date': datetime(2016,10,5,19),
> > > > 'email': ['airflow@airflow.com'],
> > > > 'email_on_failure': False,
> > > > 'email_on_retry': False,
> > > > 'retries': 1,
> > > > 'retry_delay': timedelta(seconds=1),
> > > > }
> > > >
> > > > dag = DAG('tutorial', default_args=default_args,
> > > schedule_interval='@once')
> > > >
> > > > task1 = BashOperator(
> > > > task_id='first_one',
> > > > bash_command='date',
> > > > dag=dag)
> > > >
> > > > task2 = BashOperator(
> > > > task_id='second_one',
> > > > bash_command='this_should_not_work',
> > > > dag=dag)
> > > >
> > > > task2.set_upstream(task1)
> > > >
> > > >
> > > >
> > > > task3 = BashOperator(
> > > > task_id='third_one',
> > > > bash_command='random_command_third',
> > > > dag=dag)
> > > >
> > > > task3.set_upstream(task2)
> > > >
> > > > fail_task = DummyOperator(
> > > > task_id='one_failed',
> > > > trigger_rule='one_failed',
> > > > dag=dag)
> > > >
> > > > fail_task.set_upstream([task1,task2,task3])
> > > >
> > > > > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org>
> > > wrote:
> > > > >
> > > > > Bolke, can you describe the current state of the alpha 2 release? I
> > saw
> > > > > some comments from Alex yesterday about celery instability. If I'm
> > > > running
> > > > > on LocalExecutor, should I be seeing any issues?
> > > > >
> > > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com>
> > > > wrote:
> > > > >
> > > > >> Hi All,
> > > > >>
> > > > >> I have put up Airflow 1.8.0 alpha 2 in
> https://people.apache.org/~
> > > > bolke/ <
> > > > >> https://people.apache.org/~bolke/> .
> > > > >>
> > > > >> Note: This still cannot be considered an Apache release. Working
> on
> > > > this.
> > > > >>
> > > > >> This build is signed (note it is served over https).
> > > > >>
> > > > >> Changes are in the area of scheduler stability.
> > > > >>
> > > > >> - Bolke
> > > >
> > > >
> > >
> > --
> > _/
> > _/ Alex Van Boxel
> >
>
> ________________________________
> This e-mail and any attachments may be confidential or legally privileged.
> If you received this message in error or are not the intended recipient,
> you should destroy the e-mail message and any attachments or copies, and
> you are prohibited from retaining, distributing, disclosing or using any
> information contained herein. Please inform us of the erroneous delivery by
> return e-mail. Thank you for your cooperation.
>
Re: Airflow 1.8.0 alpha 2
Posted by "Miller, Robin" <Ro...@affiliate.oliverwyman.com>.
Hi Chris,
I think I ran into this issue when setting up LDAP Auth in our environment (we're using very close to master as we needed some of the newer features/bugfixes). The problem turned out to be that the search was finding no results, so the line:
groups_list = [regex.search(i).group(1) for i in user_groups]
would fail because it had no matching groups to return. This turned out to be because Windows Active Directory (the LDAP server we're using) returned capitals "CN=" where the code expected lowercase: regex = re.compile("cn=([^,]*).*")
I haven't looked it up, but Windows Active Directory is case insensitive when it comes to usernames and groups, so I wouldn't be surprised if the protocol itself is case insensitive and both "cn=" and "CN=" should be considered valid. As such I've a PR open for a simple fix to make this regex case insensitive: https://github.com/apache/incubator-airflow/pull/1945
Hopefully this helps,
Robin Miller
OLIVER WYMAN
robin.miller@affiliate.oliverwyman.com<ma...@affiliate.oliverwyman.com>
www.oliverwyman.com<http://www.oliverwyman.com/>
________________________________
From: Chris Riccomini <cr...@apache.org>
Sent: 05 January 2017 00:34:27
To: dev@airflow.incubator.apache.org
Subject: Re: Airflow 1.8.0 alpha 2
I am now running 1.8.0a2 in our dev environment. It seems to be functioning
well.
One issue we've hit is that the LDAP auth plugin isn't working for us
anymore:
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 69,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 368,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line 657,
in login
return airflow.login.login(self, request)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/auth/backends/ldap_auth.py",
line 276, in login
flask_login.login_user(LdapUser(user))
File "<string>", line 4, in __init__
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
306, in _initialize_instance
manager.dispatch.init_failure(self, args, kwargs)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
303, in _initialize_instance
return manager.original_init(*mixed[1:], **kwargs)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/auth/backends/ldap_auth.py",
line 148, in __init__
user.username)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/auth/backends/ldap_auth.py",
line 106, in groups_user
groups_list = [regex.search(i).group(1) for i in user_groups]
AttributeError: 'NoneType' object has no attribute 'group'
I believe it's from this patch:
https://github.com/apache/incubator-airflow/commit/d6d3f53673ba3736d7a858531823933cfef2bb4e
I haven't dug into it yet. We'll need to fix it, though. In the meantime, I
just commented out the call to `groups_user`. More to come.
On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be> wrote:
> I have another fix that certainly need to be in the final release, but not
> ready to merge due to failed tests:
>
> https://github.com/apache/incubator-airflow/pull/1961
>
>
>
>
> On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <cr...@apache.org>
> wrote:
>
> > Great, I will work toward deploying this in our dev cluster today. :D
> >
> > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
> wrote:
> >
> > > Some issues remain:
> > >
> > > * one_failed not executed as dag run is marked failed seemingly
> > > prematurely (@chris yes you should see this, see below for an example
> > that
> > > is not working properly), confirmed regression
> > > * celery instability Alex
> > > * Wrong DAG state after failure inside branch
> > >
> > > So Alpha 2 is definitely not ready for production, but please do put in
> > > your canary dags and let them run. I am still quite concerned about the
> > > scheduler integrity and stability.
> > >
> > > - Bolke
> > >
> > >
> > >
> > > one_failed_not_executed.py
> > > ====
> > > from airflow import DAG
> > > from airflow.operators.bash_operator import BashOperator
> > > from airflow.operators.dummy_operator import DummyOperator
> > > from datetime import datetime, timedelta
> > >
> > > default_args = {
> > > 'owner': 'airflow',
> > > 'depends_on_past': False,
> > > 'start_date': datetime(2016,10,5,19),
> > > 'email': ['airflow@airflow.com'],
> > > 'email_on_failure': False,
> > > 'email_on_retry': False,
> > > 'retries': 1,
> > > 'retry_delay': timedelta(seconds=1),
> > > }
> > >
> > > dag = DAG('tutorial', default_args=default_args,
> > schedule_interval='@once')
> > >
> > > task1 = BashOperator(
> > > task_id='first_one',
> > > bash_command='date',
> > > dag=dag)
> > >
> > > task2 = BashOperator(
> > > task_id='second_one',
> > > bash_command='this_should_not_work',
> > > dag=dag)
> > >
> > > task2.set_upstream(task1)
> > >
> > >
> > >
> > > task3 = BashOperator(
> > > task_id='third_one',
> > > bash_command='random_command_third',
> > > dag=dag)
> > >
> > > task3.set_upstream(task2)
> > >
> > > fail_task = DummyOperator(
> > > task_id='one_failed',
> > > trigger_rule='one_failed',
> > > dag=dag)
> > >
> > > fail_task.set_upstream([task1,task2,task3])
> > >
> > > > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org>
> > wrote:
> > > >
> > > > Bolke, can you describe the current state of the alpha 2 release? I
> saw
> > > > some comments from Alex yesterday about celery instability. If I'm
> > > running
> > > > on LocalExecutor, should I be seeing any issues?
> > > >
> > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com>
> > > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~
> > > bolke/ <
> > > >> https://people.apache.org/~bolke/> .
> > > >>
> > > >> Note: This still cannot be considered an Apache release. Working on
> > > this.
> > > >>
> > > >> This build is signed (note it is served over https).
> > > >>
> > > >> Changes are in the area of scheduler stability.
> > > >>
> > > >> - Bolke
> > >
> > >
> >
> --
> _/
> _/ Alex Van Boxel
>
________________________________
This e-mail and any attachments may be confidential or legally privileged. If you received this message in error or are not the intended recipient, you should destroy the e-mail message and any attachments or copies, and you are prohibited from retaining, distributing, disclosing or using any information contained herein. Please inform us of the erroneous delivery by return e-mail. Thank you for your cooperation.
Re: This DAG isn't available in the web server's DagBag object .....
Posted by Michael Gong <go...@hotmail.com>.
Hi, Maxime,
Thanks for the quick reply.
>The different processes, potentially running on different machines, need to be configured in a way that they see the DAGS_FOLDER.
Do you mean that different process on different machines should see the EXACTLY same dags information , so that a task is potentially to run on anyone of the different machines ?
If that's the case, it's different from what we want to achieve. We hope certain tasks to run on certain machines, due to policy restriction.
Please advise.
Thanks.
Michael
________________________________
From: Maxime Beauchemin <ma...@gmail.com>
Sent: Thursday, January 5, 2017 5:54 PM
To: dev@airflow.incubator.apache.org
Subject: Re: This DAG isn't available in the web server's DagBag object .....
The different processes, potentially running on different machines, need to
be configured in a way that they see the DAGS_FOLDER. At Airbnb we
replicate our DAGs definition repository using chef, and all the machines
share an identical `airflow.cfg` that points to the DAGS_FOLDER.
Pretty much every company has their own way to configure boxes, sync git
repos, and wrap executables into services, but the important thing is that
every Airflow process that is part of the same Airflow cluster should be
configured similarly (point to the same metadata db, point to the same DAGS
folder, have the same executor configuration, ...)
On Thu, Jan 5, 2017 at 7:32 AM, Michael Gong <go...@hotmail.com> wrote:
> Hi,
>
>
> Here is the cases:
>
> I have 2 separate airflow instances: 2 dag instance, 2 airflow server, 2
> airflow webservers.
>
> But they share the same MySQL db.
>
>
> From the 1st airflow webserver, I can see the dag instances from the 2nd
> airflow.
>
> But it is greyed out , with a message:
>
>
> "This DAG isn't available in the web server's DagBag object. It shows up
> in this list because the scheduler marked it as active in the metdata
> database."
>
>
>
> It seems making sense somehow.
>
>
> My question is how to add the 2nd dag instance to the 1st airflow web
> server's DagBag object ?
>
>
> The goal to achieve is that from the 1st airflow webserver, I can see both
> dag instances active.
>
>
> Any advices are welcomed.
>
>
> Thanks.
>
> Michael
>
>
>
>
Re: This DAG isn't available in the web server's DagBag object .....
Posted by Maxime Beauchemin <ma...@gmail.com>.
The different processes, potentially running on different machines, need to
be configured in a way that they see the DAGS_FOLDER. At Airbnb we
replicate our DAGs definition repository using chef, and all the machines
share an identical `airflow.cfg` that points to the DAGS_FOLDER.
Pretty much every company has their own way to configure boxes, sync git
repos, and wrap executables into services, but the important thing is that
every Airflow process that is part of the same Airflow cluster should be
configured similarly (point to the same metadata db, point to the same DAGS
folder, have the same executor configuration, ...)
On Thu, Jan 5, 2017 at 7:32 AM, Michael Gong <go...@hotmail.com> wrote:
> Hi,
>
>
> Here is the cases:
>
> I have 2 separate airflow instances: 2 dag instance, 2 airflow server, 2
> airflow webservers.
>
> But they share the same MySQL db.
>
>
> From the 1st airflow webserver, I can see the dag instances from the 2nd
> airflow.
>
> But it is greyed out , with a message:
>
>
> "This DAG isn't available in the web server's DagBag object. It shows up
> in this list because the scheduler marked it as active in the metdata
> database."
>
>
>
> It seems making sense somehow.
>
>
> My question is how to add the 2nd dag instance to the 1st airflow web
> server's DagBag object ?
>
>
> The goal to achieve is that from the 1st airflow webserver, I can see both
> dag instances active.
>
>
> Any advices are welcomed.
>
>
> Thanks.
>
> Michael
>
>
>
>
This DAG isn't available in the web server's DagBag object .....
Posted by Michael Gong <go...@hotmail.com>.
Hi,
Here is the cases:
I have 2 separate airflow instances: 2 dag instance, 2 airflow server, 2 airflow webservers.
But they share the same MySQL db.
From the 1st airflow webserver, I can see the dag instances from the 2nd airflow.
But it is greyed out , with a message:
"This DAG isn't available in the web server's DagBag object. It shows up in this list because the scheduler marked it as active in the metdata database."
It seems making sense somehow.
My question is how to add the 2nd dag instance to the 1st airflow web server's DagBag object ?
The goal to achieve is that from the 1st airflow webserver, I can see both dag instances active.
Any advices are welcomed.
Thanks.
Michael
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
I am now running 1.8.0a2 in our dev environment. It seems to be functioning
well.
One issue we've hit is that the LDAP auth plugin isn't working for us
anymore:
Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1988, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1641, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1544, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1639, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1625, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 69,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/lib64/python2.7/site-packages/flask_admin/base.py", line 368,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/airflow/www/views.py", line 657,
in login
return airflow.login.login(self, request)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/auth/backends/ldap_auth.py",
line 276, in login
flask_login.login_user(LdapUser(user))
File "<string>", line 4, in __init__
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
306, in _initialize_instance
manager.dispatch.init_failure(self, args, kwargs)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py",
line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/state.py", line
303, in _initialize_instance
return manager.original_init(*mixed[1:], **kwargs)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/auth/backends/ldap_auth.py",
line 148, in __init__
user.username)
File
"/usr/lib/python2.7/site-packages/airflow/contrib/auth/backends/ldap_auth.py",
line 106, in groups_user
groups_list = [regex.search(i).group(1) for i in user_groups]
AttributeError: 'NoneType' object has no attribute 'group'
I believe it's from this patch:
https://github.com/apache/incubator-airflow/commit/d6d3f53673ba3736d7a858531823933cfef2bb4e
I haven't dug into it yet. We'll need to fix it, though. In the meantime, I
just commented out the call to `groups_user`. More to come.
On Wed, Jan 4, 2017 at 12:32 PM, Alex Van Boxel <al...@vanboxel.be> wrote:
> I have another fix that certainly need to be in the final release, but not
> ready to merge due to failed tests:
>
> https://github.com/apache/incubator-airflow/pull/1961
>
>
>
>
> On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <cr...@apache.org>
> wrote:
>
> > Great, I will work toward deploying this in our dev cluster today. :D
> >
> > On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com>
> wrote:
> >
> > > Some issues remain:
> > >
> > > * one_failed not executed as dag run is marked failed seemingly
> > > prematurely (@chris yes you should see this, see below for an example
> > that
> > > is not working properly), confirmed regression
> > > * celery instability Alex
> > > * Wrong DAG state after failure inside branch
> > >
> > > So Alpha 2 is definitely not ready for production, but please do put in
> > > your canary dags and let them run. I am still quite concerned about the
> > > scheduler integrity and stability.
> > >
> > > - Bolke
> > >
> > >
> > >
> > > one_failed_not_executed.py
> > > ====
> > > from airflow import DAG
> > > from airflow.operators.bash_operator import BashOperator
> > > from airflow.operators.dummy_operator import DummyOperator
> > > from datetime import datetime, timedelta
> > >
> > > default_args = {
> > > 'owner': 'airflow',
> > > 'depends_on_past': False,
> > > 'start_date': datetime(2016,10,5,19),
> > > 'email': ['airflow@airflow.com'],
> > > 'email_on_failure': False,
> > > 'email_on_retry': False,
> > > 'retries': 1,
> > > 'retry_delay': timedelta(seconds=1),
> > > }
> > >
> > > dag = DAG('tutorial', default_args=default_args,
> > schedule_interval='@once')
> > >
> > > task1 = BashOperator(
> > > task_id='first_one',
> > > bash_command='date',
> > > dag=dag)
> > >
> > > task2 = BashOperator(
> > > task_id='second_one',
> > > bash_command='this_should_not_work',
> > > dag=dag)
> > >
> > > task2.set_upstream(task1)
> > >
> > >
> > >
> > > task3 = BashOperator(
> > > task_id='third_one',
> > > bash_command='random_command_third',
> > > dag=dag)
> > >
> > > task3.set_upstream(task2)
> > >
> > > fail_task = DummyOperator(
> > > task_id='one_failed',
> > > trigger_rule='one_failed',
> > > dag=dag)
> > >
> > > fail_task.set_upstream([task1,task2,task3])
> > >
> > > > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org>
> > wrote:
> > > >
> > > > Bolke, can you describe the current state of the alpha 2 release? I
> saw
> > > > some comments from Alex yesterday about celery instability. If I'm
> > > running
> > > > on LocalExecutor, should I be seeing any issues?
> > > >
> > > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com>
> > > wrote:
> > > >
> > > >> Hi All,
> > > >>
> > > >> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~
> > > bolke/ <
> > > >> https://people.apache.org/~bolke/> .
> > > >>
> > > >> Note: This still cannot be considered an Apache release. Working on
> > > this.
> > > >>
> > > >> This build is signed (note it is served over https).
> > > >>
> > > >> Changes are in the area of scheduler stability.
> > > >>
> > > >> - Bolke
> > >
> > >
> >
> --
> _/
> _/ Alex Van Boxel
>
Re: Airflow 1.8.0 alpha 2
Posted by Alex Van Boxel <al...@vanboxel.be>.
I have another fix that certainly need to be in the final release, but not
ready to merge due to failed tests:
https://github.com/apache/incubator-airflow/pull/1961
On Wed, Jan 4, 2017 at 8:48 PM Chris Riccomini <cr...@apache.org>
wrote:
> Great, I will work toward deploying this in our dev cluster today. :D
>
> On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com> wrote:
>
> > Some issues remain:
> >
> > * one_failed not executed as dag run is marked failed seemingly
> > prematurely (@chris yes you should see this, see below for an example
> that
> > is not working properly), confirmed regression
> > * celery instability Alex
> > * Wrong DAG state after failure inside branch
> >
> > So Alpha 2 is definitely not ready for production, but please do put in
> > your canary dags and let them run. I am still quite concerned about the
> > scheduler integrity and stability.
> >
> > - Bolke
> >
> >
> >
> > one_failed_not_executed.py
> > ====
> > from airflow import DAG
> > from airflow.operators.bash_operator import BashOperator
> > from airflow.operators.dummy_operator import DummyOperator
> > from datetime import datetime, timedelta
> >
> > default_args = {
> > 'owner': 'airflow',
> > 'depends_on_past': False,
> > 'start_date': datetime(2016,10,5,19),
> > 'email': ['airflow@airflow.com'],
> > 'email_on_failure': False,
> > 'email_on_retry': False,
> > 'retries': 1,
> > 'retry_delay': timedelta(seconds=1),
> > }
> >
> > dag = DAG('tutorial', default_args=default_args,
> schedule_interval='@once')
> >
> > task1 = BashOperator(
> > task_id='first_one',
> > bash_command='date',
> > dag=dag)
> >
> > task2 = BashOperator(
> > task_id='second_one',
> > bash_command='this_should_not_work',
> > dag=dag)
> >
> > task2.set_upstream(task1)
> >
> >
> >
> > task3 = BashOperator(
> > task_id='third_one',
> > bash_command='random_command_third',
> > dag=dag)
> >
> > task3.set_upstream(task2)
> >
> > fail_task = DummyOperator(
> > task_id='one_failed',
> > trigger_rule='one_failed',
> > dag=dag)
> >
> > fail_task.set_upstream([task1,task2,task3])
> >
> > > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org>
> wrote:
> > >
> > > Bolke, can you describe the current state of the alpha 2 release? I saw
> > > some comments from Alex yesterday about celery instability. If I'm
> > running
> > > on LocalExecutor, should I be seeing any issues?
> > >
> > > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com>
> > wrote:
> > >
> > >> Hi All,
> > >>
> > >> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~
> > bolke/ <
> > >> https://people.apache.org/~bolke/> .
> > >>
> > >> Note: This still cannot be considered an Apache release. Working on
> > this.
> > >>
> > >> This build is signed (note it is served over https).
> > >>
> > >> Changes are in the area of scheduler stability.
> > >>
> > >> - Bolke
> >
> >
>
--
_/
_/ Alex Van Boxel
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
Great, I will work toward deploying this in our dev cluster today. :D
On Wed, Jan 4, 2017 at 11:47 AM, Bolke de Bruin <bd...@gmail.com> wrote:
> Some issues remain:
>
> * one_failed not executed as dag run is marked failed seemingly
> prematurely (@chris yes you should see this, see below for an example that
> is not working properly), confirmed regression
> * celery instability Alex
> * Wrong DAG state after failure inside branch
>
> So Alpha 2 is definitely not ready for production, but please do put in
> your canary dags and let them run. I am still quite concerned about the
> scheduler integrity and stability.
>
> - Bolke
>
>
>
> one_failed_not_executed.py
> ====
> from airflow import DAG
> from airflow.operators.bash_operator import BashOperator
> from airflow.operators.dummy_operator import DummyOperator
> from datetime import datetime, timedelta
>
> default_args = {
> 'owner': 'airflow',
> 'depends_on_past': False,
> 'start_date': datetime(2016,10,5,19),
> 'email': ['airflow@airflow.com'],
> 'email_on_failure': False,
> 'email_on_retry': False,
> 'retries': 1,
> 'retry_delay': timedelta(seconds=1),
> }
>
> dag = DAG('tutorial', default_args=default_args, schedule_interval='@once')
>
> task1 = BashOperator(
> task_id='first_one',
> bash_command='date',
> dag=dag)
>
> task2 = BashOperator(
> task_id='second_one',
> bash_command='this_should_not_work',
> dag=dag)
>
> task2.set_upstream(task1)
>
>
>
> task3 = BashOperator(
> task_id='third_one',
> bash_command='random_command_third',
> dag=dag)
>
> task3.set_upstream(task2)
>
> fail_task = DummyOperator(
> task_id='one_failed',
> trigger_rule='one_failed',
> dag=dag)
>
> fail_task.set_upstream([task1,task2,task3])
>
> > On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org> wrote:
> >
> > Bolke, can you describe the current state of the alpha 2 release? I saw
> > some comments from Alex yesterday about celery instability. If I'm
> running
> > on LocalExecutor, should I be seeing any issues?
> >
> > On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com>
> wrote:
> >
> >> Hi All,
> >>
> >> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~
> bolke/ <
> >> https://people.apache.org/~bolke/> .
> >>
> >> Note: This still cannot be considered an Apache release. Working on
> this.
> >>
> >> This build is signed (note it is served over https).
> >>
> >> Changes are in the area of scheduler stability.
> >>
> >> - Bolke
>
>
Re: Airflow 1.8.0 alpha 2
Posted by Bolke de Bruin <bd...@gmail.com>.
Some issues remain:
* one_failed not executed as dag run is marked failed seemingly prematurely (@chris yes you should see this, see below for an example that is not working properly), confirmed regression
* celery instability Alex
* Wrong DAG state after failure inside branch
So Alpha 2 is definitely not ready for production, but please do put in your canary dags and let them run. I am still quite concerned about the scheduler integrity and stability.
- Bolke
one_failed_not_executed.py
====
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.dummy_operator import DummyOperator
from datetime import datetime, timedelta
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'start_date': datetime(2016,10,5,19),
'email': ['airflow@airflow.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(seconds=1),
}
dag = DAG('tutorial', default_args=default_args, schedule_interval='@once')
task1 = BashOperator(
task_id='first_one',
bash_command='date',
dag=dag)
task2 = BashOperator(
task_id='second_one',
bash_command='this_should_not_work',
dag=dag)
task2.set_upstream(task1)
task3 = BashOperator(
task_id='third_one',
bash_command='random_command_third',
dag=dag)
task3.set_upstream(task2)
fail_task = DummyOperator(
task_id='one_failed',
trigger_rule='one_failed',
dag=dag)
fail_task.set_upstream([task1,task2,task3])
> On 4 Jan 2017, at 20:35, Chris Riccomini <cr...@apache.org> wrote:
>
> Bolke, can you describe the current state of the alpha 2 release? I saw
> some comments from Alex yesterday about celery instability. If I'm running
> on LocalExecutor, should I be seeing any issues?
>
> On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com> wrote:
>
>> Hi All,
>>
>> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~bolke/ <
>> https://people.apache.org/~bolke/> .
>>
>> Note: This still cannot be considered an Apache release. Working on this.
>>
>> This build is signed (note it is served over https).
>>
>> Changes are in the area of scheduler stability.
>>
>> - Bolke
Re: Airflow 1.8.0 alpha 2
Posted by Chris Riccomini <cr...@apache.org>.
Bolke, can you describe the current state of the alpha 2 release? I saw
some comments from Alex yesterday about celery instability. If I'm running
on LocalExecutor, should I be seeing any issues?
On Wed, Jan 4, 2017 at 8:20 AM, Bolke de Bruin <bd...@gmail.com> wrote:
> Hi All,
>
> I have put up Airflow 1.8.0 alpha 2 in https://people.apache.org/~bolke/ <
> https://people.apache.org/~bolke/> .
>
> Note: This still cannot be considered an Apache release. Working on this.
>
> This build is signed (note it is served over https).
>
> Changes are in the area of scheduler stability.
>
> - Bolke