You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Siddharth Anand (JIRA)" <ji...@apache.org> on 2016/05/04 04:08:12 UTC
[jira] [Commented] (AIRFLOW-46) Running Trigger DAG test with
SequentialExecutor yields error
[ https://issues.apache.org/jira/browse/AIRFLOW-46?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270013#comment-15270013 ]
Siddharth Anand commented on AIRFLOW-46:
----------------------------------------
[~bolke][~jlowin] [~maxime.beauchemin@apache.org] I used to be able to access a dag_run from within a python callable's context. Refer to example_trigger_target_dag.py in the example_dags folder.
This allowed me to pass pickled context between a trigger_dag operator and a target dag. What's changed in 1.7.0 that does not allow that. I still see the pickled data in the conf column in the dag_run table.
> Running Trigger DAG test with SequentialExecutor yields error
> -------------------------------------------------------------
>
> Key: AIRFLOW-46
> URL: https://issues.apache.org/jira/browse/AIRFLOW-46
> Project: Apache Airflow
> Issue Type: Bug
> Affects Versions: Airflow 1.7.0
> Environment: virtualenv, python 2.7.9
> Reporter: Siddharth Anand
> Assignee: Siddharth Anand
> Attachments: Screenshot 2016-05-04 02.22.28.png
>
>
> On a fresh install of Airflow 1.7.0, executed the following way :
> * git checkout -b b_1.7.0rc1 1.7.0rc1
> * python setup.py develop
> * airflow initdb (against sqllite and using SequentialExecutor - delete the previous db file)
> * airflow webserver -d
> * airflow scheduler
> * Load examples in the airflow.cfg file and disable all but 2 via the UI as shown below : example_trigger_controller_dag and example_trigger_target_dag
> The controller dag will run and you will see the following line in the scheduler output to show it succeeded :
> {panel}
> [2016-05-04 02:11:47,680] {__init__.py:36} INFO - Using executor SequentialExecutor
> Controller DAG : conditionally_trigger = True
> { 'message': 'Hello World'}
> [2016-05-04 02:11:51,809] {jobs.py:516} INFO - Prioritizing 0 queued jobs
> {panel}
> However, when the target dag runs, you will see :
> {noformat}
> [2016-05-04 02:11:56,924] {sequential_executor.py:27} INFO - Executing command: airflow run example_trigger_target_dag bash_task 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py
> [2016-05-04 02:11:57,870] {__init__.py:36} INFO - Using executor SequentialExecutor
> Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/bash_task/2016-05-04T01:11:47
> [2016-05-04 02:11:59,008] {__init__.py:36} INFO - Using executor SequentialExecutor
> [2016-05-04 02:12:03,131] {sequential_executor.py:27} INFO - Executing command: airflow run example_trigger_target_dag run_this 2016-05-04T01:11:47 --local -sd DAGS_FOLDER/example_dags/example_trigger_target_dag.py
> [2016-05-04 02:12:04,056] {__init__.py:36} INFO - Using executor SequentialExecutor
> Logging into: /Users/siddharth/Projects/airflow/logs/example_trigger_target_dag/run_this/2016-05-04T01:11:47
> [2016-05-04 02:12:05,179] {__init__.py:36} INFO - Using executor SequentialExecutor
> Traceback (most recent call last):
> File "/Users/siddharth/Projects/airflow/venv/bin/airflow", line 6, in <module>
> exec(compile(open(__file__).read(), __file__, 'exec'))
> File "/Users/siddharth/Projects/airflow/airflow/bin/airflow", line 15, in <module>
> args.func(args)
> File "/Users/siddharth/Projects/airflow/airflow/bin/cli.py", line 203, in run
> pool=args.pool,
> File "/Users/siddharth/Projects/airflow/airflow/models.py", line 1067, in run
> result = task_copy.execute(context=context)
> File "/Users/siddharth/Projects/airflow/airflow/operators/python_operator.py", line 65, in execute
> return_value = self.python_callable(*self.op_args, **self.op_kwargs)
> File "/Users/siddharth/Projects/airflow/airflow/example_dags/example_trigger_target_dag.py", line 37, in run_this_func
> print( "Remotely received value of {} for key=message".format(kwargs['dag_run'].conf['message']))
> AttributeError: 'NoneType' object has no attribute 'conf'
> [2016-05-04 02:12:09,318] {jobs.py:516} INFO - Prioritizing
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)