You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/02 14:59:13 UTC
[GitHub] [airflow] ielizarov opened a new issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
ielizarov opened a new issue #12757:
URL: https://github.com/apache/airflow/issues/12757
Airflow v2.0b3
Kubernetes v1.19.3
Discovered issue while testing KubernetesPodOperator (haven't tested with other operator).
If I create a multiline string using """ """", add some variables inside (Jinja templating), then use this string as an argument to KubernetesPodOperator:
- In Graph View DAG is not visible (just gray area where it should be a digraph);
- in browser's web console i see the following error:
`Uncaught TypeError: node is undefined
preProcessGraph http://localhost:8080/static/dist/dagre-d3.min.js:103
preProcessGraph http://localhost:8080/static/dist/dagre-d3.min.js:103
fn http://localhost:8080/static/dist/dagre-d3.min.js:103
call http://localhost:8080/static/dist/d3.min.js:3
draw http://localhost:8080/graph?dag_id=demo_base_indexing&execution_date=mydate
expand_group http://localhost:8080/graph?dag_id=mydag&execution_date=mydate
<anonymous> http://localhost:8080/graph?dag_id=mydag&execution_date=mydate`
Tree view works without issues in this case. The DAG succeeds.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil removed a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil removed a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739068523
I am unable to replicate this:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739241526
Thanks for the quick fix @yuqian90 and yes I too agree about making it explicit.
I am trying to think of the best place to show the error i.e. should we fail it during parsing the DAG itself or during serialization.
Let me know if you both any thoughts or a fix in mind @yuqian90 and @XD-DENG
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ielizarov edited a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
ielizarov edited a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738040222
@kaxil Sure, here it is. I simplified my DAG, this version gives the same error exactly. You'll need to add TEST_VAR to Airflow DB.
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
if __name__ == "__main__":
dag.cli()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738014833
@ielizarov Can you post your DAG please for us to replicate the exact error?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ielizarov removed a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
ielizarov removed a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738040222
@kaxil Sure, here it is. I simplified my DAG, this version gives the same error exactly. You'll need to add TEST_VAR to Airflow DB.
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
if __name__ == "__main__":
dag.cli()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739150904
I agree with @yuqian90 . It should be explicit. Users should either specify `dag` in all operators/sensors, or they use contest manager to avoid that.
To keep consistent behaviour with 1.10.*, I think it's ok to proceed fixing this as a bug. But later I would suggest to consider such tasks as "import errors"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738042863
Thanks @ielizarov
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739089385
@yuqian90 / @houqp If you have some time can you help looking into this issue, it looks like it has something to do with the TaskGroup change in graph.html after some debugging.
Here is the DAG (notice the 2nd task does is not directly assigned to a DAG but because of the dependency of the other task it works fine on Tree View). It used to work on Graph view too until the TaskGroup change in `airflow/www/templates/airflow/graph.html` file:
Error:
Tree View:
DAG:
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738677235
Yes, even without the `dag=dag` it shouldn't be able to cause a crash in the webserver like this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil removed a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil removed a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739071256
I am not able to reproduce this with the DAG you had provided:
Your DAG:
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
dag=dag,
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
if __name__ == "__main__":
dag.cli()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil edited a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739089385
@yuqian90 / @houqp If you have some time can you help looking into this issue, it looks like it has something to do with the TaskGroup change in graph.html after some debugging.
Here is the DAG (notice the 2nd task does is not directly assigned to a DAG but because of the dependency of the other task it works fine on Tree View). It used to work on Graph view too until the TaskGroup change in `airflow/www/templates/airflow/graph.html` file:
Error:
Tree View:
DAG:
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
```
Graph View:
Same DAG on 1.10.14:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil edited a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739071256
I am not able to reproduce this with the DAG you had provided:
Your DAG:
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
dag=dag,
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
if __name__ == "__main__":
dag.cli()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil edited a comment on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739089385
@yuqian90 / @houqp If you have some time can you help looking into this issue, it looks like it has something to do with the TaskGroup change in graph.html after some debugging.
Here is the DAG (notice the 2nd task does is not directly assigned to a DAG but because of the dependency of the other task it works fine on Tree View). It used to work on Graph view too until the TaskGroup change in `airflow/www/templates/airflow/graph.html` file:
Error:
Tree View:
DAG:
```python
"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
```
Graph View:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ielizarov commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
ielizarov commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738044191
@kaxil My bad, it seems I forgot to add "dag=dag" to this particular operator, and it led to this error.
Perhaps there should be more clear behavior (i.e. DAG should still be displayed in this case)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739071256
I am not able to reproduce this:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ielizarov commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
ielizarov commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-738040222
@kaxil Sure, here it is. I simplified my DAG, this version gives the same error exactly. You'll need to add TEST_VAR to Airflow DB.
`"""Demo DAG"""
from datetime import timedelta
from airflow import DAG
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
from airflow.utils.dates import days_ago
args = {
'owner': 'airflow',
}
dag = DAG(
dag_id='demo_test',
default_args=args,
schedule_interval='0 0 * * *',
start_date=days_ago(2),
dagrun_timeout=timedelta(minutes=60),
tags=['demo'],
params={
"pod_op_image": "localhost:5000/my-image:latest",
"pod_op_image_pull_policy": "IfNotPresent",
},
)
env_params = {
"TEST_ENV_VAR": "TEST ENV VAR",
}
normal_op = KubernetesPodOperator(
dag=dag,
name='normal_op',
task_id='normal_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
env_vars=env_params,
cmds=["/bin/bash", "-c"],
arguments=["echo $TEST_ENV_VAR"],
)
#below is templated multi-string var which is used as argument further
templated_command = """
echo "{{ var.value.TEST_VAR}}"
echo "{{ params.my_param }}"
"""
templated_op = KubernetesPodOperator(
name='templated_op',
task_id='templated_op',
namespace='airflow',
image="{{ dag_run.conf['pod_op_image'] }}",
image_pull_policy="{{ dag_run.conf['pod_op_image_pull_policy'] }}",
cmds=["bash", "-c"],
arguments=[templated_command],
params={'my_param': 'Passed_Parameter'},
)
normal_op >> templated_op
if __name__ == "__main__":
dag.cli()
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-737283467
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739068523
I am unable to replicate this:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] yuqian90 commented on issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
yuqian90 commented on issue #12757:
URL: https://github.com/apache/airflow/issues/12757#issuecomment-739147569
Thanks @kaxil and @ielizarov. This issue can be reproduced with a simpler example like this. I put up a fix in https://github.com/apache/airflow/pull/12829
```python
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.utils.dates import days_ago
dag = DAG(dag_id='example_graph_view_issue', start_date=days_ago(2))
op1 = DummyOperator(task_id='op1', dag=dag)
op2 = DummyOperator(task_id='op2')
op1 >> op2
```
While this is indeed an inconsistent behaviour and the fix is straightforward, I wonder why we wanted to support ambiguous usage like this. It would be much cleaner for the user to set the dag properly for each task. Or they can also use context manager to create the DAG to avoid this issue. I find the following much cleaner:
```python
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.utils.dates import days_ago
with DAG(dag_id='example_graph_view_issue', start_date=days_ago(2)) as dag:
op1 = DummyOperator(task_id='op1')
op2 = DummyOperator(task_id='op2')
op1 >> op2
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed issue #12757: Graph View is empty when Operator has multiline string in args (v2.0)
Posted by GitBox <gi...@apache.org>.
kaxil closed issue #12757:
URL: https://github.com/apache/airflow/issues/12757
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org