You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/03 03:57:34 UTC
[GitHub] [airflow-client-python] er1shivam opened a new issue #27: get_dag_details api is broken
er1shivam opened a new issue #27:
URL: https://github.com/apache/airflow-client-python/issues/27
Client version : 2.1.0
Code :
```with client.ApiClient(configuration) as api_client:
api_instance = dag_api.DAGApi(api_client)
dag_id = "dag_id_example"
try:
# Get a simplified representation of DAG
api_response = api_instance.get_dag_details(dag_id)
pprint(api_response)
except client.ApiException as e:
print("Exception when calling DAGApi->get_dag_details: %s\n" % e)
```
Error:
`ApiValueError: Invalid inputs given to generate an instance of 'DAGDetailAllOf`. The input data was invalid for the allOf schema 'DAGDetailAllOf' in the composed schema 'DAGDetail'. Error=Invalid type for variable 'dag_run_timeout'. Required value type is TimeDelta and passed type was NoneType at ['received_data']['dag_run_timeout']
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] er1shivam commented on issue #27: get_dag_details api is broken
Posted by GitBox <gi...@apache.org>.
er1shivam commented on issue #27:
URL: https://github.com/apache/airflow-client-python/issues/27#issuecomment-892786390
I am using this [DAG](https://airflow.apache.org/docs/apache-airflow/stable/tutorial.html#example-pipeline-definition) from the example on official Airflow website. Please visit [this link...](https://airflow.apache.org/docs/apache-airflow/stable/tutorial.html#example-pipeline-definition)
```
from datetime import timedelta
from textwrap import dedent
# The DAG object; we'll need this to instantiate a DAG
from airflow import DAG
# Operators; we need this to operate!
from airflow.operators.bash import BashOperator
from airflow.utils.dates import days_ago
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'email': ['airflow@example.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
# 'queue': 'bash_queue',
# 'pool': 'backfill',
# 'priority_weight': 10,
# 'end_date': datetime(2016, 1, 1),
# 'wait_for_downstream': False,
# 'dag': dag,
# 'sla': timedelta(hours=2),
# 'execution_timeout': timedelta(seconds=300),
# 'on_failure_callback': some_function,
# 'on_success_callback': some_other_function,
# 'on_retry_callback': another_function,
# 'sla_miss_callback': yet_another_function,
# 'trigger_rule': 'all_success'
}
with DAG(
'tutorial',
default_args=default_args,
description='A simple tutorial DAG',
schedule_interval=timedelta(days=1),
start_date=days_ago(2),
tags=['example'],
) as dag:
# t1, t2 and t3 are examples of tasks created by instantiating operators
t1 = BashOperator(
task_id='print_date',
bash_command='date',
)
t2 = BashOperator(
task_id='sleep',
depends_on_past=False,
bash_command='sleep 5',
retries=3,
)
t1.doc_md = dedent(
"""\
#### Task Documentation
You can document your task using the attributes `doc_md` (markdown),
`doc` (plain text), `doc_rst`, `doc_json`, `doc_yaml` which gets
rendered in the UI's Task Instance Details page.
"""
)
dag.doc_md = __doc__ # providing that you have a docstring at the beggining of the DAG
dag.doc_md = """
This is a documentation placed anywhere
""" # otherwise, type it like this
templated_command = dedent(
"""
{% for i in range(5) %}
echo "{{ ds }}"
echo "{{ macros.ds_add(ds, 7)}}"
echo "{{ params.my_param }}"
{% endfor %}
"""
)
t3 = BashOperator(
task_id='templated',
depends_on_past=False,
bash_command=templated_command,
params={'my_param': 'Parameter I passed in'},
)
t1 >> [t2, t3]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] er1shivam commented on issue #27: get_dag_details api is broken
Posted by GitBox <gi...@apache.org>.
er1shivam commented on issue #27:
URL: https://github.com/apache/airflow-client-python/issues/27#issuecomment-892786390
I am using this [DAG](https://airflow.apache.org/docs/apache-airflow/stable/tutorial.html#example-pipeline-definition) from the example on official Airflow website. Please visit [this link...](https://airflow.apache.org/docs/apache-airflow/stable/tutorial.html#example-pipeline-definition)
```
from datetime import timedelta
from textwrap import dedent
# The DAG object; we'll need this to instantiate a DAG
from airflow import DAG
# Operators; we need this to operate!
from airflow.operators.bash import BashOperator
from airflow.utils.dates import days_ago
# These args will get passed on to each operator
# You can override them on a per-task basis during operator initialization
default_args = {
'owner': 'airflow',
'depends_on_past': False,
'email': ['airflow@example.com'],
'email_on_failure': False,
'email_on_retry': False,
'retries': 1,
'retry_delay': timedelta(minutes=5),
# 'queue': 'bash_queue',
# 'pool': 'backfill',
# 'priority_weight': 10,
# 'end_date': datetime(2016, 1, 1),
# 'wait_for_downstream': False,
# 'dag': dag,
# 'sla': timedelta(hours=2),
# 'execution_timeout': timedelta(seconds=300),
# 'on_failure_callback': some_function,
# 'on_success_callback': some_other_function,
# 'on_retry_callback': another_function,
# 'sla_miss_callback': yet_another_function,
# 'trigger_rule': 'all_success'
}
with DAG(
'tutorial',
default_args=default_args,
description='A simple tutorial DAG',
schedule_interval=timedelta(days=1),
start_date=days_ago(2),
tags=['example'],
) as dag:
# t1, t2 and t3 are examples of tasks created by instantiating operators
t1 = BashOperator(
task_id='print_date',
bash_command='date',
)
t2 = BashOperator(
task_id='sleep',
depends_on_past=False,
bash_command='sleep 5',
retries=3,
)
t1.doc_md = dedent(
"""\
#### Task Documentation
You can document your task using the attributes `doc_md` (markdown),
`doc` (plain text), `doc_rst`, `doc_json`, `doc_yaml` which gets
rendered in the UI's Task Instance Details page.
"""
)
dag.doc_md = __doc__ # providing that you have a docstring at the beggining of the DAG
dag.doc_md = """
This is a documentation placed anywhere
""" # otherwise, type it like this
templated_command = dedent(
"""
{% for i in range(5) %}
echo "{{ ds }}"
echo "{{ macros.ds_add(ds, 7)}}"
echo "{{ params.my_param }}"
{% endfor %}
"""
)
t3 = BashOperator(
task_id='templated',
depends_on_past=False,
bash_command=templated_command,
params={'my_param': 'Parameter I passed in'},
)
t1 >> [t2, t3]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] er1shivam commented on issue #27: get_dag_details api is broken
Posted by GitBox <gi...@apache.org>.
er1shivam commented on issue #27:
URL: https://github.com/apache/airflow-client-python/issues/27#issuecomment-892322524
I will share it in few hours when I log back in.
But as you said regarding dag_run_timeout, I haven't defined this in my dag as it is not a REQUIRED parameter. So I believe python api should be able to handle NoneType if this is not defined in the DAG.
Also just wanted to point out that REST API successfully retrieve the details of the same DAG.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow-client-python] ephraimbuddy commented on issue #27: get_dag_details api is broken
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #27:
URL: https://github.com/apache/airflow-client-python/issues/27#issuecomment-891592026
Looking at the error message, it seems the problem is from your DAG, dagrun_timeout should be a timedelta. It seems you have it misconfigured. Can you share the DAG?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org