You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "getaaron (via GitHub)" <gi...@apache.org> on 2023/11/28 18:01:55 UTC
[I] There's no way to pass params to dag.test() [airflow]
getaaron opened a new issue, #35930:
URL: https://github.com/apache/airflow/issues/35930
### Description
I'm writing a new dag which takes params and trying out calling `dag.test()` in my unit test. The basic dag could look like:
```python
emotions = ['elated', 'ennui']
@task
def some_task(emotion: str, date: str):
url = f"http://example.com/do_something/{date}?emotion={emotion}"
response = requests.post(url)
if response.status_code != 200:
raise AirflowException(f"Request failed with status {response.status_code}")
@dag(default_args=default_args,
schedule=None,
params={"emotion": Param(None, enum=emotions),
"date": Param(None, type="string", format="date")})
def process_my_emotion():
some_task('{{ params.emotion }}', '{{ params.date }}')
```
In my test, I want to call `dag.test() with an `emotion` and `date` and assert the correct URL is formed. I tried this:
```python
@patch("requests.post")
@patch("time.sleep")
@freeze_time("2024-01-01")
def test_process_my_emotion(mock_sleep, mock_post, manual_dag):
mock_post.return_value = Mock(status_code=200)
manual_dag.test(execution_date=datetime(2024, 1, 1),
params={'emotion': 'ennui', 'date': '2024-01-01'})
mock_post.assert_any_call(
f"http://example.com/do_something/2024-01-01?emotion=ennui"
)
```
This fails with:
> TypeError: test() got an unexpected keyword argument 'params'
Looking at
https://github.com/apache/airflow/blob/f6962a929b839215613d1b6f99f43511759c1e5b/airflow/models/dag.py#L2769-L2776
I don't see a way to pass params in.
If I'm correct, the feature I'm requesting is a way to pass params in. If I'm incorrect, I'm requesting an improvement to the docs.
### Use case/motivation
I would like to test dags that use params by calling `dag.test()`
### Related issues
I didn't find any.
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] There's no way to pass params to dag.test() [airflow]
Posted by "getaaron (via GitHub)" <gi...@apache.org>.
getaaron commented on issue #35930:
URL: https://github.com/apache/airflow/issues/35930#issuecomment-1834462654
@RNHTTR Can you help me get any guidance on implementing this? I'm happy to open a PR but could use some general design guidance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] There's no way to pass params to dag.test() [airflow]
Posted by "RNHTTR (via GitHub)" <gi...@apache.org>.
RNHTTR commented on issue #35930:
URL: https://github.com/apache/airflow/issues/35930#issuecomment-1886097256
@getaaron apologies for the delayed response. It looks like @keithbentrup 's solution ought to work. With that said, Keith's point about [dag.test() docs](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/executor/debug.html#testing-dags-with-dag-test) being is spot on -- updating these docs would be a great idea. Let me know if you'd like some help with that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] There's no way to pass params to dag.test() [airflow]
Posted by "RNHTTR (via GitHub)" <gi...@apache.org>.
RNHTTR closed issue #35930: There's no way to pass params to dag.test()
URL: https://github.com/apache/airflow/issues/35930
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] There's no way to pass params to dag.test() [airflow]
Posted by "keithbentrup (via GitHub)" <gi...@apache.org>.
keithbentrup commented on issue #35930:
URL: https://github.com/apache/airflow/issues/35930#issuecomment-1886020087
@getaaron Try `dag.test(run_conf={"paramA": "valueA", "paramB": "valueB"})`. The [test() docs](https://airflow.apache.org/docs/apache-airflow/stable/_api/airflow/models/dag/index.html#airflow.models.dag.DAG.test) are vague, but it worked for me when I was looking for the solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org