You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/06/24 03:29:17 UTC
[GitHub] [airflow] aspain opened a new pull request, #23424: update `task-generated mapping` example
aspain opened a new pull request, #23424:
URL: https://github.com/apache/airflow/pull/23424
current example results in `TypeError: 'int' object is not iterable` for mapped tasks 0 & 1. Updating example with values that can be used with list()
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragement file, named `{pr_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23424:
URL: https://github.com/apache/airflow/pull/23424#issuecomment-1165157239
The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aspain commented on pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
aspain commented on PR #23424:
URL: https://github.com/apache/airflow/pull/23424#issuecomment-1115383602
🤦 thanks @jhtimmins sorry about that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr merged pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
uranusjr merged PR #23424:
URL: https://github.com/apache/airflow/pull/23424
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] aspain commented on pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
aspain commented on PR #23424:
URL: https://github.com/apache/airflow/pull/23424#issuecomment-1115428277
I copied the example DAG as-is from the [Task-generated Mapping](https://airflow.apache.org/docs/apache-airflow/2.3.0/concepts/dynamic-task-mapping.html#task-generated-mapping) section which results in the first 2 mapped tasks failing:
Task log error:
```
[2022-05-02, 22:20:26 UTC] {taskinstance.py:1888} ERROR - Task failed with exception
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/airflow/decorators/base.py", line 179, in execute
return_value = super().execute(context)
File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 171, in execute
return_value = self.execute_callable()
File "/usr/local/lib/python3.9/site-packages/airflow/operators/python.py", line 189, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/usr/local/airflow/dags/dyn2.py", line 15, in consumer
print(list(arg))
TypeError: 'int' object is not iterable
[2022-05-02, 22:20:26 UTC] {taskinstance.py:1394} INFO - Marking task as FAILED. dag_id=dynamic-map2, task_id=consumer, map_index=0, execution_date=20220412T000000, start_date=20220502T222026, end_date=20220502T222026
[2022-05-02, 22:20:26 UTC] {standard_task_runner.py:92} ERROR - Failed to execute job 909 for task consumer ('int' object is not iterable; 390)
[2022-05-02, 22:20:26 UTC] {local_task_job.py:156} INFO - Task exited with return code 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #23424:
URL: https://github.com/apache/airflow/pull/23424#issuecomment-1115432217
Ah, I know where the problem is. `make_list` is fine, but `consumer` should not do `list(arg)`. We should change it to simply `print(arg)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #23424: update `task-generated mapping` example
URL: https://github.com/apache/airflow/pull/23424
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #23424:
URL: https://github.com/apache/airflow/pull/23424#issuecomment-1159313698
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
uranusjr commented on PR #23424:
URL: https://github.com/apache/airflow/pull/23424#issuecomment-1165182279
I’m just going to merge this since it’s a one-line documentation change that I should have gotten right in the first place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jhtimmins commented on a diff in pull request #23424: update `task-generated mapping` example
Posted by GitBox <gi...@apache.org>.
jhtimmins commented on code in PR #23424:
URL: https://github.com/apache/airflow/pull/23424#discussion_r863171762
##########
docs/apache-airflow/concepts/dynamic-task-mapping.rst:
##########
@@ -152,7 +152,7 @@ Up until now the examples we've shown could all be achieved with a ``for`` loop
def make_list():
# This can also be from an API call, checking a database, -- almost anything you like, as long as the
# resulting list/dictionary can be stored in the current XCom backend.
- return [1, 2, {"a": "b"}, "str"]
+ return [[1, 2], ("a", "b") , {"a": "b"}, "str"]
Review Comment:
```suggestion
return [[1, 2], ("a", "b"), {"a": "b"}, "str"]
```
The extra space is causing static tests to fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org