[GitHub] [airflow] shawn-le opened a new issue, #24310: non-TaskFlow expand op_kwargs errors

[GitBox <gi...@apache.org>]

shawn-le opened a new issue, #24310:
URL: https://github.com/apache/airflow/issues/24310

   ### Apache Airflow version
   
   2.3.0
   
   ### What happened
   
   ```python
   from datetime import datetime
   
   from airflow import DAG
   from airflow.decorators import task
   from typing import List, Dict
   from airflow.operators.dummy import DummyOperator
   from airflow.operators.python import PythonOperator
   from airflow.utils.trigger_rule import TriggerRule
   
   def end_function(name):
       print(f"Calling end {name}")
   
   with DAG(dag_id="not_simple_mapping", start_date=datetime(2022, 6, 7), schedule_interval=None) as dag:
   
       @task()
       def show(x: dict):
           print(f"x={x}")
   
       x_list: List[Dict] = [{}]
       end_list: list = []
   
       for i in range(0, 5):
           x_list.append({
               'i': i,
               'i_plus_one': i+1
           })
           end_list.append(f'Task{i}')
   
       start_region_job_task = DummyOperator(
           task_id='Start_Region',
           trigger_rule=TriggerRule.ALL_DONE
       )
       
       main_run = show.expand(x=x_list)
       end_run = PythonOperator.partial(task_id='End_Region', python_callable=end_function, do_xcom_push=False).expand(
           op_kwargs = [{'name': 'Task1'}, {'name': 'Task2'}]
       )
   
       start_region_job_task >> main_run >> end_run
       
   ```
   
   Could not works because of passing list of kwargs
   
   ```bash
     File "/home/sang/airflow-dm-dev/venv/lib/python3.10/site-packages/airflow/serialization/serialized_objects.py", line 343, in _serialize
       return SerializedBaseOperator.serialize_mapped_operator(var)
     File "/home/sang/airflow-dm-dev/venv/lib/python3.10/site-packages/airflow/serialization/serialized_objects.py", line 608, in serialize_mapped_operator
       assert op_kwargs[Encoding.TYPE] == DAT.DICT
   TypeError: list indices must be integers or slices, not Encoding
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "/home/sang/airflow-dm-dev/venv/lib/python3.10/site-packages/airflow/serialization/serialized_objects.py", line 1105, in to_dict
       json_dict = {"__version": cls.SERIALIZER_VERSION, "dag": cls.serialize_dag(var)}
     File "/home/sang/airflow-dm-dev/venv/lib/python3.10/site-packages/airflow/serialization/serialized_objects.py", line 1013, in serialize_dag
       raise SerializationError(f'Failed to serialize DAG {dag.dag_id!r}: {e}')
   airflow.exceptions.SerializationError: Failed to serialize DAG 'not_simple_mapping': list indices must be integers or slices, not Encoding
   ```
   
   ### What you think should happen instead
   
   It's should accept a list of kwargs to trigger dynamic mapping instead of one kwargs
   
   ### How to reproduce
   
   _No response_
   
   ### Operating System
   
   Ubuntu 22.04
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Virtualenv installation
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on issue #24310: non-TaskFlow expand op_kwargs errors

[GitBox <gi...@apache.org>]

uranusjr commented on issue #24310:
URL: https://github.com/apache/airflow/issues/24310#issuecomment-1149479913

   duplicate of #23833


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #24310: non-TaskFlow expand op_kwargs errors

[GitBox <gi...@apache.org>]

boring-cyborg[bot] commented on issue #24310:
URL: https://github.com/apache/airflow/issues/24310#issuecomment-1149391803

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr closed issue #24310: non-TaskFlow expand op_kwargs errors

[GitBox <gi...@apache.org>]

uranusjr closed issue #24310: non-TaskFlow expand op_kwargs errors
URL: https://github.com/apache/airflow/issues/24310


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org