Re: [I] Dynamic task mapping- Expand not working when provided value from a task [airflow]

["uranusjr (via GitHub)" <gi...@apache.org>]

uranusjr commented on issue #46580:
URL: https://github.com/apache/airflow/issues/46580#issuecomment-2687010299

   When Airflow pushes an XCom for task expansion, the worker that creates the upstream XCom calculates how long the list is (in this case `[3, 6, 9]` so it’s 3), and write that number into a separate table TaskMap
   
   This allows the scheduler to simply reads that table when it needs to expand the task into TIs. The scheduler cannot read the XCom directly because it may contain huge data and cripple the process (also deserialisation can be unsafe depending on the XCom backend).
   
   I’m guessing maybe something along this pipeline is not working correctly, and Airflow is complaining about the TaskMap record is missing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org