You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "lanxinxs (via GitHub)" <gi...@apache.org> on 2025/06/09 08:19:47 UTC

[I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

lanxinxs opened a new issue, #51521:
URL: https://github.com/apache/airflow/issues/51521

   ### Apache Airflow version
   
   3.0.1
   
   ### If "Other Airflow 2 version" selected, which one?
   
   _No response_
   
   ### What happened?
   
               xcoms = context['ti'].xcom_pull(
                   run_id=run_id,
                   task_ids=task_ids,
                   key=None,
               )
   
   the above code will report error said key cannot be None:
   `Key`
   `Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]`
   
   ### What you think should happen instead?
   
   should remove the key filter if key=None or if users don't pass key parameter.
   In Airflow 2 this works.
   
   ### How to reproduce
   
               xcoms = context['ti'].xcom_pull(
                   run_id=run_id,
                   task_ids=task_ids,
                   key=None,
               )
   
   ### Operating System
   
   ubuntu
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Official Apache Airflow Helm Chart
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-3550979103

   @seabass011 are you actively working on this or do you want me to take a look?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "lanxinxs (via GitHub)" <gi...@apache.org>.
lanxinxs commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-2960988909

   > Hi [@lanxinxs](https://github.com/lanxinxs) [@jroachgolf84](https://github.com/jroachgolf84) [@sjyangkevin](https://github.com/sjyangkevin) —
   > 
   > I’d like to **take ownership of this fix**. Plan:
   > 
   > * ✅ First, reproduce the error on 3.0.1 to confirm the validator path.
   > * 🔧 Patch: if `key is None`, translate to the literal `"return_value"` (restoring Airflow 2 behaviour).
   > * 🧪 Add a unit test covering the legacy case.
   > * 📝 Tweak the docstring so future readers know `None → "return_value"`.
   > 
   > Two-click reply:
   > 
   > 1. 👍 = “sounds good, go for it”
   > 2. ➡️ Anything else you’d like double-checked (e.g., back-port label for 3.0.3)?
   > 
   > Happy to adjust if you see a better approach—just let me know.
   > 
   > Thanks for all your work on Airflow!
   > 
   > * Sebastian (UTC-7)
   
   my expected behaviour is:
   if `key is None`, translate to 'get all xcoms no matter what their keys are', that is, if run_id is given, then pull all xcoms in this dag run; if run_id and task_ids are given, then pull all xcoms within these tasks in this dag run.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-3986453148

   @seabass011 We are unassigning you from this issue as part of our updated [assignment policy](https://github.com/apache/airflow/blob/main/contributing-docs/04_how_to_contribute.rst#contribute-code-changes).
   
   This is not meant to discourage your contribution — quite the opposite! You are still very welcome to work on this issue and submit a PR for it. Simply comment that you are working on it and open a PR when ready.
   
   We found that formal assignments were not working well, as they often prevented others from contributing when the assignee was not actively working on the issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "lanxinxs (via GitHub)" <gi...@apache.org>.
lanxinxs commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-3204048501

   Hi @seabass011 @jroachgolf84 , any update on this?
   https://github.com/apache/airflow/issues/51821 is not the same issue, it fix the `task_ids` parameter not `key`
   
   what I expect is that when I use
   ```
           xcoms = context['ti'].xcom_pull(
               run_id=<run_id>,
               task_ids=<task_ids>,
               key=None,
           )
   ```
   I expect to get all xcoms of these tasks in this dag run,
   means if `key=None`, it should return all xcoms regardless of their xcom key,
   
   the sql statement may be like:
   
   ```
   SELECT * from xcom
   WHERE run_id=run_id and task_ids in [...]
   ```
   no filter on xcom key


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "seabass011 (via GitHub)" <gi...@apache.org>.
seabass011 commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-2957294030

   Hi @lanxinxs @jroachgolf84 @sjyangkevin —
   
   I’d like to **take ownership of this fix**. Plan:
   
   * ✅ First, reproduce the error on 3.0.1 to confirm the validator path.
   * 🔧 Patch: if `key is None`, translate to the literal `"return_value"` (restoring Airflow 2 behaviour).
   * 🧪 Add a unit test covering the legacy case.
   * 📝 Tweak the docstring so future readers know `None → "return_value"`.
   
   Two-click reply:
   
   1. 👍 = “sounds good, go for it”
   2. ➡️ Anything else you’d like double-checked (e.g., back-port label for 3.0.3)?
   
   Happy to adjust if you see a better approach—just let me know.
   
   Thanks for all your work on Airflow!
   
   - Sebastian (UTC-7) 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-2955514924

   Is this for the scenario where the key may be something like `return_value`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "sjyangkevin (via GitHub)" <gi...@apache.org>.
sjyangkevin commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-2956143919

   Trying to get more understanding about it. From my understanding, the `key` is to pull xcoms from a task that is associated with that `key`. What would be the use case to set it to `None`? Or you want to pull all the xcoms pushed by any task?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "lanxinxs (via GitHub)" <gi...@apache.org>.
lanxinxs commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-2960983868

   > Trying to get more understanding about it. From my understanding, the `key` is to pull xcoms from a task that is associated with that `key`. What would be the use case to set it to `None`? Or you want to pull all the xcoms pushed by any task?
   
   yes, key=None means I want to pull all the xcoms


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] xcom_pull doesn't allow passing key=None or not specifying key in Airflow 3 [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #51521:
URL: https://github.com/apache/airflow/issues/51521#issuecomment-3550978522

   @lanxinxs have you tried it with 3.1.3? Does the issue persist?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org