[GitHub] [airflow] vanchaxy opened a new issue, #24483: Allow context usage in user_defined_macros

[GitBox <gi...@apache.org>]

vanchaxy opened a new issue, #24483:
URL: https://github.com/apache/airflow/issues/24483

   ### Description
   
   Allow context usage in user_defined_macros. For this move `set_current_context` context manager from [wraping _execute_task](https://github.com/apache/airflow/blob/0c41f437674f135fe7232a368bf9c198b0ecd2f0/airflow/models/taskinstance.py#L1595) to [wraping _execute_task_with_callbacks](https://github.com/apache/airflow/blob/0c41f437674f135fe7232a368bf9c198b0ecd2f0/airflow/models/taskinstance.py#L1443).
   
   ### Use case/motivation
   
   After this change, we will be able to use `get_current_context` while rendering template fields. 
   
   Example use case. We have dag with the first task that prepares configuration for this run and after this, all other task uses this config to render templates. We have no control over other task codes, as it's providers operators etc. 
   
   After this change we can define class like:
   
   ```
   class XcomResult:
       def __init__(self, task_id):
           self.task_id = task_id
           self._result = None
   
       def __get__(self, item):
           if self._result is None:
               context = get_current_context()
               self._result = context['ti'].xcom_pull(task_ids=self.task_id)
           self._result.__get__(item)
   
   user_defined_macros = {"config": XcomResult("prepare_config_task_id")}
   ```
   Then we can use this in all templates as `config.needed_key`. 
   
   For now, if we want something similar we need to define function like:
   ```
   def built_get_xcom_result(task_id):
       def get_config(ti):
           return ti.xcom_pull(task_ids=task_id)
       
   user_defined_macros = {"get_config": built_get_xcom_result("prepare_config_task_id")}
   ```
   and use it in all templates as `get_config(ti).needed_key`
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] vanchaxy commented on issue #24483: Allow context usage in user_defined_macros

[GitBox <gi...@apache.org>]

vanchaxy commented on issue #24483:
URL: https://github.com/apache/airflow/issues/24483#issuecomment-1159822437

   I will try to propose a PR. Just need some time to configure env and read guidelines.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] potiuk commented on issue #24483: Allow context usage in user_defined_macros

[GitBox <gi...@apache.org>]

potiuk commented on issue #24483:
URL: https://github.com/apache/airflow/issues/24483#issuecomment-1159814977

   You seem to know what you want to do - would you like to propose a PR for that? Discussing on actual PR would be much more productive and you could become one of the > 2K contributors to airflow this way.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #24483: Allow context usage in user_defined_macros

[GitBox <gi...@apache.org>]

boring-cyborg[bot] commented on issue #24483:
URL: https://github.com/apache/airflow/issues/24483#issuecomment-1156888630

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org