[I] Potentially unintended behaviour with TaskGroups and user-made attributes attached to them [airflow]

["Meehai (via GitHub)" <gi...@apache.org>]

Meehai opened a new issue, #42931:
URL: https://github.com/apache/airflow/issues/42931

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### If "Other Airflow 2 version" selected, which one?
   
   2.5.3
   
   ### What happened?
   
   Is this the intended behaviour? It seems that airflow clones the task group object which is different from regular python semantic, leading to potentially hidden bugs. I got bitten by it and spent a good few hours before realizing. I can try newer airflow versions too, if requested.
   
   ```python
   class RunnerTaskGroup(TaskGroup):
     def __init__(self...):
       super().__init__()
       self.skipped_because_not_trained = True
   
     @task(task_group=self)
     def some_task():
       if random.random() < 0.5:
         self.skipped_because_not_trained = False
       print(self, id(self))
       # <RunnerTaskGroup object at 0x7f7a2d3c5be0> 140162721668064 <- see object address
   
     (
       some_task
     )
   
   .... in main dag .....
   
   with DAG(dag_id=....):
     def update_inference_table_label(_groups):
       for group in _groups:
         print(group, id(group),  group.skipped_because_not_trained)
         # <RunnerTaskGroup object at 0x7f7a2d3cbb50> 140162721692496 <- OTHER OBJECT 
         # group.skipped_because_not_trained=True <- always TRUE
   
     affinities_groups = []
       for affinity in customer.
          affn_runner_tg = RunnerTaskGroup()
          affinities_group.append(affn_runner_tg)
     ( 
       affinities_groups >>
       update_inference_table_label(affinities_groups)
     )
   ```
   
   
   ### What you think should happen instead?
   
   _No response_
   
   ### How to reproduce
   
   run the code above
   
   ### Operating System
   
   Ubuntu 22.04.3 LTS
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Potentially unintended behaviour with TaskGroups and user-made attributes attached to them [airflow]

["boring-cyborg[bot] (via GitHub)" <gi...@apache.org>]

boring-cyborg[bot] commented on issue #42931:
URL: https://github.com/apache/airflow/issues/42931#issuecomment-2407015593

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Potentially unintended behaviour with TaskGroups and user-made attributes attached to them [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk closed issue #42931: Potentially unintended behaviour with TaskGroups and user-made attributes attached to them
URL: https://github.com/apache/airflow/issues/42931


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org