[GitHub] [airflow] ashb commented on pull request #12228: taskgroup decorator (#11870)

[GitBox <gi...@apache.org>]

ashb commented on pull request #12228:
URL: https://github.com/apache/airflow/pull/12228#issuecomment-749004546


   Does this actually need to be a decorator? How about this:
   
   ```python
   @task
   collect_dataset(...)
     ...
   
   @task
   def train_model(...)
      ...
   
   @task_group
   def movielens_model_pipeline(learning_rate: int, feature_list: list, dataset: XComArg):  
     dataset = filter_dataset(dataset, feature_list)
     train_model(dataset, learning_rate)
   
   @dag
   def movielens_hpt(dataset_path: str, feature_list:list=['year', 'director']):
     dataset = load_dataset(dataset_path)
   
     # NEW HERE
     with dag.taskgroup():
        for i in range(0.1, 0.9, step=0.1):
          movielens_model_pipeline(i, feature_list, dataset)
     
     decide_best_model(...)
   ```
   
   Or nested groups:
   
   ```python
   with dag.taskgroup() as group
     with group.taskgroup(name="Inner group"):
        for i in range(0.1, 0.9, step=0.1):
          movielens_model_pipeline(i, feature_list, dataset)
   ```
   
   What do you think @VBhojawala @casassg ?
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org