You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Eason09053360 (via GitHub)" <gi...@apache.org> on 2026/09/13 16:38:07 UTC

[PR] Make airflowctl command generation safe to read more than once [airflow]

Eason09053360 opened a new pull request, #73097:
URL: https://github.com/apache/airflow/pull/73097

   ## Why
   
   `CommandFactory.group_commands` in `airflow-ctl/src/airflowctl/ctl/cli_config.py` is a plain
   `@property`, so every access re-runs the four builder methods. Those builders **append** to
   `self.operations`, `self.commands_map` and `self.group_commands_list` instead of replacing them,
   and the `GroupCommand`s hold the `commands_map` lists by reference. Reading the property a second
   time therefore doubles the command tree:
   
   ```
   read #1:  14 groups,  7 `connections` subcommands
   read #2:  28 groups, 21 `connections` subcommands   (still only 7 distinct names)
   ```
   
   Two groups named `connections` is not something `argparse` accepts, so the second read leaves
   `airflowctl` unable to build its parser at all.
   
   This is preventive — there is no user-visible bug today, because the only access is the single
   import-time one at `cli_config.py:1305`. What makes it worth fixing now is that `command_factory`
   is a module-level singleton any caller can import, and the existing tests construct
   `CommandFactory()` in a dozen places. The likely way in is a routine refactor that shares one
   factory instance across test cases; the resulting failure would point nowhere near this property.
   
   ## What
   
   - `cli_config.py`: `group_commands` becomes a `cached_property`, so the non-idempotent builders
     run once per instance. The docstring records why the caching is load-bearing rather than an
     optimisation, since reverting it to `@property` would keep every current test green.
   - `test_cli_config.py`: adds `test_group_commands_is_stable_across_repeated_access`. Both accesses
     return the same list object, so the test snapshots `(name, len(subcommands))` before the second
     access — comparing the lists directly would compare a list against itself and pass either way.
   
   Nothing mutates the cached list: `merge_commands` and `add_auth_token_to_all_commands` both copy
   before extending and rebuild the namedtuples via `_replace`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make airflowctl command generation safe to read more than once [airflow]

Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on code in PR #73097:
URL: https://github.com/apache/airflow/pull/73097#discussion_r4002488064


##########
airflow-ctl/src/airflowctl/ctl/cli_config.py:
##########
@@ -959,9 +959,15 @@ def _create_group_commands_from_operation(self):
                 )
             )
 
-    @property
+    @cached_property

Review Comment:
   The cache is load-bearing because the builders still append. del factory.group_commands (the documented way to refresh a cached_property) runs them again on already-filled operations / commands_map / group_commands_list and recreates the doubled tree this PR exists to stop.
   
   Zero those three collections at the start of group_commands, then cache or not. After that, invalidation stays safe.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make airflowctl command generation safe to read more than once [airflow]

Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on code in PR #73097:
URL: https://github.com/apache/airflow/pull/73097#discussion_r4002488064


##########
airflow-ctl/src/airflowctl/ctl/cli_config.py:
##########
@@ -959,9 +959,15 @@ def _create_group_commands_from_operation(self):
                 )
             )
 
-    @property
+    @cached_property

Review Comment:
   The cache is load-bearing because the builders still append. del factory.group_commands (the documented way to refresh a cached_property) runs them again on already-filled operations / commands_map / group_commands_list and recreates the doubled tree this PR exists to stop.
   
   Zero those three collections at the start of group_commands, then cache or not. After that, invalidation stays safe.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make airflowctl command generation safe to read more than once [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #73097:
URL: https://github.com/apache/airflow/pull/73097#issuecomment-5665267074

   ### Backport failed to create: airflow-ctl/v0-1-test. View the failure log <a href='https://github.com/apache/airflow/actions/runs/34853276752'> Run details </a>
   
   Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
   the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
   
   In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
   
   <table>
       <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>❌</td>
           <td>airflow-ctl/v0-1-test</td>
           <td><a href="https://github.com/apache/airflow/commit/17d887c87229f3fbae8dcca33b266ae24af514e1"><img src='https://img.shields.io/badge/Commit-17d887c-red' alt='Commit Link'></a></td>
       </tr>
   </table>
   
   You can attempt to backport this manually by running:
   
   ```bash
   cherry_picker 17d887c airflow-ctl/v0-1-test
   ```
   
   This should apply the commit to the airflow-ctl/v0-1-test branch and leave the commit in conflict state marking
   the files that need manual conflict resolution.
   
   After you have resolved the conflicts, you can continue the backport process by running:
   
   ```bash
   cherry_picker --continue
   ```
   
   If you don't have cherry-picker installed, see the [installation guide](https://github.com/apache/airflow/blob/main/dev/README.md#how-to-backport-pr-with-cherry-picker-cli).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Make airflowctl command generation safe to read more than once [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #73097:
URL: https://github.com/apache/airflow/pull/73097


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org