[PR] Speed up provider asset change detection by pruning dependency directories [airflow]

["Andrushika (via GitHub)" <gi...@apache.org>]

Andrushika opened a new pull request, #72784:
URL: https://github.com/apache/airflow/pull/72784

   ## Why
   
   Before each `breeze start-airflow`, `compile_provider_assets.py` hashes the provider `www` directory to decide if the plugin UI needs a rebuild. That directory also holds `node_modules` and `.pnpm-store` (the pnpm cache lands there because our lint hooks set `store-dir` to a relative path), so the hash excludes both. 
   
   But the exclusion is a regexp over the result of `rglob`, which means every path under them is still listed and `resolve()`  first, then thrown away. That is about 2s per provider, and common.ai runs on every start. 
   
   #72783 fixed the same thing in `compile_ui_assets.py`.
   
   ## What
   
   Walk with `os.walk` and prune the two directories, so nothing under them is listed or resolved. Hash time for common.ai and edge3 goes from about 2s to under 10ms each. The hash order changes, so each provider rebuilds once after this lands.
   
   related: #72783
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Claude Code (Fable 5.1)
   
   Generated-by: Claude Code (Fable 5.1) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Speed up provider asset change detection by pruning dependency directories [airflow]

["github-actions[bot] (via GitHub)" <gi...@apache.org>]

github-actions[bot] commented on PR #72784:
URL: https://github.com/apache/airflow/pull/72784#issuecomment-5600968729

   ### Backport successfully created: v3-3-test
   
   Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
   the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
   
   In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
   
   <table>
           <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>✅</td>
           <td>v3-3-test</td>
           <td><a href="https://github.com/apache/airflow/pull/72800"><img src="https://img.shields.io/badge/PR-72800-blue" alt="PR Link"></a></td>
       </tr>
   </table>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Speed up provider asset change detection by pruning dependency directories [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk merged PR #72784:
URL: https://github.com/apache/airflow/pull/72784


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org