You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2026/03/12 12:37:46 UTC

[PR] Add LLM safety checks, should_report handling, and dynamic model discovery [airflow]

potiuk opened a new pull request, #63440:
URL: https://github.com/apache/airflow/pull/63440

## Summary

- Handle `should_report` flag from LLM assessment for prompt injection, automated spam,
and ToS violations. Potentially flagged PRs default to SKIP, appear first in review
order (yellow), and downgrade to regular "flagged" when the user takes a non-skip action.
- Add defense-in-depth LLM CLI safety: dangerous env var checks, Claude CLI hardened with
`--permission-mode plan` + `--allowedTools` whitelist, Codex CLI with `--sandbox read-only`
+ `--ephemeral`. Interactive confirmation prompt (Y/n/q/d/a) with persistent "always" option.
- Restrict GitHub MCP to read-only tools via `@modelcontextprotocol/server-github --tools`.
Auto-detect `gh auth` status and offer to configure MCP interactively.
- Validate trusted repository and safe `--answer-triage` values before starting LLM threads.
- Rename `--check-mode ci` to `--check-mode api`. Display check mode and provider/model/version
at startup.
- Fetch available models from Anthropic/OpenAI APIs with 24h cache in `.build/llm_models_cache.json`.
- Save LLM errors to temp files instead of printing verbose output inline.
- Add "Potentially flagged for report" row (red) to summary table.

## Test plan

- [ ] Run `breeze pr auto-triage --check-mode both` and verify LLM safety prompt appears
- [ ] Verify `d` shows security details, `a` persists to `.build/llm_confirmed`
- [ ] Test with `--check-mode api` to confirm LLM is skipped
- [ ] Verify should_report PRs appear first in review order with yellow panel
- [ ] Verify taking non-skip action on a report PR clears the report status
- [ ] Check summary table shows red "Potentially flagged for report" count

---

##### Was generative AI tooling used to co-author this PR?

- [X] Yes — Claude Opus 4.6

Generated-by: Claude Opus 4.6 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add LLM safety checks, should_report handling, and dynamic model discovery [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on PR #63440:
URL: https://github.com/apache/airflow/pull/63440#issuecomment-4050051361

   Would love to add safety to our future reviews / triages :) 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add LLM safety checks, should_report handling, and dynamic model discovery [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #63440:
URL: https://github.com/apache/airflow/pull/63440#issuecomment-4052463304

   ### Backport failed to create: v3-1-test. View the failure log <a href='https://github.com/apache/airflow/actions/runs/23035622134'> Run details </a>
   
   Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
   the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
   
   In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
   
   <table>
       <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>❌</td>
           <td>v3-1-test</td>
           <td><a href="https://github.com/apache/airflow/commit/8db394977f3e0618b02047bd31b7f5b3415b62ef"><img src='https://img.shields.io/badge/Commit-8db3949-red' alt='Commit Link'></a></td>
       </tr>
   </table>
   
   You can attempt to backport this manually by running:
   
   ```bash
   cherry_picker 8db3949 v3-1-test
   ```
   
   This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
   the files that need manual conflict resolution.
   
   After you have resolved the conflicts, you can continue the backport process by running:
   
   ```bash
   cherry_picker --continue
   ```
   
   If you don't have cherry-picker installed, see the [installation guide](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#how-to-backport-pr-with-cherry-picker-cli).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add LLM safety checks, should_report handling, and dynamic model discovery [airflow]

Posted by "choo121600 (via GitHub)" <gi...@apache.org>.

choo121600 merged PR #63440:
URL: https://github.com/apache/airflow/pull/63440


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add LLM safety checks, should_report handling, and dynamic model discovery [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.

potiuk commented on PR #63440:
URL: https://github.com/apache/airflow/pull/63440#issuecomment-4046440361

   Another one .. This time  - security for the LLM calls. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org