[I] PR stats parity tests can publish or overwrite real gists [magpie]

["MakerFlyFly (via GitHub)" <gi...@apache.org>]

MakerFlyFly opened a new issue, #1602:
URL: https://github.com/apache/magpie/issues/1602

   ## What's broken
   
   The PR stats JSON parity tests call `dashboard.main()` with gist publishing enabled.
   On a machine where `gh` is authenticated with the `gist` scope, running these unit tests can overwrite a real dashboard gist or create a secret gist in the developer's account, even though both tests pass.
   
   ## Which layer
   
   `tools/pr-management/tests/stats/test_json_parity.py`: `_run()` omits `--no-publish`, and `_install_stubs()` replaces only fetch functions.
   The publication and session-state branch is in `tools/pr-management/src/pr_management/stats/dashboard.py`.
   
   ## How to reproduce
   
   Reproduced at upstream `dadb315caa8b881fc1d2195bef03223dac25915a` in an isolated Linux checkout with CPython 3.12.12 and workspace development dependencies.
   Use a disposable checkout and this recording replacement; it makes no network requests and never delegates to real `gh`.
   
   ```bash
   set -euo pipefail
   stub_dir="$(mktemp -d)"
   export H09_GH_LOG="$stub_dir/calls.log"
   cat > "$stub_dir/gh" <<'SH'
   #!/bin/sh
   printf 'gh %s\n' "$*" >> "$H09_GH_LOG"
   case "${1:-} ${2:-}" in
     'auth status') printf "Token scopes: 'gist', 'read:org', 'repo'\n" ;;
     'api -X') cat > /dev/null ;;
     'gist create') printf '%s\n' 'https://gist.github.com/tester/0123456789abcdef0123456789abcdef' ;;
     *) exit 98 ;;
   esac
   SH
   chmod +x "$stub_dir/gh"
   PATH="$stub_dir:$PATH" uv run --directory tools/pr-management --project . \
     python -m pytest -q tests/stats/test_json_parity.py
   cat "$H09_GH_LOG"
   ```
   
   With the tracked legacy gist ID and no personal session state, both tests pass while the replacement records two `gh auth status` and two gist PATCH calls.
   Without legacy or personal state, verified in a separate disposable run, the tests create a gist, PATCH it, and store the fake ID in `.apache-magpie-local/session-state.json`.
   
   ## Expected vs actual
   
   Expected: fixture-based tests compare JSON without authenticating with GitHub, publishing, or reading/writing gist session state.
   Actual: the publication branch remains reachable outside the assertions.
   
   The reviewed fix adds `--no-publish` to dashboard argv only and makes authentication, publication, and session-state entry points reject unexpected test access with `pytest.fail`.
   The `reference` entry point rejects that flag, so it must not be added to shared argv unconditionally.
   
   ## Surface area
   
   One test file, 15 added lines; existing JSON assertions and production behavior are preserved.
   Publication-wiring coverage, a stats-wide subprocess guard, tracked legacy-state cleanup, and explicit UTF-8 production HTML reads/writes are independent follow-ups.
   
   ## Environment
   
   Linux, CPython 3.12.12, pytest 9.1.1, non-root Docker validation with a recording `gh` replacement.
   After the fix, both state scenarios pass with zero `gh` calls and no state changes; removing the opt-out or forcing any of the four entries ahead of it fails even inside `except Exception`.
   
   Suggested labels: `bug`, `family:pr-management`, `capability:stats`.
   Developed with Codex (GPT-6); the contributor has completed manual code review and approved the patch for upstream submission.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] PR stats parity tests can publish or overwrite real gists [magpie]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk closed issue #1602: PR stats parity tests can publish or overwrite real gists
URL: https://github.com/apache/magpie/issues/1602


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@magpie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org