[PR] [ci] Refactor CI of test cases in integration module [flink-agents]

["weiqingy (via GitHub)" <gi...@apache.org>]

weiqingy opened a new pull request, #702:
URL: https://github.com/apache/flink-agents/pull/702

   Linked issue: #161
   
   ### Purpose of change
   
   Splits the Python unit-test job from the integration-test job using a new `@pytest.mark.integration` marker.
   
   Today, 10 tests under `python/flink_agents/integrations/*/tests/` need live external services (Ollama, DashScope, OpenAI, Azure, Anthropic, Chroma) or optional Python deps (mem0), but they live in the unit-test job and silently skip in 5 of 6 matrix cells — only the Python 3.10 cell auto-pulls Ollama. When Ollama regresses, exactly one cell reports failure, which looks like environmental noise.
   
   After this PR:
   
   - **`python/pyproject.toml`** — new `[tool.pytest.ini_options]` block registers the `integration` marker and enables `strict_markers = true`.
   - **10 test files under `python/flink_agents/integrations/*/tests/`** — module-level `pytestmark = pytest.mark.integration` (list form for `test_mem0_vector_store.py` to preserve its existing `skipif`).
   - **`tools/ut.sh`** — both `if $run_e2e; then ... else ... fi` blocks (uv branch + pip fallback) updated:
     - `ut-python` (`run_e2e=false`): appends `-m "not integration"` to the existing `-k "not e2e_tests"` filter. Drops 70 tests from the unit-test job (was 518, now 448).
     - `it-python` (`run_e2e=true`): runs two sequential pytest invocations — existing `-k "e2e_tests_integration"` (27 tests, unchanged) and new `-m "integration"` (70 tests) — with aggregated exit codes and an exit-5 trap on the integration arm to defend against `-m` selector typos that `--strict-markers` cannot catch.
   
   No `.github/workflows/ci.yml` change required.
   
   `integrations/mcp/tests/test_mcp.py` is intentionally not tagged: it self-hosts an MCP server via `multiprocessing` and runs deterministically without external deps.
   
   ### Tests
   
   Locally verified empirical contract (`pytest --collect-only -q`):
   
   | Selector | Tests | Notes |
   |----------|-------|-------|
   | `pytest -m "not integration" -k "not e2e_tests"` | **448** | New `ut-python` arm (was 518) |
   | `pytest -m "integration"` | **70** | New `it-python` arm 2 |
   | `pytest -k "e2e_tests_integration"` | **27** | `it-python` arm 1, unchanged |
   | no selector | 552 | Unchanged |
   
   Overlap between `-m "integration"` and `-k "e2e_tests_integration"` is **0** — disjoint by directory structure.
   
   Additional verification:
   
   - `./tools/lint.sh -c` passes
   - `./tools/check-license.sh` passes
   - `./tools/ut.sh -p` passes locally (448 passed, 104 deselected, ~28s)
   - `pytest --markers` shows the registered `integration` marker with description
   - Typo'd `@pytest.mark.intergration` triggers a collection error (`strict_markers = true` enforcement is active)
   
   ### API
   
   No public API changes. This PR is CI plumbing plus a pytest marker added to existing test files.
   
   ### Documentation
   
   - [ ] `doc-needed`
   - [x] `doc-not-needed`
   - [ ] `doc-included`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [ci] Refactor CI of test cases in integration module [flink-agents]

["wenjin272 (via GitHub)" <gi...@apache.org>]

wenjin272 merged PR #702:
URL: https://github.com/apache/flink-agents/pull/702


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org