You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "terrymanu (via GitHub)" <gi...@apache.org> on 2026/07/20 02:56:06 UTC
[PR] Improve MCP validation and LLM quality gates [shardingsphere]
terrymanu opened a new pull request, #39187:
URL: https://github.com/apache/shardingsphere/pull/39187
- apply identifier policies consistently in workflow validation
- clarify SQL and metadata contracts for model clients
- compact resource discovery payloads and fix recovery scoring
- pin a verified LLM baseline and split long-running CI suites
- enforce the intended MCP metadata coverage gate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve MCP validation and LLM quality gates [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu merged PR #39187:
URL: https://github.com/apache/shardingsphere/pull/39187
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Improve MCP validation and LLM quality gates [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu commented on PR #39187:
URL: https://github.com/apache/shardingsphere/pull/39187#issuecomment-5018470888
### Summary
**Review Result: Mergeable**
**Reason:** No code-level blocker was found in the latest commit, `84df69a8bd77f1b06f506ced2aab9b2b701c7906`. The patch addresses the root causes, provides appropriate test coverage, stays within scope, and avoids overengineering. This is a code-scope-only conclusion; CI not reviewed by request.
### Evidence
- Encrypt rule validation now consistently applies `IdentifierScope.COLUMN` to logic, cipher, assisted-query, and LIKE-query columns. The fix directly addresses the incorrect identifier comparison scope and includes a case-policy regression test: `mcp/features/encrypt/src/main/java/org/apache/shardingsphere/mcp/feature/encrypt/tool/service/EncryptWorkflowValidationService.java:270` and `mcp/features/encrypt/src/test/java/org/apache/shardingsphere/mcp/feature/encrypt/tool/service/EncryptWorkflowValidationServiceTest.java:109`.
- Sharding columns, complex sharding columns, and key-generation columns use `COLUMN`, while table names, table references, and algorithm names correctly retain `TABLE`: `mcp/features/sharding/src/main/java/org/apache/shardingsphere/mcp/feature/sharding/tool/service/ShardingWorkflowValidationService.java:232`, `:253`, `:261`, `:305`, and `:349`.
- Resource and resource-template payloads are compacted only in the LLM E2E model-facing layer. Raw MCP responses and production protocol behavior remain unchanged, while URIs, names, titles, and MIME types are preserved: `test/e2e/mcp/src/test/java/org/apache/shardingsphere/test/e2e/mcp/llm/conversation/LLMMCPModelFacingToolResponseFormatter.java:46`.
- Recovery scoring now checks the top-level payload, `recovery`, `empty_state`, `ambiguity_state`, and `error_code`. This prevents a generic category from hiding a more specific category and includes an `unknown_database` regression scenario: `test/e2e/mcp/src/test/java/org/apache/shardingsphere/test/e2e/mcp/llm/suite/usability/assessment/LLMUsabilityTraceMetrics.java:129` and `LLMUsabilityMetricCalculatorTest.java:151`.
- The pinned LLM revision, filename, and SHA-256 are consistent with the [fixed Hugging Face model artifact](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF/blob/3885219b6810b007914f3a7950a8d1b469d598a5/Qwen3.5-9B-Q3_K_M.gguf): `test/e2e/mcp/src/test/resources/env/e2e-env.properties:32`.
- All three LLM E2E test classes are covered by the matrix, with isolated test selectors, run IDs, and artifact names: `.github/workflows/e2e-mcp.yml:248`.
- The latest commit additionally removes redundant constructor Javadoc without changing any signature or runtime behavior: `mcp/core/src/main/java/org/apache/shardingsphere/mcp/core/tool/handler/execute/SQLToolMismatchException.java:41`.
### Review Details
- **Review Focus:** Code Correctness Review. CI not reviewed by request.
- **Reviewed Scope:** All 19 files in PR head `84df69a8bd77f1b06f506ced2aab9b2b701c7906`. The base and local merge base are both `8b15b93e49cbf675c2274ef01bef3b0b62259f08`. The local changed-file list exactly matches GitHub `/pulls/39187/files`.
- **Not Reviewed Scope:** GitHub Actions status and logs, and modules outside the 19-file PR scope that are not on a directly affected dependency path.
- **Verification:**
- Encrypt and Sharding focused tests: 40 tests passed, exit `0`.
- LLM configuration, response formatting, recovery scoring, and artifact tests: 46 tests passed, exit `0`.
- MCP Core coverage gate: 711 tests passed, and JaCoCo reported that all targeted coverage checks were met, exit `0`.
- MCP Core Checkstyle: 0 violations, exit `0`.
- Both changed GitHub Actions YAML files parsed successfully, exit `0`.
- Local LLM E2E on behavior-equivalent code: Smoke `2/2`, Usability `12/12` with a full score, and MCP Builder `10/10` with a full score. The latest commit only removes Javadoc.
- **Release Note / User Docs:** Not required. The production changes are limited to validation-policy corrections and embedded model-facing contract descriptions. They introduce no user configuration, API/SPI migration, or compatibility requirements.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org