You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hugegraph.apache.org by ji...@apache.org on 2026/05/31 04:07:26 UTC
(hugegraph-ai) branch goal-scan updated (11e5b407 -> 53f3ebac)
This is an automated email from the ASF dual-hosted git repository.
imbajin pushed a change to branch goal-scan
in repository https://gitbox.apache.org/repos/asf/hugegraph-ai.git
from 11e5b407 fix(llm): validate request-scoped config
new 0f03a131 fix(llm): replace fake integration smokes
new 53f3ebac docs(code-scan): sync fixed issue statuses
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.workflow/code-scan/code-scan-state.json | 8 +-
.../code-scan/reports/final-code-scan-report.md | 3 +-
.workflow/code-scan/reports/issues.md | 22 +-
.workflow/code-scan/reports/test-quality-ledger.md | 5 +-
.../tests/integration/test_graph_rag_pipeline.py | 295 ++------------------
.../src/tests/integration/test_kg_construction.py | 299 ++++++---------------
.../src/tests/integration/test_rag_pipeline.py | 226 ++++------------
7 files changed, 172 insertions(+), 686 deletions(-)
(hugegraph-ai) 02/02: docs(code-scan): sync fixed issue statuses
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
imbajin pushed a commit to branch goal-scan
in repository https://gitbox.apache.org/repos/asf/hugegraph-ai.git
commit 53f3ebac1717e589f57d3cbb979e86e75a8d5b31
Author: imbajin <ji...@apache.org>
AuthorDate: Sun May 31 11:59:32 2026 +0800
docs(code-scan): sync fixed issue statuses
- correct status rows for completed requested fixes
- leave unrelated open scan findings marked open
- keep CS-001, CS-002, CS-004, CS-008, CS-011, CS-012, CS-013, and CS-030 fixed
---
.workflow/code-scan/reports/issues.md | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/.workflow/code-scan/reports/issues.md b/.workflow/code-scan/reports/issues.md
index 665de54d..6c8277f5 100644
--- a/.workflow/code-scan/reports/issues.md
+++ b/.workflow/code-scan/reports/issues.md
@@ -45,7 +45,7 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-python-client`
- Layer: transport / routing
- Paths: `hugegraph-python-client/src/pyhugegraph/utils/huge_requests.py:100`, `hugegraph-python-client/src/pyhugegraph/utils/huge_requests.py:120`, `hugegraph-python-client/src/pyhugegraph/api/auth.py:37`, `hugegraph-python-client/src/pyhugegraph/api/auth.py:84`
-- Status: fixed
+- Status: open
- Evidence: `HGraphSession.resolve()` passes leading-slash paths to `urljoin`, which replaces the base URL path. A base like `http://host/proxy` plus `/auth/groups` resolves to `http://host/auth/groups`.
- Impact: HugeGraph behind path-prefix proxies or ingress routes is addressed incorrectly.
- Recommendation: Treat route strings as API-root-relative paths inside the client, or reject base URLs with path prefixes explicitly.
@@ -69,7 +69,7 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-python-client`
- Layer: transport / routing
- Paths: `hugegraph-python-client/src/pyhugegraph/utils/huge_config.py:44`, `hugegraph-python-client/src/pyhugegraph/utils/huge_config.py:73`, `hugegraph-python-client/src/pyhugegraph/utils/huge_router.py:139`
-- Status: fixed
+- Status: open
- Evidence: `HGraphConfig.__post_init__()` probes `/versions` with a fixed `0.5s` timeout. Most probe failures silently set `gs_supported=False`, making graphspace auth routes fail locally with `ValueError`.
- Impact: HugeGraph 1.7+ auth behavior becomes sensitive to a single transient version probe.
- Recommendation: Make graphspace configuration explicit or retry/observe the capability probe using the configured timeout.
@@ -81,7 +81,7 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-python-client`
- Layer: client-api
- Paths: `hugegraph-python-client/src/pyhugegraph/api/graph.py:68`, `hugegraph-python-client/src/pyhugegraph/api/graph.py:73`, `hugegraph-python-client/src/pyhugegraph/api/graph.py:173`, `hugegraph-python-client/src/pyhugegraph/utils/huge_requests.py:144`
-- Status: fixed
+- Status: open
- Evidence: `getVertexByPage()`, `getVertexByCondition()`, and `getEdgeByPage()` concatenate labels and JSON properties directly into the URL.
- Impact: Property values containing `&`, `?`, `#`, `=`, or spaces can corrupt filters and query different data.
- Recommendation: Use `params=` or `urllib.parse.urlencode()` for all query parameters.
@@ -105,7 +105,7 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-python-client`
- Layer: gremlin / error contract
- Paths: `hugegraph-python-client/src/pyhugegraph/api/gremlin.py:48`, `hugegraph-python-client/src/pyhugegraph/api/gremlin.py:54`, `hugegraph-python-client/src/tests/api/test_gremlin.py:99`
-- Status: open
+- Status: fixed
- Evidence: `GremlinManager.exec()` catches every exception and raises `NotFoundError`.
- Impact: Auth failures, transport errors, server errors, and syntax errors are indistinguishable upstream.
- Fix: `GremlinManager.exec()` no longer wraps all lower-level exceptions as `NotFoundError`; typed auth/transport/server/parse exceptions propagate from the transport boundary.
@@ -141,7 +141,7 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-llm`
- Layer: API / models / config
- Paths: `hugegraph-llm/src/hugegraph_llm/api/models/rag_requests.py:27`, `hugegraph-llm/src/hugegraph_llm/api/rag_api.py:92`
-- Status: open
+- Status: fixed
- Evidence: `GraphConfigRequest` has concrete defaults and `set_graph_config()` writes every field to global `huge_settings` whenever `client_config` exists.
- Impact: A request intended to override one field can clear credentials, graph name, or graphspace for the whole process.
- Fix: request-scoped graph config updates now use `model_fields_set`, so only explicitly provided nested fields mutate `huge_settings`.
@@ -165,7 +165,7 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-llm`
- Layer: LLM operator / graph import
- Paths: `hugegraph-llm/src/hugegraph_llm/operators/llm_op/property_graph_extract.py:53`, `hugegraph-llm/src/hugegraph_llm/operators/hugegraph_op/commit_to_hugegraph.py:134`, `hugegraph-llm/src/hugegraph_llm/flows/import_graph_data.py:65`
-- Status: open
+- Status: fixed
- Evidence: `filter_item()` converts non-string properties to strings, while `Commit2Graph` enforces schema Python types and skips mismatches. Edge writes can continue and the flow reports success.
- Impact: KG import can partially fail while the UI reports success.
- Fix: `filter_item()` no longer stringifies non-string property values, so extracted JSON numeric values survive into `Commit2Graph` schema validation.
(hugegraph-ai) 01/02: fix(llm): replace fake integration smokes
Posted by ji...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
imbajin pushed a commit to branch goal-scan
in repository https://gitbox.apache.org/repos/asf/hugegraph-ai.git
commit 0f03a1319c9a586e52809214d99240ceba54fb12
Author: imbajin <ji...@apache.org>
AuthorDate: Sun May 31 11:58:03 2026 +0800
fix(llm): replace fake integration smokes
- remove test-local RAG and KG stand-ins from integration tests
- cover production flow and operator wiring with deterministic doubles
- update code-scan ledgers for CS-030
---
.workflow/code-scan/code-scan-state.json | 8 +-
.../code-scan/reports/final-code-scan-report.md | 3 +-
.workflow/code-scan/reports/issues.md | 10 +-
.workflow/code-scan/reports/test-quality-ledger.md | 5 +-
.../tests/integration/test_graph_rag_pipeline.py | 295 ++------------------
.../src/tests/integration/test_kg_construction.py | 299 ++++++---------------
.../src/tests/integration/test_rag_pipeline.py | 226 ++++------------
7 files changed, 166 insertions(+), 680 deletions(-)
diff --git a/.workflow/code-scan/code-scan-state.json b/.workflow/code-scan/code-scan-state.json
index 704cd470..d08cac98 100644
--- a/.workflow/code-scan/code-scan-state.json
+++ b/.workflow/code-scan/code-scan-state.json
@@ -61,7 +61,8 @@
"CS-008",
"CS-011",
"CS-012",
- "CS-013"
+ "CS-013",
+ "CS-030"
],
"files_touched": [
"docs/specs/2026-05-31-hugegraph-ai-code-scan-design.md",
@@ -85,9 +86,6 @@
"hugegraph-llm/src/tests/config/test_config.py",
"hugegraph-llm/src/tests/config/test_prompt_config.py",
"hugegraph-llm/src/tests/indices/test_faiss_vector_index.py",
- "hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py",
- "hugegraph-llm/src/tests/integration/test_kg_construction.py",
- "hugegraph-llm/src/tests/integration/test_rag_pipeline.py",
"hugegraph-llm/src/tests/models/llms/test_openai_client.py",
"hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py",
"hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.py",
@@ -127,5 +125,5 @@
"uv run ruff check <edited-python-files>",
"git status --short"
],
- "next_recommended_action": "Replace fake integration tests with production-flow smoke coverage."
+ "next_recommended_action": "Continue remaining lower-priority findings outside the requested report-order fix plan."
}
diff --git a/.workflow/code-scan/reports/final-code-scan-report.md b/.workflow/code-scan/reports/final-code-scan-report.md
index 2ca80e8e..808be10c 100644
--- a/.workflow/code-scan/reports/final-code-scan-report.md
+++ b/.workflow/code-scan/reports/final-code-scan-report.md
@@ -80,7 +80,7 @@ None.
2. Fix the client error-contract cluster: `CS-002`, `CS-004`, `CS-008`. Status: fixed after the scan.
3. Fix graph import correctness: `CS-013`, then add the missing round-trip tests. Status: fixed after the scan.
4. Fix global config mutation and provider validation: `CS-011`, `CS-012`. Status: fixed after the scan.
-5. Replace fake integration tests with production-flow smoke coverage before broader refactors.
+5. Replace fake integration tests with production-flow smoke coverage before broader refactors. Status: fixed after the scan.
## Fix Progress After Scan
@@ -91,3 +91,4 @@ None.
- `CS-013`: fixed by preserving extracted property value types through the full extraction-to-commit path.
- `CS-011`: fixed by updating global graph settings only for explicitly supplied `client_config` fields.
- `CS-012`: fixed by rejecting unsupported provider types at request validation before global settings are mutated.
+- `CS-030`: fixed by replacing local fake integration flows with production Flow/Operator smoke tests.
diff --git a/.workflow/code-scan/reports/issues.md b/.workflow/code-scan/reports/issues.md
index 63dbc69f..665de54d 100644
--- a/.workflow/code-scan/reports/issues.md
+++ b/.workflow/code-scan/reports/issues.md
@@ -369,11 +369,11 @@ Issue IDs use `CS-NNN`. Priorities are P0 highest through P5 lowest.
- Module: `hugegraph-llm`
- Layer: test-quality
- Paths: `hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py:39`, `hugegraph-llm/src/tests/integration/test_rag_pipeline.py:33`, `hugegraph-llm/src/tests/integration/test_kg_construction.py:46`
-- Status: open
-- Evidence: Tests define local `RAGPipeline`, local document/vector/LLM classes, and local `KGConstructor`.
-- Impact: Green integration tests do not validate production flow wiring.
-- Recommendation: Use production flows/nodes/operators with deterministic fake LLM/vector/HugeGraph boundaries.
-- Test note: Added `FIXME:` comments in all three files.
+- Status: fixed
+- Evidence: Tests defined local `RAGPipeline`, local document/vector/LLM classes, and local `KGConstructor`.
+- Impact: Green integration tests did not validate production flow wiring.
+- Fix: Replaced the local stand-ins with production `RAGVectorOnlyFlow`, `ChunkSplitter`, vector index operators, `GraphExtractFlow`, and `PropertyGraphExtract` smoke tests using deterministic test doubles only at external boundaries.
+- Test note: Removed the resolved `FIXME:` comments and verified the three integration smoke files directly.
### CS-031: Commit2Graph tests mock away core branch behavior
diff --git a/.workflow/code-scan/reports/test-quality-ledger.md b/.workflow/code-scan/reports/test-quality-ledger.md
index 4f41704a..70353447 100644
--- a/.workflow/code-scan/reports/test-quality-ledger.md
+++ b/.workflow/code-scan/reports/test-quality-ledger.md
@@ -11,9 +11,6 @@ Added specific `FIXME:` comments in these files:
- `hugegraph-llm/src/tests/indices/test_faiss_vector_index.py`
- `hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph.py`
- `hugegraph-llm/src/tests/operators/hugegraph_op/test_commit_to_hugegraph_load_into_graph.py`
-- `hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py`
-- `hugegraph-llm/src/tests/integration/test_rag_pipeline.py`
-- `hugegraph-llm/src/tests/integration/test_kg_construction.py`
- `hugegraph-llm/src/tests/operators/index_op/test_vector_index_query.py`
- `hugegraph-python-client/src/tests/api/test_auth_routing.py`
- `hugegraph-python-client/src/tests/api/test_graph.py`
@@ -24,7 +21,6 @@ Added specific `FIXME:` comments in these files:
| ID | Paths | Weakness |
|---|---|---|
-| TQ-001 | `test_graph_rag_pipeline.py`, `test_rag_pipeline.py`, `test_kg_construction.py` | Local stand-ins replace production flows/classes. |
| TQ-002 | `test_commit_to_hugegraph.py`, `test_commit_to_hugegraph_load_into_graph.py` | Helper patching bypasses branchy graph import logic. |
| TQ-003 | `test_rag_api.py` | Missing `/rag` and `/rag/graph` happy-path response-shaping tests. |
| TQ-004 | `test_graph.py`, `test_gremlin.py` | Shared graph state and fixed primary keys make tests order-dependent. |
@@ -60,3 +56,4 @@ Added specific `FIXME:` comments in these files:
- `CS-013`: extraction-to-commit round-trip tests now prove schema-typed numeric values survive `PropertyGraphExtract.run()` and are accepted by `Commit2Graph`.
- `CS-011`: `/rag` contract tests now prove partial `client_config` only updates explicitly supplied graph settings.
- `CS-012`: config API tests now prove unsupported LLM/reranker providers fail validation before mutating global settings.
+- `CS-030`: GraphRAG/RAG/KG integration smoke tests now use production flows/operators instead of test-local pipeline and KG constructor stand-ins.
diff --git a/hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py b/hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py
index 53e27055..ae21e08d 100644
--- a/hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py
+++ b/hugegraph-llm/src/tests/integration/test_graph_rag_pipeline.py
@@ -15,274 +15,27 @@
# specific language governing permissions and limitations
# under the License.
-
-import shutil
-import tempfile
-import unittest
-from unittest.mock import MagicMock
-
-from tests.utils.mock import MockEmbedding
-
-# FIXME: replace the local RAGPipeline stand-in with a smoke test that
-# instantiates production GraphRAG flows and verifies real node wiring.
-
-
-class BaseLLM:
- def generate(self, prompt, **kwargs):
- pass
-
- async def async_generate(self, prompt, **kwargs):
- pass
-
- def get_llm_type(self):
- pass
-
-
-# 模拟RAGPipeline类
-class RAGPipeline:
- def __init__(self, llm=None, embedding=None):
- self.llm = llm
- self.embedding = embedding
- self.operators = {}
-
- def extract_word(self, text=None, language="english"):
- if "word_extract" in self.operators:
- return self.operators["word_extract"]({"query": text})
- return {"words": []}
-
- def extract_keywords(self, text=None, max_keywords=5, language="english", extract_template=None):
- if "keyword_extract" in self.operators:
- return self.operators["keyword_extract"]({"query": text})
- return {"keywords": []}
-
- def keywords_to_vid(self, by="keywords", topk_per_keyword=5, topk_per_query=10):
- if "semantic_id_query" in self.operators:
- return self.operators["semantic_id_query"]({"keywords": []})
- return {"match_vids": []}
-
- def query_graphdb(
- self,
- max_deep=2,
- max_graph_items=10,
- max_v_prop_len=2048,
- max_e_prop_len=256,
- prop_to_match=None,
- num_gremlin_generate_example=1,
- gremlin_prompt=None,
- ):
- if "graph_rag_query" in self.operators:
- return self.operators["graph_rag_query"]({"match_vids": []})
- return {"graph_result": []}
-
- def query_vector_index(self, max_items=3):
- if "vector_index_query" in self.operators:
- return self.operators["vector_index_query"]({"query": ""})
- return {"vector_result": []}
-
- def merge_dedup_rerank(
- self, graph_ratio=0.5, rerank_method="bleu", near_neighbor_first=False, custom_related_information=""
- ):
- if "merge_dedup_rerank" in self.operators:
- return self.operators["merge_dedup_rerank"]({"graph_result": [], "vector_result": []})
- return {"merged_result": []}
-
- def synthesize_answer(
- self,
- raw_answer=False,
- vector_only_answer=True,
- graph_only_answer=False,
- graph_vector_answer=False,
- answer_prompt=None,
- ):
- if "answer_synthesize" in self.operators:
- return self.operators["answer_synthesize"]({"merged_result": []})
- return {"answer": ""}
-
- def run(self, **kwargs):
- context = {"query": kwargs.get("query", "")}
-
- # 执行各个步骤
- if not kwargs.get("skip_extract_word", False):
- context.update(self.extract_word(text=context["query"]))
-
- if not kwargs.get("skip_extract_keywords", False):
- context.update(self.extract_keywords(text=context["query"]))
-
- if not kwargs.get("skip_keywords_to_vid", False):
- context.update(self.keywords_to_vid())
-
- if not kwargs.get("skip_query_graphdb", False):
- context.update(self.query_graphdb())
-
- if not kwargs.get("skip_query_vector_index", False):
- context.update(self.query_vector_index())
-
- if not kwargs.get("skip_merge_dedup_rerank", False):
- context.update(self.merge_dedup_rerank())
-
- if not kwargs.get("skip_synthesize_answer", False):
- context.update(
- self.synthesize_answer(
- vector_only_answer=kwargs.get("vector_only_answer", False),
- graph_only_answer=kwargs.get("graph_only_answer", False),
- graph_vector_answer=kwargs.get("graph_vector_answer", False),
- )
- )
-
- return context
-
-
-class MockLLM(BaseLLM):
- """Mock LLM class for testing"""
-
- def __init__(self):
- self.model = "mock_llm"
-
- def generate(self, prompt, **kwargs):
- # Return a simple mock response based on the prompt
- if "person" in prompt.lower():
- return "This is information about a person."
- if "movie" in prompt.lower():
- return "This is information about a movie."
- return "I don't have specific information about that."
-
- async def async_generate(self, prompt, **kwargs):
- # Async version returns the same as the sync version
- return self.generate(prompt, **kwargs)
-
- def get_llm_type(self):
- return "mock"
-
-
-class TestGraphRAGPipeline(unittest.TestCase):
- def setUp(self):
- # Create a temporary directory for testing
- self.test_dir = tempfile.mkdtemp()
-
- # Create mock models
- self.embedding = MockEmbedding()
- self.llm = MockLLM()
-
- # Create mock operators
- self.mock_word_extract = MagicMock()
- self.mock_word_extract.return_value = {"words": ["person", "movie"]}
-
- self.mock_keyword_extract = MagicMock()
- self.mock_keyword_extract.return_value = {"keywords": ["person", "movie"]}
-
- self.mock_semantic_id_query = MagicMock()
- self.mock_semantic_id_query.return_value = {"match_vids": ["1:person", "2:movie"]}
-
- self.mock_graph_rag_query = MagicMock()
- self.mock_graph_rag_query.return_value = {
- "graph_result": ["Person: John Doe, Age: 30", "Movie: The Matrix, Year: 1999"]
- }
-
- self.mock_vector_index_query = MagicMock()
- self.mock_vector_index_query.return_value = {
- "vector_result": ["John Doe is a software engineer.", "The Matrix is a science fiction movie."]
- }
-
- self.mock_merge_dedup_rerank = MagicMock()
- self.mock_merge_dedup_rerank.return_value = {
- "merged_result": [
- "Person: John Doe, Age: 30",
- "Movie: The Matrix, Year: 1999",
- "John Doe is a software engineer.",
- "The Matrix is a science fiction movie.",
- ]
- }
-
- self.mock_answer_synthesize = MagicMock()
- self.mock_answer_synthesize.return_value = {
- "answer": (
- "John Doe is a 30-year-old software engineer. The Matrix is a science fiction movie released in 1999."
- )
- }
-
- # 创建RAGPipeline实例
- self.pipeline = RAGPipeline(llm=self.llm, embedding=self.embedding)
- self.pipeline.operators = {
- "word_extract": self.mock_word_extract,
- "keyword_extract": self.mock_keyword_extract,
- "semantic_id_query": self.mock_semantic_id_query,
- "graph_rag_query": self.mock_graph_rag_query,
- "vector_index_query": self.mock_vector_index_query,
- "merge_dedup_rerank": self.mock_merge_dedup_rerank,
- "answer_synthesize": self.mock_answer_synthesize,
- }
-
- def tearDown(self):
- # Clean up the temporary directory
- shutil.rmtree(self.test_dir)
-
- def test_rag_pipeline_end_to_end(self):
- # Run the pipeline with a query
- query = "Tell me about John Doe and The Matrix movie"
- result = self.pipeline.run(query=query)
-
- # Verify the result
- self.assertIn("answer", result)
- self.assertEqual(
- result["answer"],
- "John Doe is a 30-year-old software engineer. The Matrix is a science fiction movie released in 1999.",
- )
-
- # Verify that all operators were called
- self.mock_word_extract.assert_called_once()
- self.mock_keyword_extract.assert_called_once()
- self.mock_semantic_id_query.assert_called_once()
- self.mock_graph_rag_query.assert_called_once()
- self.mock_vector_index_query.assert_called_once()
- self.mock_merge_dedup_rerank.assert_called_once()
- self.mock_answer_synthesize.assert_called_once()
-
- def test_rag_pipeline_vector_only(self):
- # Run the pipeline with a query, skipping graph-related steps
- query = "Tell me about John Doe and The Matrix movie"
- result = self.pipeline.run(
- query=query,
- skip_keywords_to_vid=True,
- skip_query_graphdb=True,
- skip_merge_dedup_rerank=True,
- vector_only_answer=True,
- )
-
- # Verify the result
- self.assertIn("answer", result)
- self.assertEqual(
- result["answer"],
- "John Doe is a 30-year-old software engineer. The Matrix is a science fiction movie released in 1999.",
- )
-
- # Verify that only vector-related operators were called
- self.mock_word_extract.assert_called_once()
- self.mock_keyword_extract.assert_called_once()
- self.mock_semantic_id_query.assert_not_called()
- self.mock_graph_rag_query.assert_not_called()
- self.mock_vector_index_query.assert_called_once()
- self.mock_merge_dedup_rerank.assert_not_called()
- self.mock_answer_synthesize.assert_called_once()
-
- def test_rag_pipeline_graph_only(self):
- # Run the pipeline with a query, skipping vector-related steps
- query = "Tell me about John Doe and The Matrix movie"
- result = self.pipeline.run(
- query=query, skip_query_vector_index=True, skip_merge_dedup_rerank=True, graph_only_answer=True
- )
-
- # Verify the result
- self.assertIn("answer", result)
- self.assertEqual(
- result["answer"],
- "John Doe is a 30-year-old software engineer. The Matrix is a science fiction movie released in 1999.",
- )
-
- # Verify that only graph-related operators were called
- self.mock_word_extract.assert_called_once()
- self.mock_keyword_extract.assert_called_once()
- self.mock_semantic_id_query.assert_called_once()
- self.mock_graph_rag_query.assert_called_once()
- self.mock_vector_index_query.assert_not_called()
- self.mock_merge_dedup_rerank.assert_not_called()
- self.mock_answer_synthesize.assert_called_once()
+import pytest
+
+pytestmark = [pytest.mark.smoke, pytest.mark.integration]
+
+
+def test_vector_only_rag_flow_builds_production_pipeline():
+ from hugegraph_llm.flows.rag_flow_vector_only import RAGVectorOnlyFlow
+
+ pipeline = RAGVectorOnlyFlow().build_flow(
+ query="Who created lop?",
+ topk_return_results=2,
+ vector_dis_threshold=0.8,
+ )
+ prepared = pipeline.getGParamWithNoEmpty("wkflow_input")
+ dot = pipeline.dump()
+
+ assert prepared.query == "Who created lop?"
+ assert prepared.vector_search is True
+ assert prepared.graph_search is False
+ assert prepared.topk_return_results == 2
+ assert prepared.vector_dis_threshold == 0.8
+ assert 'label="only_vector"' in dot
+ assert 'label="merge_two"' in dot
+ assert 'label="vector"' in dot
diff --git a/hugegraph-llm/src/tests/integration/test_kg_construction.py b/hugegraph-llm/src/tests/integration/test_kg_construction.py
index 857ca811..cbca4102 100644
--- a/hugegraph-llm/src/tests/integration/test_kg_construction.py
+++ b/hugegraph-llm/src/tests/integration/test_kg_construction.py
@@ -15,218 +15,89 @@
# specific language governing permissions and limitations
# under the License.
-# pylint: disable=import-error,wrong-import-position,unused-argument
-
-import json
-import os
-import unittest
-from unittest.mock import patch
-
-# 导入测试工具
-from src.tests.test_utils import (
- create_test_document,
- should_skip_external,
- with_mock_openai_client,
-)
-
-# FIXME: cover the production KG extraction/import path with deterministic LLM
-# fakes instead of defining a test-local KGConstructor.
-
-
-# Create mock classes to replace missing modules
-class OpenAILLM:
- """Mock OpenAILLM class"""
-
- def __init__(self, api_key=None, model=None):
- self.api_key = api_key
- self.model = model or "gpt-3.5-turbo"
-
- def generate(self, prompt):
- # Return a mock response
- return f"This is a mock response to '{prompt}'"
-
-
-class KGConstructor:
- """Mock KGConstructor class"""
-
- def __init__(self, llm, schema):
- self.llm = llm
- self.schema = schema
-
- def extract_entities(self, document):
- # Mock entity extraction
- if "张三" in document.content:
- return [
- {"type": "Person", "name": "张三", "properties": {"occupation": "Software Engineer"}},
- {
- "type": "Company",
- "name": "ABC Company",
- "properties": {"industry": "Technology", "location": "Beijing"},
- },
- ]
- if "李四" in document.content:
- return [
- {"type": "Person", "name": "李四", "properties": {"occupation": "Data Scientist"}},
- {"type": "Person", "name": "张三", "properties": {"occupation": "Software Engineer"}},
- ]
- if "ABC Company" in document.content or "ABC公司" in document.content:
- return [
- {
- "type": "Company",
- "name": "ABC Company",
- "properties": {"industry": "Technology", "location": "Beijing"},
- }
- ]
- return []
-
- def extract_relations(self, document):
- # Mock relation extraction
- if "张三" in document.content and ("ABC Company" in document.content or "ABC公司" in document.content):
- return [
- {
- "source": {"type": "Person", "name": "张三"},
- "relation": "works_for",
- "target": {"type": "Company", "name": "ABC Company"},
- }
- ]
- if "李四" in document.content and "张三" in document.content:
- return [
- {
- "source": {"type": "Person", "name": "李四"},
- "relation": "colleague",
- "target": {"type": "Person", "name": "张三"},
- }
- ]
- return []
-
- def construct_from_documents(self, documents):
- # Mock knowledge graph construction
- entities = []
- relations = []
-
- # Collect all entities and relations
- for doc in documents:
- entities.extend(self.extract_entities(doc))
- relations.extend(self.extract_relations(doc))
-
- # Deduplicate entities
- unique_entities = []
- entity_names = set()
- for entity in entities:
- if entity["name"] not in entity_names:
- unique_entities.append(entity)
- entity_names.add(entity["name"])
-
- return {"entities": unique_entities, "relations": relations}
-
-
-class TestKGConstruction(unittest.TestCase):
- """Integration tests for knowledge graph construction"""
-
- def setUp(self):
- """Setup work before testing"""
- # Skip if external service tests should be skipped
- if should_skip_external():
- self.skipTest("Skipping tests that require external services")
-
- # Load test schema
- schema_path = os.path.join(os.path.dirname(__file__), "../data/kg/schema.json")
- with open(schema_path, "r", encoding="utf-8") as f:
- self.schema = json.load(f)
-
- # Create test documents
- self.test_docs = [
- create_test_document("张三 is a software engineer working at ABC Company."),
- create_test_document("李四 is 张三's colleague and works as a data scientist."),
- create_test_document("ABC Company is a tech company headquartered in Beijing."),
- ]
-
- # Create LLM model
- self.llm = OpenAILLM()
-
- # Create knowledge graph constructor
- self.kg_constructor = KGConstructor(llm=self.llm, schema=self.schema)
-
- @with_mock_openai_client
- def test_entity_extraction(self, *args):
- """Test entity extraction"""
- # Extract entities from document
- doc = self.test_docs[0]
- entities = self.kg_constructor.extract_entities(doc)
-
- # Verify extracted entities
- self.assertEqual(len(entities), 2)
- self.assertEqual(entities[0]["name"], "张三")
- self.assertEqual(entities[1]["name"], "ABC Company")
-
- @with_mock_openai_client
- def test_relation_extraction(self, *args):
- """Test relation extraction"""
- # Extract relations from document
- doc = self.test_docs[0]
- relations = self.kg_constructor.extract_relations(doc)
-
- # Verify extracted relations
- self.assertEqual(len(relations), 1)
- self.assertEqual(relations[0]["source"]["name"], "张三")
- self.assertEqual(relations[0]["relation"], "works_for")
- self.assertEqual(relations[0]["target"]["name"], "ABC Company")
-
- @with_mock_openai_client
- def test_kg_construction_end_to_end(self, *args):
- """Test end-to-end knowledge graph construction process"""
- # Mock entity and relation extraction
- mock_entities = [
- {"type": "Person", "name": "张三", "properties": {"occupation": "Software Engineer"}},
- {"type": "Company", "name": "ABC Company", "properties": {"industry": "Technology"}},
- ]
-
- mock_relations = [
- {
- "source": {"type": "Person", "name": "张三"},
- "relation": "works_for",
- "target": {"type": "Company", "name": "ABC Company"},
- }
+import pytest
+from fixtures.fake_llm import FakeLLM
+
+pytestmark = [pytest.mark.smoke, pytest.mark.integration]
+
+
+PROPERTY_GRAPH_SCHEMA = {
+ "vertexlabels": [
+ {
+ "id": 1,
+ "name": "person",
+ "properties": ["name", "occupation"],
+ "primary_keys": ["name"],
+ "nullable_keys": ["occupation"],
+ },
+ {
+ "id": 2,
+ "name": "company",
+ "properties": ["name", "industry"],
+ "primary_keys": ["name"],
+ "nullable_keys": ["industry"],
+ },
+ ],
+ "edgelabels": [
+ {
+ "name": "works_for",
+ "source_label": "person",
+ "target_label": "company",
+ "properties": ["since"],
+ }
+ ],
+}
+
+
+def test_graph_extract_flow_builds_production_property_graph_pipeline():
+ from hugegraph_llm.flows.graph_extract import GraphExtractFlow
+
+ pipeline = GraphExtractFlow().build_flow(
+ schema=PROPERTY_GRAPH_SCHEMA,
+ texts=["Marko works for HugeGraph."],
+ example_prompt="extract property graph",
+ extract_type="property_graph",
+ language="en",
+ )
+ prepared = pipeline.getGParamWithNoEmpty("wkflow_input")
+ dot = pipeline.dump()
+
+ assert prepared.extract_type == "property_graph"
+ assert prepared.texts == ["Marko works for HugeGraph."]
+ assert 'label="schema_node"' in dot
+ assert 'label="chunk_split"' in dot
+ assert 'label="graph_extract"' in dot
+
+
+def test_property_graph_extract_uses_production_operator_with_deterministic_llm():
+ from hugegraph_llm.operators.llm_op.property_graph_extract import PropertyGraphExtract
+
+ llm = FakeLLM(
+ [
+ """{
+ "vertices": [
+ {"label": "person", "properties": {"name": "Marko", "occupation": "developer"}},
+ {"label": "company", "properties": {"name": "HugeGraph", "industry": "graph"}}
+ ],
+ "edges": [
+ {
+ "label": "works_for",
+ "properties": {"since": "2024"},
+ "source": {"label": "person", "properties": {"name": "Marko"}},
+ "target": {"label": "company", "properties": {"name": "HugeGraph"}}
+ }
+ ]
+ }"""
]
-
- # Mock KG constructor methods
- with (
- patch.object(self.kg_constructor, "extract_entities", return_value=mock_entities),
- patch.object(self.kg_constructor, "extract_relations", return_value=mock_relations),
- ):
- # Construct knowledge graph - use only one document to avoid duplicate relations from mocking
- kg = self.kg_constructor.construct_from_documents([self.test_docs[0]])
-
- # Verify knowledge graph
- self.assertIsNotNone(kg)
- self.assertEqual(len(kg["entities"]), 2)
- self.assertEqual(len(kg["relations"]), 1)
-
- # Verify entities
- entity_names = [e["name"] for e in kg["entities"]]
- self.assertIn("张三", entity_names)
- self.assertIn("ABC Company", entity_names)
-
- # Verify relations
- relation = kg["relations"][0]
- self.assertEqual(relation["source"]["name"], "张三")
- self.assertEqual(relation["relation"], "works_for")
- self.assertEqual(relation["target"]["name"], "ABC Company")
-
- def test_schema_validation(self):
- """Test schema validation"""
- # Verify schema structure
- self.assertIn("vertices", self.schema)
- self.assertIn("edges", self.schema)
-
- # Verify entity types
- vertex_labels = [v["vertex_label"] for v in self.schema["vertices"]]
- self.assertIn("person", vertex_labels)
-
- # Verify relation types
- edge_labels = [e["edge_label"] for e in self.schema["edges"]]
- self.assertIn("works_at", edge_labels)
-
-
-if __name__ == "__main__":
- unittest.main()
+ )
+ context = PropertyGraphExtract(llm=llm, example_prompt=None).run(
+ {
+ "schema": PROPERTY_GRAPH_SCHEMA,
+ "chunks": ["Marko works for HugeGraph."],
+ }
+ )
+
+ assert [vertex["label"] for vertex in context["vertices"]] == ["person", "company"]
+ assert context["edges"][0]["label"] == "works_for"
+ assert context["edges"][0]["outV"] == "1:Marko"
+ assert context["edges"][0]["inV"] == "2:HugeGraph"
diff --git a/hugegraph-llm/src/tests/integration/test_rag_pipeline.py b/hugegraph-llm/src/tests/integration/test_rag_pipeline.py
index 59dd8f2b..0093dc99 100644
--- a/hugegraph-llm/src/tests/integration/test_rag_pipeline.py
+++ b/hugegraph-llm/src/tests/integration/test_rag_pipeline.py
@@ -15,201 +15,67 @@
# specific language governing permissions and limitations
# under the License.
-import os
-import tempfile
-import unittest
+import pytest
-# 导入测试工具
-from src.tests.test_utils import (
- create_test_document,
- should_skip_external,
- with_mock_openai_client,
- with_mock_openai_embedding,
-)
-from tests.utils.mock import VectorIndex
+pytestmark = [pytest.mark.smoke, pytest.mark.integration]
-# FIXME: stop gating this local-only test behind SKIP_EXTERNAL_SERVICES, or
-# replace the local stand-ins with production loader/splitter/retrieval paths.
+class DeterministicEmbedding:
+ def get_embedding_dim(self):
+ return 2
-# 创建模拟类,替代缺失的模块
-class Document:
- """模拟的Document类"""
-
- def __init__(self, content, metadata=None):
- self.content = content
- self.metadata = metadata or {}
+ def get_texts_embeddings(self, texts):
+ return [[float("hugegraph" in text.lower()), float("olap" in text.lower())] for text in texts]
+ def get_text_embedding(self, text):
+ return self.get_texts_embeddings([text])[0]
-class TextLoader:
- """模拟的TextLoader类"""
+ async def async_get_texts_embeddings(self, texts):
+ return self.get_texts_embeddings(texts)
- def __init__(self, file_path):
- self.file_path = file_path
- def load(self):
- with open(self.file_path, "r", encoding="utf-8") as f:
- content = f.read()
- return [Document(content, {"source": self.file_path})]
+class InMemoryVectorIndex:
+ stores = {}
+ def __init__(self, name):
+ self.name = name
+ self.entries = []
-class RecursiveCharacterTextSplitter:
- """模拟的RecursiveCharacterTextSplitter类"""
+ @classmethod
+ def from_name(cls, embedding_dim, graph_name, index_name):
+ key = (embedding_dim, graph_name, index_name)
+ cls.stores.setdefault(key, cls(index_name))
+ return cls.stores[key]
- def __init__(self, chunk_size=1000, chunk_overlap=0):
- self.chunk_size = chunk_size
- self.chunk_overlap = chunk_overlap
+ def add(self, embeddings, chunks):
+ self.entries.extend(zip(embeddings, chunks))
- def split_documents(self, documents):
- result = []
- for doc in documents:
- # 简单地按照chunk_size分割文本
- text = doc.content
- chunks = [text[i : i + self.chunk_size] for i in range(0, len(text), self.chunk_size - self.chunk_overlap)]
- result.extend([Document(chunk, doc.metadata) for chunk in chunks])
- return result
+ def save_index_by_name(self, graph_name, index_name):
+ return None
+ def search(self, query_embedding, topk, dis_threshold=2):
+ scored = [(sum(a * b for a, b in zip(query_embedding, embedding)), chunk) for embedding, chunk in self.entries]
+ return [chunk for _, chunk in sorted(scored, reverse=True)[:topk]]
-class OpenAIEmbedding:
- """模拟的OpenAIEmbedding类"""
- def __init__(self, api_key=None, model=None):
- self.api_key = api_key
- self.model = model or "text-embedding-ada-002"
+def test_rag_document_split_index_and_retrieve_use_production_operators():
+ from hugegraph_llm.document.chunk_split import ChunkSplitter
+ from hugegraph_llm.operators.index_op.build_vector_index import BuildVectorIndex
+ from hugegraph_llm.operators.index_op.vector_index_query import VectorIndexQuery
- def get_text_embedding(self, text):
- # 返回一个固定维度的模拟嵌入向量
- return [0.1] * 1536
+ InMemoryVectorIndex.stores.clear()
+ documents = [
+ "HugeGraph is a high performance graph database.",
+ "HugeGraph supports OLTP and OLAP workloads.",
+ ]
+ chunks = ChunkSplitter(split_type="paragraph", language="en").split(documents)
+ embedding = DeterministicEmbedding()
+ BuildVectorIndex(embedding=embedding, vector_index=InMemoryVectorIndex).run({"chunks": chunks})
+ context = VectorIndexQuery(vector_index=InMemoryVectorIndex, embedding=embedding, topk=2).run(
+ {"query": "HugeGraph OLAP"}
+ )
-class OpenAILLM:
- """模拟的OpenAILLM类"""
-
- def __init__(self, api_key=None, model=None):
- self.api_key = api_key
- self.model = model or "gpt-3.5-turbo"
-
- def generate(self, prompt):
- # 返回一个模拟的回答
- return f"这是对'{prompt}'的模拟回答"
-
-
-class VectorIndexRetriever:
- """模拟的VectorIndexRetriever类"""
-
- def __init__(self, vector_index, embedding_model, top_k=5):
- self.vector_index = vector_index
- self.embedding_model = embedding_model
- self.top_k = top_k
-
- def retrieve(self, query):
- query_vector = self.embedding_model.get_text_embedding(query)
- return self.vector_index.search(query_vector, self.top_k)
-
-
-class TestRAGPipeline(unittest.TestCase):
- """测试RAG流程的集成测试"""
-
- def setUp(self):
- """测试前的准备工作"""
- # 如果需要跳过外部服务测试,则跳过
- if should_skip_external():
- self.skipTest("跳过需要外部服务的测试")
-
- # 创建测试文档
- self.test_docs = [
- create_test_document("HugeGraph是一个高性能的图数据库"),
- create_test_document("HugeGraph支持OLTP和OLAP"),
- create_test_document("HugeGraph-LLM是HugeGraph的LLM扩展"),
- ]
-
- # 创建向量索引
- self.embedding_model = OpenAIEmbedding()
- self.vector_index = VectorIndex(dimension=1536)
-
- # 创建LLM模型
- self.llm = OpenAILLM()
-
- # 创建检索器
- self.retriever = VectorIndexRetriever(
- vector_index=self.vector_index, embedding_model=self.embedding_model, top_k=2
- )
-
- @with_mock_openai_embedding
- def test_document_indexing(self, *args):
- """测试文档索引过程"""
- # 将文档添加到向量索引
- for doc in self.test_docs:
- self.vector_index.add_document(doc, self.embedding_model)
-
- # 验证索引中的文档数量
- self.assertEqual(len(self.vector_index), len(self.test_docs))
-
- @with_mock_openai_embedding
- def test_document_retrieval(self, *args):
- """测试文档检索过程"""
- # 将文档添加到向量索引
- for doc in self.test_docs:
- self.vector_index.add_document(doc, self.embedding_model)
-
- # 执行检索
- query = "什么是HugeGraph"
- results = self.retriever.retrieve(query)
-
- # 验证检索结果
- self.assertIsNotNone(results)
- self.assertLessEqual(len(results), 2) # top_k=2
-
- @with_mock_openai_embedding
- @with_mock_openai_client
- def test_rag_end_to_end(self, *args):
- """测试RAG端到端流程"""
- # 将文档添加到向量索引
- for doc in self.test_docs:
- self.vector_index.add_document(doc, self.embedding_model)
-
- # 执行检索
- query = "什么是HugeGraph"
- retrieved_docs = self.retriever.retrieve(query)
-
- # 构建提示词
- context = "\n".join([doc.content for doc in retrieved_docs])
- prompt = f"基于以下信息回答问题:\n\n{context}\n\n问题: {query}"
-
- # 生成回答
- response = self.llm.generate(prompt)
-
- # 验证回答
- self.assertIsNotNone(response)
- self.assertIsInstance(response, str)
- self.assertGreater(len(response), 0)
-
- def test_document_loading_and_splitting(self):
- """测试文档加载和分割"""
- # 创建临时文件
- with tempfile.NamedTemporaryFile(mode="w+", delete=False, encoding="utf-8") as temp_file:
- temp_file.write("这是一个测试文档。\n它包含多个段落。\n\n这是第二个段落。")
- temp_file_path = temp_file.name
-
- try:
- # 加载文档
- loader = TextLoader(temp_file_path)
- docs = loader.load()
-
- # 验证文档加载
- self.assertEqual(len(docs), 1)
- self.assertIn("这是一个测试文档", docs[0].content)
-
- # 分割文档
- splitter = RecursiveCharacterTextSplitter(chunk_size=10, chunk_overlap=0)
- split_docs = splitter.split_documents(docs)
-
- # 验证文档分割
- self.assertGreater(len(split_docs), 1)
- finally:
- # 清理临时文件
- os.unlink(temp_file_path)
-
-
-if __name__ == "__main__":
- unittest.main()
+ assert chunks
+ assert context["vector_result"]
+ assert any("OLAP" in item for item in context["vector_result"])