You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "chiruu12 (via GitHub)" <gi...@apache.org> on 2025/02/26 04:31:52 UTC
[PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
chiruu12 opened a new pull request, #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2683878586
@imbajin Can you please review and tell me if I need to make any changes??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1983215293
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
@@ -19,34 +19,36 @@
from fastapi import status, APIRouter, HTTPException
-from hugegraph_llm.api.exceptions.rag_exceptions import generate_response
from hugegraph_llm.api.models.rag_requests import (
RAGRequest,
- GraphConfigRequest,
- LLMConfigRequest,
- RerankerConfigRequest,
GraphRAGRequest,
)
from hugegraph_llm.config import huge_settings
-from hugegraph_llm.api.models.rag_response import RAGResponse
-from hugegraph_llm.config import llm_settings, prompt
+from hugegraph_llm.config import prompt
from hugegraph_llm.utils.log import log
+from hugegraph_llm.api.config_api import (
+ graph_config_route,
+ llm_config_route,
+ embedding_config_route,
+ rerank_config_route
+)
+
# pylint: disable=too-many-statements
-def rag_http_api(
+async def rag_http_api(
Review Comment:
if we use a async api call, seems we should also update the usage in `app.py`?
<img width="1135" alt="image" src="https://github.com/user-attachments/assets/37f79a4d-1256-40d3-aa51-95f8af9418da" />
still not work as expected (Also note `admin_api` need used in the same way
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687053334
@imbajin sir I am having a little problem locating pyhugegraph can you tell me where it is ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687193698
@imbajin shall I commit the changes that I made?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1974696942
##########
hugegraph-llm/src/hugegraph_llm/api/stream_api.py:
##########
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import json
+import asyncio
+
+from fastapi import status, APIRouter, HTTPException
+from fastapi.responses import StreamingResponse
+
+from hugegraph_llm.api.models.rag_requests import (
+ RAGRequest,
+ GraphRAGRequest,
+)
+from hugegraph_llm.config import prompt
+from hugegraph_llm.utils.log import log
+
+
+async def stream_http_api(
+ router: APIRouter,
+ rag_answer_stream_func,
+ graph_rag_recall_stream_func,
+):
+ @router.post("/rag/stream", status_code=status.HTTP_200_OK)
+ async def rag_answer_stream_api(req: RAGRequest):
+ if not req.stream:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
+ detail="Stream parameter must be set to True for streaming endpoint"
+ )
+
+ async def generate_stream():
+ try:
+ async for chunk in rag_answer_stream_func(
+ text=req.query,
+ raw_answer=req.raw_answer,
+ vector_only_answer=req.vector_only,
+ graph_only_answer=req.graph_only,
+ graph_vector_answer=req.graph_vector_answer,
+ graph_ratio=req.graph_ratio,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ custom_related_information=req.custom_priority_info,
+ answer_prompt=req.answer_prompt or prompt.answer_prompt,
+ keywords_extract_prompt=req.keywords_extract_prompt
+ or prompt.keywords_extract_prompt,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
+ ):
+ # Format as Server-Sent Events
+ data = json.dumps({
+ "query": req.query,
+ "chunk": chunk
+ })
+ yield f"data: {data}\n\n"
+ await asyncio.sleep(0.01) # Small delay to prevent overwhelming
+ except Exception as e:
+ log.error(f"Error in streaming RAG response: {e}")
+ error_data = json.dumps({"error": str(e)})
+ yield f"data: {error_data}\n\n"
+
+ return StreamingResponse(
+ generate_stream(),
Review Comment:
## Information exposure through an exception
[Stack trace information](1) flows to this location and may be exposed to an external user.
[Show more details](https://github.com/apache/incubator-hugegraph-ai/security/code-scanning/26)
##########
hugegraph-llm/src/hugegraph_llm/api/stream_api.py:
##########
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import json
+import asyncio
+
+from fastapi import status, APIRouter, HTTPException
+from fastapi.responses import StreamingResponse
+
+from hugegraph_llm.api.models.rag_requests import (
+ RAGRequest,
+ GraphRAGRequest,
+)
+from hugegraph_llm.config import prompt
+from hugegraph_llm.utils.log import log
+
+
+async def stream_http_api(
+ router: APIRouter,
+ rag_answer_stream_func,
+ graph_rag_recall_stream_func,
+):
+ @router.post("/rag/stream", status_code=status.HTTP_200_OK)
+ async def rag_answer_stream_api(req: RAGRequest):
+ if not req.stream:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
+ detail="Stream parameter must be set to True for streaming endpoint"
+ )
+
+ async def generate_stream():
+ try:
+ async for chunk in rag_answer_stream_func(
+ text=req.query,
+ raw_answer=req.raw_answer,
+ vector_only_answer=req.vector_only,
+ graph_only_answer=req.graph_only,
+ graph_vector_answer=req.graph_vector_answer,
+ graph_ratio=req.graph_ratio,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ custom_related_information=req.custom_priority_info,
+ answer_prompt=req.answer_prompt or prompt.answer_prompt,
+ keywords_extract_prompt=req.keywords_extract_prompt
+ or prompt.keywords_extract_prompt,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
+ ):
+ # Format as Server-Sent Events
+ data = json.dumps({
+ "query": req.query,
+ "chunk": chunk
+ })
+ yield f"data: {data}\n\n"
+ await asyncio.sleep(0.01) # Small delay to prevent overwhelming
+ except Exception as e:
+ log.error(f"Error in streaming RAG response: {e}")
+ error_data = json.dumps({"error": str(e)})
+ yield f"data: {error_data}\n\n"
+
+ return StreamingResponse(
+ generate_stream(),
+ media_type="text/event-stream",
+ headers={
+ "Cache-Control": "no-cache",
+ "Connection": "keep-alive",
+ }
+ )
+
+ @router.post("/rag/graph/stream", status_code=status.HTTP_200_OK)
+ async def graph_rag_recall_stream_api(req: GraphRAGRequest):
+ if not req.stream:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
+ detail="Stream parameter must be set to True for streaming endpoint"
+ )
+
+ async def generate_graph_stream():
+ try:
+ async for chunk in graph_rag_recall_stream_func(
+ query=req.query,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ custom_related_information=req.custom_priority_info,
+ gremlin_prompt=req.gremlin_prompt or
+ prompt.gremlin_generate_prompt,
+ ):
+ if isinstance(chunk, dict):
+ params = [
+ "query",
+ "keywords",
+ "match_vids",
+ "graph_result_flag",
+ "gremlin",
+ "graph_result",
+ "vertex_degree_list",
+ ]
+ user_result = {key: chunk[key] for key in params if key in chunk}
+ data = json.dumps({"graph_recall": user_result})
+ else:
+ data = json.dumps({"graph_recall": json.dumps(chunk)})
+
+ yield f"data: {data}\n\n"
+ await asyncio.sleep(0.01) # Small delay
+ except TypeError as e:
+ log.error(f"TypeError in streaming graph RAG recall: {e}")
+ error_data = json.dumps({"error": str(e)})
+ yield f"data: {error_data}\n\n"
+ except Exception as e:
+ log.error(f"Unexpected error in streaming graph RAG recall: {e}")
+ error_data = json.dumps({"error": "An unexpected error occurred."})
+ yield f"data: {error_data}\n\n"
+
+ return StreamingResponse(
+ generate_graph_stream(),
Review Comment:
## Information exposure through an exception
[Stack trace information](1) flows to this location and may be exposed to an external user.
[Show more details](https://github.com/apache/incubator-hugegraph-ai/security/code-scanning/27)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2692532335
> If you're saying that repairing `pylint` is troublesome, you can ignore it for now and I'll help to fix it together. (Focus on your code logic)
>
> As for automated Lint repair tools, I'm not very familiar with them. If there are any suitable ones, could recommend it and introduce them into our project, which would be even better
Sir I check the repo I was telling you about shall I share the script that the used?
I think they used black and ruff
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2698462826
> > @imbajin sure sir will do can give me a little guidance on what will be the most effective way I can resolve the conflict?
>
> Maybe in most cases, it should be to confirm the difference before and after the conflict? Then you can compare the uncertain areas in your IDE.
Sir we are using python 2.7 for the project rigth?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1970897554
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
@@ -33,76 +36,243 @@
def rag_http_api(
- router: APIRouter,
- rag_answer_func,
- graph_rag_recall_func,
- apply_graph_conf,
- apply_llm_conf,
- apply_embedding_conf,
- apply_reranker_conf,
+ router: APIRouter,
+ rag_answer_func,
+ graph_rag_recall_func,
+ apply_graph_conf,
+ apply_llm_conf,
+ apply_embedding_conf,
+ apply_reranker_conf,
+ rag_answer_stream_func=None,
+ graph_rag_recall_stream_func=None,
):
- @router.post("/rag", status_code=status.HTTP_200_OK)
- def rag_answer_api(req: RAGRequest):
- result = rag_answer_func(
- text=req.query,
- raw_answer=req.raw_answer,
- vector_only_answer=req.vector_only,
- graph_only_answer=req.graph_only,
- graph_vector_answer=req.graph_vector_answer,
- graph_ratio=req.graph_ratio,
- rerank_method=req.rerank_method,
- near_neighbor_first=req.near_neighbor_first,
- custom_related_information=req.custom_priority_info,
- answer_prompt=req.answer_prompt or prompt.answer_prompt,
- keywords_extract_prompt=req.keywords_extract_prompt or prompt.keywords_extract_prompt,
- gremlin_tmpl_num=req.gremlin_tmpl_num,
- gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
- )
- # TODO: we need more info in the response for users to understand the query logic
- return {
- "query": req.query,
- **{
- key: value
- for key, value in zip(["raw_answer", "vector_only", "graph_only", "graph_vector_answer"], result)
- if getattr(req, key)
- },
- }
+ async def stream_rag_answer(
+ text,
+ raw_answer,
+ vector_only_answer,
+ graph_only_answer,
+ graph_vector_answer,
+ graph_ratio,
+ rerank_method,
+ near_neighbor_first,
+ custom_related_information,
+ answer_prompt,
+ keywords_extract_prompt,
+ gremlin_tmpl_num,
+ gremlin_prompt,
+ ) -> AsyncGenerator[str, None]:
+ """
+ Stream the RAG answer results
+ """
+ if rag_answer_stream_func:
+ # If a streaming-specific function exists, use it
+ async for chunk in rag_answer_stream_func(
+ text=text,
+ raw_answer=raw_answer,
+ vector_only_answer=vector_only_answer,
+ graph_only_answer=graph_only_answer,
+ graph_vector_answer=graph_vector_answer,
+ graph_ratio=graph_ratio,
+ rerank_method=rerank_method,
+ near_neighbor_first=near_neighbor_first,
+ custom_related_information=custom_related_information,
+ answer_prompt=answer_prompt,
+ keywords_extract_prompt=keywords_extract_prompt,
+ gremlin_tmpl_num=gremlin_tmpl_num,
+ gremlin_prompt=gremlin_prompt,
+ ):
+ yield f"data: {json.dumps({'chunk': chunk})}\n\n"
+ else:
+ # Otherwise, use the normal function but adapt it for streaming
+ # by sending the entire result at once
+ result = rag_answer_func(
+ text=text,
+ raw_answer=raw_answer,
+ vector_only_answer=vector_only_answer,
+ graph_only_answer=graph_only_answer,
+ graph_vector_answer=graph_vector_answer,
+ graph_ratio=graph_ratio,
+ rerank_method=rerank_method,
+ near_neighbor_first=near_neighbor_first,
+ custom_related_information=custom_related_information,
+ answer_prompt=answer_prompt,
+ keywords_extract_prompt=keywords_extract_prompt,
+ gremlin_tmpl_num=gremlin_tmpl_num,
+ gremlin_prompt=gremlin_prompt,
+ )
- @router.post("/rag/graph", status_code=status.HTTP_200_OK)
- def graph_rag_recall_api(req: GraphRAGRequest):
- try:
- result = graph_rag_recall_func(
- query=req.query,
- gremlin_tmpl_num=req.gremlin_tmpl_num,
+ response_data = {
+ "query": text,
+ **{
+ key: value
+ for key, value in zip(["raw_answer", "vector_only", "graph_only", "graph_vector_answer"], result)
+ if eval(key) # Convert string to boolean
+ },
+ }
+
+ yield f"data: {json.dumps(response_data)}\n\n"
+ # Signal end of stream
+ yield "data: [DONE]\n\n"
+
+ async def stream_graph_rag_recall(
+ query,
+ gremlin_tmpl_num,
+ rerank_method,
+ near_neighbor_first,
+ custom_related_information,
+ gremlin_prompt,
+ ) -> AsyncGenerator[str, None]:
+ """
+ Stream the graph RAG recall results
+ """
+ if graph_rag_recall_stream_func:
+ # If a streaming-specific function exists, use it
+ async for chunk in graph_rag_recall_stream_func(
+ query=query,
+ gremlin_tmpl_num=gremlin_tmpl_num,
+ rerank_method=rerank_method,
+ near_neighbor_first=near_neighbor_first,
+ custom_related_information=custom_related_information,
+ gremlin_prompt=gremlin_prompt,
+ ):
+ yield f"data: {json.dumps({'chunk': chunk})}\n\n"
+ else:
+ # Otherwise, use the normal function but adapt it for streaming
+ try:
+ result = graph_rag_recall_func(
+ query=query,
+ gremlin_tmpl_num=gremlin_tmpl_num,
+ rerank_method=rerank_method,
+ near_neighbor_first=near_neighbor_first,
+ custom_related_information=custom_related_information,
+ gremlin_prompt=gremlin_prompt,
+ )
+
+ if isinstance(result, dict):
+ params = [
+ "query",
+ "keywords",
+ "match_vids",
+ "graph_result_flag",
+ "gremlin",
+ "graph_result",
+ "vertex_degree_list",
+ ]
+ user_result = {key: result[key] for key in params if key in result}
+ yield f"data: {json.dumps({'graph_recall': user_result})}\n\n"
+ else:
+ # Note: Maybe only for qianfan/wenxin
+ yield f"data: {json.dumps({'graph_recall': json.dumps(result)})}\n\n"
+
+ # Signal end of stream
+ yield "data: [DONE]\n\n"
+
+ except TypeError as e:
+ log.error("TypeError in stream_graph_rag_recall: %s", e)
+ yield f"data: {json.dumps({'error': str(e), 'status': 400})}\n\n"
+ except Exception as e:
+ log.error("Unexpected error occurred: %s", e)
+ yield f"data: {json.dumps({'error': 'An unexpected error occurred.', 'status': 500})}\n\n"
+
+ @router.post("/rag", status_code=status.HTTP_200_OK)
+ async def rag_answer_api(req: RAGRequest):
+ if req.stream:
+ # Return a streaming response
+ return StreamingResponse(
+ stream_rag_answer(
+ text=req.query,
+ raw_answer=req.raw_answer,
+ vector_only_answer=req.vector_only,
+ graph_only_answer=req.graph_only,
+ graph_vector_answer=req.graph_vector_answer,
+ graph_ratio=req.graph_ratio,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ custom_related_information=req.custom_priority_info,
+ answer_prompt=req.answer_prompt or prompt.answer_prompt,
+ keywords_extract_prompt=req.keywords_extract_prompt or prompt.keywords_extract_prompt,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
+ ),
+ media_type="text/event-stream",
+ )
+ else:
+ # Synchronous response (original behavior)
+ result = rag_answer_func(
+ text=req.query,
+ raw_answer=req.raw_answer,
+ vector_only_answer=req.vector_only,
+ graph_only_answer=req.graph_only,
+ graph_vector_answer=req.graph_vector_answer,
+ graph_ratio=req.graph_ratio,
rerank_method=req.rerank_method,
near_neighbor_first=req.near_neighbor_first,
custom_related_information=req.custom_priority_info,
+ answer_prompt=req.answer_prompt or prompt.answer_prompt,
+ keywords_extract_prompt=req.keywords_extract_prompt or prompt.keywords_extract_prompt,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
)
+ # TODO: we need more info in the response for users to understand the query logic
+ return {
+ "query": req.query,
+ **{
+ key: value
+ for key, value in zip(["raw_answer", "vector_only", "graph_only", "graph_vector_answer"], result)
+ if getattr(req, key)
+ },
+ }
- if isinstance(result, dict):
- params = [
- "query",
- "keywords",
- "match_vids",
- "graph_result_flag",
- "gremlin",
- "graph_result",
- "vertex_degree_list",
- ]
- user_result = {key: result[key] for key in params if key in result}
- return {"graph_recall": user_result}
- # Note: Maybe only for qianfan/wenxin
- return {"graph_recall": json.dumps(result)}
-
- except TypeError as e:
- log.error("TypeError in graph_rag_recall_api: %s", e)
- raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
- except Exception as e:
- log.error("Unexpected error occurred: %s", e)
- raise HTTPException(
- status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="An unexpected error occurred."
- ) from e
+ @router.post("/rag/graph", status_code=status.HTTP_200_OK)
+ async def graph_rag_recall_api(req: GraphRAGRequest):
+ if req.stream:
+ # Return a streaming response
+ return StreamingResponse(
+ stream_graph_rag_recall(
+ query=req.query,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ custom_related_information=req.custom_priority_info,
+ gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
+ ),
Review Comment:
## Information exposure through an exception
[Stack trace information](1) flows to this location and may be exposed to an external user.
[Show more details](https://github.com/apache/incubator-hugegraph-ai/security/code-scanning/25)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2683883675
> @imbajin Can you please review and tell me if I need to make any changes??
Thanks, can we add some basic tests to it? You can create it separately first, and we will also enable it in the CI actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2703238109
@imbajin Sir please check the pr and let me know if there is any changes that I need to make
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2705546278
> > > @imbajin Sir please check the pr and let me know if there is any changes that I need to make
> >
> >
> > Fine, we add a separate `async_streaming_generate` in the latest change/commit, seems we could use it to provide the aysnc_streaming api?
> > refer
> > https://github.com/apache/incubator-hugegraph-ai/blob/7ae5d6fcc1013bc39164672355a8270f078bff8d/hugegraph-llm/src/hugegraph_llm/models/llms/base.py#L51
> >
> > PS: Remember pull code first~
>
> @chiruu12 Also note the `stream_http_api` is also not bounded to gradio, and it seems that the design may need to be improved. (and remember to use `agenerate_streaming()` function if we can)
>
> And it's better to follow the design of LLM such as `OpenAI` to provide a `stream` parameter instead of adding a separate API path. The internal logic can be implemented separately in two functions to provide a unified interface for the user layer
understood sir I will try to get this done as soon as my exams are over and then maybe we can work on the other issue too!
Also had made an elementary demo for the agent. Let me know if you will get some time to review it too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2689681714
> > sir I have been going through them manually one by one, which is pretty time-consuming. Is there a better approach I'm missing? Maybe a tool or command that can automatically fix some of the common pylint issues instead of just listing them? I'm not very experienced with pylint and wondered if there's a more efficient way to handle this than the manual corrections I've been doing. Like I have contributed to openvino once and they told me to run a small command using black and then the issue with spacing was resolved automatically
> > and I saw that there are a few errors which are coming like the FIXME: line 31: E0702: Raising dict while only classes or instances are allowed (raising-bad-type) This error stems from admin_api.py class
>
> If you're saying that repairing `pylint` is troublesome, you can ignore it for now and I'll help to fix it together.
>
> As for automated Lint repair tools, I'm not very familiar with them. If there are any suitable ones, could recommend it and introduce them into our project, which would be even better
Thank you so much sir I really appreciate your guidance. I'm still getting up to speed with best practices, but I'm confident about handling the code logic correctly. If any issues arise, I'll make sure to work on them swiftly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1983215293
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
@@ -19,34 +19,36 @@
from fastapi import status, APIRouter, HTTPException
-from hugegraph_llm.api.exceptions.rag_exceptions import generate_response
from hugegraph_llm.api.models.rag_requests import (
RAGRequest,
- GraphConfigRequest,
- LLMConfigRequest,
- RerankerConfigRequest,
GraphRAGRequest,
)
from hugegraph_llm.config import huge_settings
-from hugegraph_llm.api.models.rag_response import RAGResponse
-from hugegraph_llm.config import llm_settings, prompt
+from hugegraph_llm.config import prompt
from hugegraph_llm.utils.log import log
+from hugegraph_llm.api.config_api import (
+ graph_config_route,
+ llm_config_route,
+ embedding_config_route,
+ rerank_config_route
+)
+
# pylint: disable=too-many-statements
-def rag_http_api(
+async def rag_http_api(
Review Comment:
if we use a async api call, seems we should also update the usage in `app.py`?
<img width="1135" alt="image" src="https://github.com/user-attachments/assets/37f79a4d-1256-40d3-aa51-95f8af9418da" />
still not work as expected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2684613361
Also need handle the lint error: (see https://github.com/apache/incubator-hugegraph-ai/blob/main/style/code_format_and_analysis.sh to run it local)
<img width="717" alt="image" src="https://github.com/user-attachments/assets/5f48e77a-629d-4846-b7c4-500366235f24" />
Another improvement is that the streaming api can be extracted separately into a separate file similar to `stream_api.py`? It is advisable to separate the functions related to/config into `config_api.py`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2688765627
@imbajin I have separated the files for stream_api and config too and tried to make the format better for passing the pylint test too I had to manually do it any way we can do it using a script? whenever i am running the script it is telling me I have to do it manually
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2685833528
@imbajin thx for the suggestion will do so!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2683885492
> > @imbajin Can you please review and tell me if I need to make any changes??
>
> Thanks, can we add some basic tests to it? You can create it separately first, and we will also enable it in the CI actions
Ok sir will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2693602410
Due to #184 merged, we could resolve the conflicts first
<img width="748" alt="image" src="https://github.com/user-attachments/assets/710e342f-0bc5-4152-bc25-b06700ae8e7a" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2689631708
> > @imbajin I have separated the files for stream_api and config too and tried to make the format better for passing the pylint test too I had to manually do it any way we can do it using a script? whenever i am running the script it is telling me I have to do it manually
>
> Not quite sure what you mean when you say the `script` content? Should Pytest be able to specify multiple tests at once without having to execute them separately multiple times? Can you provide specific explanations for it?
sir I have been going through them manually one by one, which is pretty time-consuming. Is there a better approach I'm missing? Maybe a tool or command that can automatically fix some of the common pylint issues instead of just listing them? I'm not very experienced with pylint and wondered if there's a more efficient way to handle this than the manual corrections I've been doing.
Like I have contributed to openvino once and they told me to run a small command using black and then the issue with spacing was resolved automatically
and I saw that there are a few errors which are coming like the
# FIXME: line 31: E0702: Raising dict while only classes or instances are allowed (raising-bad-type)
This error stems from admin_api.py class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1983188212
##########
hugegraph-llm/src/hugegraph_llm/api/stream_api.py:
##########
@@ -0,0 +1,176 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import json
+import asyncio
+
+from fastapi import status, APIRouter, HTTPException
+from fastapi.responses import StreamingResponse
+
+from hugegraph_llm.api.models.rag_requests import (
+ RAGRequest,
+ GraphRAGRequest,
+)
+from hugegraph_llm.config import prompt, huge_settings
+from hugegraph_llm.utils.log import log
+
+# pylint: disable=too-many-statements
+async def stream_http_api(
+ router: APIRouter,
+ rag_answer_stream_func,
+ graph_rag_recall_stream_func,
+):
+ @router.post("/rag/stream", status_code=status.HTTP_200_OK)
+ async def rag_answer_stream_api(req: RAGRequest):
+ if not req.stream:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
+ detail="Stream parameter must be set to True for streaming endpoint"
+ )
+
+ if req.client_config:
+ huge_settings.graph_ip = req.client_config.ip
+ huge_settings.graph_port = req.client_config.port
+ huge_settings.graph_name = req.client_config.name
+ huge_settings.graph_user = req.client_config.user
+ huge_settings.graph_pwd = req.client_config.pwd
+ huge_settings.graph_space = req.client_config.gs
+
+ async def generate_stream():
+ try:
+ async for chunk in rag_answer_stream_func(
+ text=req.query,
+ raw_answer=req.raw_answer,
+ vector_only_answer=req.vector_only,
+ graph_only_answer=req.graph_only,
+ graph_vector_answer=req.graph_vector_answer,
+ graph_ratio=req.graph_ratio,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ max_graph_items=req.max_graph_items,
+ topk_return_results=req.topk_return_results,
+ vector_dis_threshold=req.vector_dis_threshold,
+ topk_per_keyword=req.topk_per_keyword,
+ # Keep prompt params in the end
+ custom_related_information=req.custom_priority_info,
+ answer_prompt=req.answer_prompt or prompt.answer_prompt,
+ keywords_extract_prompt=req.keywords_extract_prompt or prompt.keywords_extract_prompt,
+ gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
+ ):
+ # Format as Server-Sent Events
+ data = json.dumps({
+ "query": req.query,
+ "chunk": chunk
+ })
+ yield f"data: {data}\n\n"
+ await asyncio.sleep(0.01) # Small delay to prevent overwhelming
+ except (ValueError, TypeError) as e: # More specific exceptions
+ log.error("Error in streaming RAG response: %s", e)
+ error_data = json.dumps({"error": str(e)})
+ yield f"data: {error_data}\n\n"
+ except Exception as e: # pylint: disable=broad-exception-caught
+ # We need to catch all exceptions here to ensure proper error response
+ log.error("Unexpected error in streaming RAG response: %s", e)
+ error_data = json.dumps({"error": "An unexpected error occurred"})
+ yield f"data: {error_data}\n\n"
+
+ return StreamingResponse(
+ generate_stream(),
+ media_type="text/event-stream",
+ headers={
+ "Cache-Control": "no-cache",
+ "Connection": "keep-alive",
+ }
+ )
+
+ @router.post("/rag/graph/stream", status_code=status.HTTP_200_OK)
+ async def graph_rag_recall_stream_api(req: GraphRAGRequest):
+ if not req.stream:
+ raise HTTPException(
+ status_code=status.HTTP_400_BAD_REQUEST,
+ detail="Stream parameter must be set to True for streaming endpoint"
+ )
+
+ # Set graph config if provided
+ if req.client_config:
+ huge_settings.graph_ip = req.client_config.ip
+ huge_settings.graph_port = req.client_config.port
+ huge_settings.graph_name = req.client_config.name
+ huge_settings.graph_user = req.client_config.user
+ huge_settings.graph_pwd = req.client_config.pwd
+ huge_settings.graph_space = req.client_config.gs
+
+ async def generate_graph_stream():
+ try:
+ async for chunk in graph_rag_recall_stream_func(
+ query=req.query,
+ max_graph_items=req.max_graph_items,
+ topk_return_results=req.topk_return_results,
+ vector_dis_threshold=req.vector_dis_threshold,
+ topk_per_keyword=req.topk_per_keyword,
+ gremlin_tmpl_num=req.gremlin_tmpl_num,
+ rerank_method=req.rerank_method,
+ near_neighbor_first=req.near_neighbor_first,
+ custom_related_information=req.custom_priority_info,
+ gremlin_prompt=req.gremlin_prompt or prompt.gremlin_generate_prompt,
+ get_vertex_only=req.get_vertex_only
+ ):
+ # Handle vertex details for get_vertex_only flag
+ if req.get_vertex_only and isinstance(chunk, dict) and "match_vids" in chunk:
+ from hugegraph_llm.operators.hugegraph_op.graph_rag_query import GraphRAGQuery
+ graph_rag = GraphRAGQuery()
+ graph_rag.init_client(chunk)
+ vertex_details = await graph_rag.get_vertex_details(chunk["match_vids"])
Review Comment:
'await' operator can't be used on list?
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
@@ -108,7 +110,7 @@ def graph_rag_recall_api(req: GraphRAGRequest):
from hugegraph_llm.operators.hugegraph_op.graph_rag_query import GraphRAGQuery
graph_rag = GraphRAGQuery()
graph_rag.init_client(result)
- vertex_details = graph_rag.get_vertex_details(result["match_vids"])
+ vertex_details = await graph_rag.get_vertex_details(result["match_vids"])
Review Comment:
so as here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2705518073
> > @imbajin Sir please check the pr and let me know if there is any changes that I need to make
>
> Fine, we add a separate `async_streaming_generate` in the latest change/commit, seems we could use it to provide the aysnc_streaming api?
>
> refer
>
> https://github.com/apache/incubator-hugegraph-ai/blob/7ae5d6fcc1013bc39164672355a8270f078bff8d/hugegraph-llm/src/hugegraph_llm/models/llms/base.py#L51
>
> PS: Remember pull code first~
@chiruu12 Also note the `stream_http_api` is also not bound, and it seems that the design may need to be improved. (and remember to use `async_streaming_generate` function if we can)
And it's better to follow the design of LLM such as `OpenAI` to provide a `stream` parameter instead of adding a separate API path. The internal logic can be implemented separately in two functions to provide a unified interface for the user layer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2694080592
> @imbajin sure sir will do can give me a little guidance on what will be the most effective way I can resolve the conflict?
Maybe in most cases, it should be to confirm the difference before and after the conflict? Then you can compare the uncertain areas in your IDE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687138908
@imbajin yes sir will start doing it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2689621558
> @imbajin I have separated the files for stream_api and config too and tried to make the format better for passing the pylint test too I had to manually do it any way we can do it using a script? whenever i am running the script it is telling me I have to do it manually
Not quite sure what you mean when you say the `script` content? Should Pytest be able to specify multiple tests at once without having to execute them separately multiple times? Can you provide specific explanations for it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2702680896
@imbajin sir I think the error is not related to my pr ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 closed pull request #179: feat(llm): support async/streaming output mode in api layer
URL: https://github.com/apache/hugegraph-ai/pull/179
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687100432
> @imbajin sir I am having a little problem locating pyhugegraph can you tell me where it is ?
I have already fix the problem, u could ignore it
<img width="1269" alt="image" src="https://github.com/user-attachments/assets/24c5cbc7-fd31-468d-8477-baefe9f99a78" />
BTW, py-client here-> https://github.com/apache/incubator-hugegraph-ai/tree/main/hugegraph-python-client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687131352
> @imbajin I will push the changes I have made and shall i also add the test file you asked me to make? I was just working on finishing it up and one concern that I have is how will we know if the api is working fine without testing it sir??
Yes, we do need to start testing. The previous section was missing. Can you first refer to the previous tests and add an API test file, and then test it briefly? Additionally, it can also be enabled in CI, similar to Python client's tests (https://github.com/apache/incubator-hugegraph-ai/blob/e78792f5b25443e953671c2f752a1be2661499e2/.github/workflows/hugegraph-python-client.yml#L66)
<img width="686" alt="image" src="https://github.com/user-attachments/assets/e7ffed37-9854-4340-a64d-6b07857b95ab" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2703432168
> @imbajin Sir please check the pr and let me know if there is any changes that I need to make
Fine, we add a separate `async_streaming_generate` in the latest change/commit, seems we could use it to provide the aysnc_streaming api?
refer https://github.com/apache/incubator-hugegraph-ai/blob/7ae5d6fcc1013bc39164672355a8270f078bff8d/hugegraph-llm/src/hugegraph_llm/models/llms/base.py#L51
PS: I'll fix CI problems soon:)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1983265705
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
@@ -19,34 +19,36 @@
from fastapi import status, APIRouter, HTTPException
-from hugegraph_llm.api.exceptions.rag_exceptions import generate_response
from hugegraph_llm.api.models.rag_requests import (
RAGRequest,
- GraphConfigRequest,
- LLMConfigRequest,
- RerankerConfigRequest,
GraphRAGRequest,
)
from hugegraph_llm.config import huge_settings
-from hugegraph_llm.api.models.rag_response import RAGResponse
-from hugegraph_llm.config import llm_settings, prompt
+from hugegraph_llm.config import prompt
from hugegraph_llm.utils.log import log
+from hugegraph_llm.api.config_api import (
+ graph_config_route,
+ llm_config_route,
+ embedding_config_route,
+ rerank_config_route
+)
+
# pylint: disable=too-many-statements
-def rag_http_api(
+async def rag_http_api(
Review Comment:
I will try to resolve the issues and then I will ping you again sir!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1983215293
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
@@ -19,34 +19,36 @@
from fastapi import status, APIRouter, HTTPException
-from hugegraph_llm.api.exceptions.rag_exceptions import generate_response
from hugegraph_llm.api.models.rag_requests import (
RAGRequest,
- GraphConfigRequest,
- LLMConfigRequest,
- RerankerConfigRequest,
GraphRAGRequest,
)
from hugegraph_llm.config import huge_settings
-from hugegraph_llm.api.models.rag_response import RAGResponse
-from hugegraph_llm.config import llm_settings, prompt
+from hugegraph_llm.config import prompt
from hugegraph_llm.utils.log import log
+from hugegraph_llm.api.config_api import (
+ graph_config_route,
+ llm_config_route,
+ embedding_config_route,
+ rerank_config_route
+)
+
# pylint: disable=too-many-statements
-def rag_http_api(
+async def rag_http_api(
Review Comment:
if we use a async api call, seems we should also update the usage in `app.py`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/hugegraph-ai/pull/179#issuecomment-5364330594
Closing this one. It stalled on my end back in March 2025, and the api layer has moved a long way since then, so the diff no longer applies and reworking it in place would be a rewrite rather than a rebase.
Thanks for the patience you showed on it @imbajin. If async and streaming output in the api layer is still wanted, I would rather open a fresh PR against current main with the tests in from the start.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2689673306
> sir I have been going through them manually one by one, which is pretty time-consuming. Is there a better approach I'm missing? Maybe a tool or command that can automatically fix some of the common pylint issues instead of just listing them? I'm not very experienced with pylint and wondered if there's a more efficient way to handle this than the manual corrections I've been doing. Like I have contributed to openvino once and they told me to run a small command using black and then the issue with spacing was resolved automatically
>
> and I saw that there are a few errors which are coming like the FIXME: line 31: E0702: Raising dict while only classes or instances are allowed (raising-bad-type) This error stems from admin_api.py class
If you're saying that repairing `pylint` is troublesome, you can ignore it for now and I'll help to fix it together.
As for automated Lint repair tools, I'm not very familiar with them. If there are any suitable ones, I can recommend them and introduce them into our project, which would be even better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687123963
@imbajin I will push the changes I have made and shall i also add the test file you asked me to make? I was just working on finishing it up and one concern that I have is how will we know if the api is working fine without testing it sir??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2687293550
> @imbajin shall I commit the changes that I made?
Sure, u can submit new changes at any time without any concerns
Also note pylint~
<img width="808" alt="image" src="https://github.com/user-attachments/assets/d50f59e8-95ea-4c34-afdf-5e9d05c4eb42" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] Adding support for async and streaming output mode [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2692778256
> Sir I check the repo I was telling you about shall I share the script that the used? I think they used black and ruff
Okay, regarding the issue of lint and automatic formatting, we can create a separate PR or issue for management, and we will keep the current PR independent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "chiruu12 (via GitHub)" <gi...@apache.org>.
chiruu12 commented on PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#issuecomment-2693880857
@imbajin sure sir will do can give me a little guidance on what will be the most effective way I can resolve the conflict?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] feat(llm): support async/streaming output mode in api layer [incubator-hugegraph-ai]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin commented on code in PR #179:
URL: https://github.com/apache/incubator-hugegraph-ai/pull/179#discussion_r1983109364
##########
hugegraph-llm/src/hugegraph_llm/api/rag_api.py:
##########
Review Comment:
Hi, could tell me how u test the APIs? By directly request them?
The gradio UI loss the API link now
Before:
<img width="1304" alt="image" src="https://github.com/user-attachments/assets/094a5865-6b6c-4b67-9b15-d45ffd268cba" />
Now:
<img width="1543" alt="image" src="https://github.com/user-attachments/assets/9adeaade-d38b-4d69-92cc-f1ffe524e7fc" />
Maybe refer here: (Or Gradio's mount doc?)
https://github.com/apache/incubator-hugegraph-ai/blob/7ae5d6fcc1013bc39164672355a8270f078bff8d/hugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.py#L182
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org