You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "cshuo (via GitHub)" <gi...@apache.org> on 2026/10/08 08:13:05 UTC
[I] Integrate hudi-agent-gateway and BI clients with Spark Connect [hudi]
cshuo opened a new issue, #20232:
URL: https://github.com/apache/hudi/issues/20232
### Task Description
Connect `hudi-agent-gateway`, Superset, and Zeppelin to the Spark Connect endpoint through Foundry ADBC. Prioritize gateway integration, then package the BI examples. Background: [discussion #19843](https://github.com/apache/hudi/discussions/19843).
### Scope
- Add an ADBC-backed Spark Connect client to the gateway, including Spark SQL and metadata handling, bounded result fetching, connection cleanup, and timeout/cancellation handling with clearly documented limitations.
- Preserve `list_tables`, `describe_table`, and `query_lakehouse` interfaces and SELECT-only guardrails. Gateway write tools remain out of scope.
- Update gateway dependencies, connection settings, readiness checks, and Helm values. Install the ADBC Driver Manager and Foundry driver in the client runtime.
- Package the Superset SQLAlchemy adapter with pinned dependencies, SQL Lab instructions, and a dashboard example. Support direct DML with Allow DML enabled and handle statements without result sets.
- Provide a Zeppelin `%python` notebook using ADBC for queries, INSERT/UPDATE/DELETE, and result rendering; this is not native `%jdbc` support.
### Acceptance Criteria
- [ ] Gateway table discovery, table descriptions, and Agent/MCP queries work against the Helm-deployed endpoint.
- [ ] Gateway rejects write statements and preserves existing read-only behavior.
- [ ] Superset and Zeppelin can query, execute INSERT/UPDATE/DELETE, and read back the changes through ADBC.
- [ ] Examples include reproducible dependency and connection setup without modifying upstream driver binaries.
### Task Type
Other — client integration and examples.
### Related Issues
Depends on #20231 for the Helm-deployed endpoint.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Integrate hudi-agent-gateway with Spark Connect for Agent and MCP users [hudi]
Posted by "cshuo (via GitHub) via commits" <co...@hudi.apache.org>.
cshuo commented on issue #20232:
URL: https://github.com/apache/hudi/issues/20232#issuecomment-6567638761
Fixed via https://github.com/apache/hudi/pull/20329.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Integrate hudi-agent-gateway with Spark Connect for Agent and MCP users [hudi]
Posted by "cshuo (via GitHub) via commits" <co...@hudi.apache.org>.
cshuo closed issue #20232: Integrate hudi-agent-gateway with Spark Connect for Agent and MCP users
URL: https://github.com/apache/hudi/issues/20232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org