You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "guan404ming (via GitHub)" <gi...@apache.org> on 2026/01/08 07:31:05 UTC
[I] [QDP] QDP POC Roadmap [mahout]
guan404ming opened a new issue, #799:
URL: https://github.com/apache/mahout/issues/799
# QDP POC Roadmap
> Quantum Data Plane development progress tracking
## Phase 1: Foundation & Infrastructure
| Status | Task |
|--------|------|
| ✅ | Rust workspace & config |
| ✅ | Implement robust `build.rs` for Kernels |
---
## Phase 2: The Core
| Status | Task |
|--------|------|
| ✅ | CUDA Kernel Implementation |
| ✅ | Rust CPU Pre-processing |
| ✅ | GPU Memory Allocation and Load |
---
## Phase 3: Zero-copy and Safety
| Status | Task |
|--------|------|
| ✅ | Define `DLManagedTensor` and DLPack structure |
| ✅ | Implement resource `tensor_holder` and deleter |
| ✅ | Core engine API refinement |
---
## Phase 4: Python Binding
| Status | Task |
|--------|------|
| ✅ | PyO3 class wrapping |
| ✅ | Implement DLPack protocol |
---
## Phase 5: Input Format Support
| Status | Task |
|--------|------|
| ⬜ | PyTorch tensor input ([#717](https://github.com/apache/mahout/issues/717)) |
| ⬜ | NumPy array input ([#718](https://github.com/apache/mahout/issues/718)) |
| ⬜ | TensorFlow tensor input ([#719](https://github.com/apache/mahout/issues/719)) |
| ⬜ | Pandas DataFrame input |
---
## Phase 6: Verification and Test
| Status | Task |
|--------|------|
| ✅ | Fidelity test |
| ✅ | Benchmark |
| ⬜ | Different devices testing |
| ⬜ | Benchmark with real dataset |
---
## Phase 7: More Encoders & Architecture (optional)
| Status | Task |
|--------|------|
| ✅ | Amplitude encoding |
| ⬜ | Angle encoding |
| ⬜ | Basis encoding |
| ⬜ | Multi-architecture GPU optimization |
---
## Phase 8: Integration & Release
| Status | Task |
|--------|------|
| ⬜ | Documentation |
| ⬜ | Integrate with QuMat |
| ⬜ | Example notebooks |
| ⬜ | Performance benchmarks |
| ⬜ | PyPI package publishing |
---
## Open Issues by Category
### Meta/Tracking Issues
| Issue | Title | Description |
|-------|-------|-------------|
| [#752](https://github.com/apache/mahout/issues/752) | ROADMAP for new input format support | Master tracking issue |
| [#716](https://github.com/apache/mahout/issues/716) | Supply more input formats | Parent for torch/numpy/tf |
| [#772](https://github.com/apache/mahout/issues/772) | [RFC] Benchmark roadmap | Benchmark planning |
---
### Input Format Support (tracked by #752)
#### PyTorch Support
> **Parent:** [#717](https://github.com/apache/mahout/issues/717) - Supply torch.Tensor in input format
| Issue | Task |
|-------|------|
| [#726](https://github.com/apache/mahout/issues/726) | GPU tensor zero-copy via dlpack protocol |
| [#727](https://github.com/apache/mahout/issues/727) | Add PyTorch tensor optimizations, tests, docs |
#### NumPy Support
> **Parent:** [#718](https://github.com/apache/mahout/issues/718) - Supply np.ndarray in input format
| Issue | Task |
|-------|------|
| [#721](https://github.com/apache/mahout/issues/721) | Add NumPy 1D array input for single sample |
| [#722](https://github.com/apache/mahout/issues/722) | Add NumPy batch input + memory optimizations |
| [#723](https://github.com/apache/mahout/issues/723) | Add NumPy input documentation and more tests |
| [#787](https://github.com/apache/mahout/issues/787) | NumPy file size check to prevent OOM (followup) |
| [#789](https://github.com/apache/mahout/issues/789) | NumPy input speed & memory improvement (followup) |
#### TensorFlow Support
> **Parent:** [#719](https://github.com/apache/mahout/issues/719) - Supply tf.Tensor in input format
| Issue | Task |
|-------|------|
| [#728](https://github.com/apache/mahout/issues/728) | Add TensorFlow eager tensor + CPU/GPU detection |
| [#729](https://github.com/apache/mahout/issues/729) | GPU TF tensor support with dlpack |
| [#730](https://github.com/apache/mahout/issues/730) | TF-Specific Optimizations and Documentation |
---
### Bugs
| Issue | Title |
|-------|-------|
| [#696](https://github.com/apache/mahout/issues/696) | [Bug] CUDA memory illegal address error |
| [#743](https://github.com/apache/mahout/issues/743) | Fix async amplitude pipeline out-of-bounds write |
| [#765](https://github.com/apache/mahout/issues/765) | Unify Null Value Handling (Batch vs Streaming) |
---
### Performance & Optimization
| Issue | Title |
|-------|-------|
| [#714](https://github.com/apache/mahout/issues/714) | Merge norm and encode into a kernel method |
| [#746](https://github.com/apache/mahout/issues/746) | DLPack shape/strides: Support batch 2D tensor |
---
### Infrastructure & Tooling
| Issue | Title |
|-------|-------|
| [#678](https://github.com/apache/mahout/issues/678) | Make CI runs when push on local branch |
| [#736](https://github.com/apache/mahout/issues/736) | Add clippy & rustfmt as linter |
| [#750](https://github.com/apache/mahout/issues/750) | Reconstruct project files into multiple platforms |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [QDP] QDP POC Roadmap [mahout]
Posted by "viiccwen (via GitHub)" <gi...@apache.org>.
viiccwen closed issue #799: [QDP] QDP POC Roadmap
URL: https://github.com/apache/mahout/issues/799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mahout.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [QDP] QDP POC Roadmap [mahout]
Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on issue #799:
URL: https://github.com/apache/mahout/issues/799#issuecomment-4222761002
Hi @alisha-1000 feel free to open pr to solve them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mahout.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [QDP] QDP POC Roadmap [mahout]
Posted by "alisha-1000 (via GitHub)" <gi...@apache.org>.
alisha-1000 commented on issue #799:
URL: https://github.com/apache/mahout/issues/799#issuecomment-3939385058
Hi @guan404ming,
Thanks for maintaining this detailed roadmap — it’s very helpful for understanding QDP’s current maturity.
I’m looking to contribute in a way that aligns with the Qumat 0.5.x milestone priorities. From the roadmap, it seems the remaining high-impact areas are:
Phase 5: Input format support (Torch / NumPy / TensorFlow)
Phase 6: Cross-device testing
Phase 8: Documentation, example notebooks, benchmarks
Multi-architecture GPU optimization (Phase 7)
Could you advise which of these areas is currently the highest priority for contributors?
I’m particularly interested in:
TensorFlow DLPack GPU pathway (#729)
Batch shape/stride correctness for DLPack (#746)
Multi-architecture GPU optimization (if scoped)
I’d like to make sure my contribution aligns with the near-term release goals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mahout.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [QDP] QDP POC Roadmap [mahout]
Posted by "viiccwen (via GitHub)" <gi...@apache.org>.
viiccwen commented on issue #799:
URL: https://github.com/apache/mahout/issues/799#issuecomment-5306267774
Closing as superseded. The original QDP proof of concept has been delivered and released. Remaining input-format and benchmark work continues under #716, #752, and #772, while current pipeline cleanup is tracked by #1338.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mahout.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org