You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mahout.apache.org by "rich7420 (via GitHub)" <gi...@apache.org> on 2026/03/02 11:09:39 UTC

[PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

rich7420 opened a new pull request, #1106:
URL: https://github.com/apache/mahout/pull/1106

   ### Related Issues
   
   <!-- Closes #123 -->
   
   ### Changes
     - **New benchmark**: `encoding_benchmarks/qdp_pipeline/creditcardfraud_amplitude.py` — 5-qubit
     amplitude VQC on Credit Card Fraud data, aligned with PennyLane baseline (same circuit, loss,
     optimizer). Closes the QDP vs baseline training time gap from ~22% slower to <1% gap.
     - **New baseline**: `encoding_benchmarks/pennylane_baseline/creditcardfraud_amplitude.py` —
     PennyLane reference implementation with AUPRC/F1 metrics for imbalanced data.
     - **`QuantumDataLoader` API**: added `source_array(X)` (in-memory, no temp file),
     `as_torch(device)`, and `as_numpy()` for ergonomic batch output format.
     - **Rust `PipelineIterator`**: added `new_from_array()` constructor; `InMemory` `next_batch` now
      passes `&data[start..end]` slice directly (no per-batch `to_vec()`).
     - **`amplitude.rs`**: moved D2H norm validation to after encode kernel + `device.synchronize()`,
      eliminating a mid-pipeline GPU→CPU roundtrip in `encode_batch`.
     - **Bug fixes** (iris + creditcard benchmarks): `requires_grad=False` on all data arrays to
     prevent AdamOptimizer from computing unnecessary gradients through state vectors;
     `AmplitudeEmbedding(normalize=False)` in place of `StatePrep`; `.real` extraction after
     `torch.from_dlpack()` to convert complex128 DLPack output to float64.
   
   - [ ] Bug fix
   - [x] New feature
   - [x] Refactoring
   - [ ] Documentation
   - [ ] Test
   - [ ] CI/CD pipeline
   - [ ] Other
   
   ### Why
   
   <!-- Why is this change needed? -->
   
   ### How
   
   <!-- What was done? -->
   
   ## Checklist
   
   - [x] Added or updated unit tests for all changes
   - [x] Added or updated documentation for all changes
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-3983713085

   It seems a little be too big one. Sorry about that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4888549141

   This pull request has been automatically closed because there has been no more activity in the 7 days since being marked stale.
   
   Please feel free to reopen or open a new pull request if you'd still like this to be addressed. You can always ask for help on the [Mahout dev mailing list](https://lists.apache.org/list.html?dev@mahout.apache.org) or in [GitHub Discussions](https://github.com/apache/mahout/discussions).
   
   Thanks again for your contribution!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4824180843

   This pull request has been automatically marked as stale because it has not had any activity for 30 days.
   
   It will be closed in another 7 days if no further activity occurs. Thank you for your contribution. If you'd like to keep this open, leave any comment and the stale label will be removed.
   
   You can always ask for help on the [Mahout dev mailing list](https://lists.apache.org/list.html?dev@mahout.apache.org) or in [GitHub Discussions](https://github.com/apache/mahout/discussions).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4028336068

   no problem!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "ryankert01 (via GitHub)" <gi...@apache.org>.
ryankert01 commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-3987632821

   > amplitude.rs: moved D2H norm validation to after encode kernel + device.synchronize(),
   eliminating a mid-pipeline GPU→CPU roundtrip in encode_batch.
   
   nice


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "400Ping (via GitHub)" <gi...@apache.org>.
400Ping commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-3988228883

   Please solve conflicts


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #1106: [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations
URL: https://github.com/apache/mahout/pull/1106


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4114933991

   > The array-loader optimization claim does not match the implementation in this PR. `create_array_loader()` says batching uses slices without per-batch `to_vec()`, but `PipelineIterator::take_batch_from_source()` still clones each in-memory batch with `data[start..end].to_vec()`.
   
   Great catch! You are right that the code in take_batch_from_source() was misleading.
   Actually, next_batch() already handles the InMemory variant directly using the zero-copy data[start..end] slice logic and never calls take_batch_from_source() for it. To avoid confusion and make the implementation match the documentation claims, I've replaced the dead-code InMemory arm in take_batch_from_source() with unreachable!().
   I've also rebased the branch onto main and resolved all the merge conflicts. Thanks for the review!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "400Ping (via GitHub)" <gi...@apache.org>.
400Ping commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4564256045

   @rich7420 Are you planning to continue to work on this ? If so please rebase and solve conflicts.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on code in PR #1106:
URL: https://github.com/apache/mahout/pull/1106#discussion_r2875654664


##########
qdp/qdp-python/qumat_qdp/loader.py:
##########
@@ -254,6 +343,25 @@ def _create_iterator(self) -> Iterator[object]:
             )
         )
 
+    def _wrap_iterator(self, raw_iter: Iterator[object]) -> Iterator[Any]:
+        if self._output_format is None:
+            yield from raw_iter
+            return
+        kind = self._output_format[0]
+        if kind == "torch":
+            device = self._output_format[1]
+            for qt in raw_iter:
+                t = _torch.from_dlpack(qt)
+                yield t.cpu() if device == "cpu" else t
+        elif kind == "numpy":
+            for qt in raw_iter:
+                yield _torch.from_dlpack(qt).cpu().numpy()

Review Comment:
   oh , nice catch! you're right



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "viiccwen (via GitHub)" <gi...@apache.org>.
viiccwen commented on code in PR #1106:
URL: https://github.com/apache/mahout/pull/1106#discussion_r2874103097


##########
qdp/qdp-python/qumat_qdp/loader.py:
##########
@@ -254,6 +343,25 @@ def _create_iterator(self) -> Iterator[object]:
             )
         )
 
+    def _wrap_iterator(self, raw_iter: Iterator[object]) -> Iterator[Any]:
+        if self._output_format is None:
+            yield from raw_iter
+            return
+        kind = self._output_format[0]
+        if kind == "torch":
+            device = self._output_format[1]
+            for qt in raw_iter:
+                t = _torch.from_dlpack(qt)
+                yield t.cpu() if device == "cpu" else t
+        elif kind == "numpy":
+            for qt in raw_iter:
+                yield _torch.from_dlpack(qt).cpu().numpy()

Review Comment:
   `as_torch()` validates that `torch` is installed, but `as_numpy()` does not. Then `_wrap_iterator()` calls `_torch.from_dlpack(...)` for the `"numpy"` path.
   
   Does it mean `as_numpy()` can succeed at configuration time and then fail during iteration with an unclear runtime error if PyTorch is not installed. 🤔



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4191008578

   > Great catch! You are right that the code in take_batch_from_source() was misleading.
   Actually, next_batch() already handles the InMemory variant directly using the zero-copy data[start..end] slice logic and never calls take_batch_from_source() for it. To avoid confusion and make the implementation match the documentation claims, I've replaced the dead-code InMemory arm in take_batch_from_source() with unreachable!().
   I've also rebased the branch onto main and resolved all the merge conflicts. Thanks for the review!
   
   Make sense to me, thanks for the update. I could code change wise looks nie, plz help resolve the conflict, thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "rich7420 (via GitHub)" <gi...@apache.org>.
rich7420 commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4010061307

   plz take a look and test, not hurry once you have time


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "400Ping (via GitHub)" <gi...@apache.org>.
400Ping commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4088145718

   Please solve conflicts


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [QDP] feat: add credit card fraud benchmark + amplitude encoding optimizations [mahout]

Posted by "400Ping (via GitHub)" <gi...@apache.org>.
400Ping commented on PR #1106:
URL: https://github.com/apache/mahout/pull/1106#issuecomment-4088451330

   Can you also give some more context in the PR description for why is this needed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org