You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2026/08/12 07:05:32 UTC
[PR] [SPARK-58732][ML] Avoid MLlib vector conversion in Normalizer [spark]
zhengruifeng opened a new pull request, #57955:
URL: https://github.com/apache/spark/pull/57955
### What changes were proposed in this pull request?
Update ML Normalizer to calculate the norm with ML vectors directly. Clone the dense or sparse value array and scale the cloned output with BLAS.scal, while reusing sparse indices. The implementation follows the existing MLlib Normalizer structure.
### Why are the changes needed?
ML Normalizer currently converts every input vector from ML to MLlib and converts the normalized result back to ML. Avoiding those conversions reduces allocation and transform overhead.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The existing NormalizerSuite covers dense, sparse, zero-vector, and parameterized normalization behavior.
build/sbt 'mllib/testOnly org.apache.spark.ml.feature.NormalizerSuite'
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58732][ML] Avoid MLlib vector conversion in Normalizer [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #57955: [SPARK-58732][ML] Avoid MLlib vector conversion in Normalizer
URL: https://github.com/apache/spark/pull/57955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58732][ML] Avoid MLlib vector conversion in Normalizer [spark]
Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #57955:
URL: https://github.com/apache/spark/pull/57955#issuecomment-5274139605
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/cddd9aefe2ab342d62073a5babde9930015dd78d
- merged into branch-4.x https://github.com/apache/spark/commit/70c1be3a572a21271c522a2d7e99b4690ae581bd
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58732][ML] Avoid MLlib vector conversion in Normalizer [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57955:
URL: https://github.com/apache/spark/pull/57955#issuecomment-5268403044
Thank you @zhengruifeng!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org