(spark) branch branch-4.x updated: [SPARK-58293][ML] Delay VectorIndexerModel attribute initialization

[ru...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zhengruifeng pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 7eb542ae5d33 [SPARK-58293][ML] Delay VectorIndexerModel attribute initialization
7eb542ae5d33 is described below

commit 7eb542ae5d33af2b83123de5d46d5d246675ba87
Author: Ruifeng Zheng <ru...@apache.org>
AuthorDate: Fri Jul 24 18:27:13 2026 +0800

    [SPARK-58293][ML] Delay VectorIndexerModel attribute initialization
    
    ### What changes were proposed in this pull request?
    
    Replaces the retained `partialFeatureAttributes` field in `VectorIndexerModel` with an on-demand private method. The attributes are now computed only when output metadata is prepared.
    
    ### Why are the changes needed?
    
    The attributes are derived state used only while preparing output metadata. Retaining them on every fitted or loaded model allocates one attribute per feature even when the model is never transformed. Computing them at the call site avoids that unnecessary retained state.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    The test suite was not run locally.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Codex
    
    Closes #57483 from zhengruifeng/delay-partial-feature-attributes-dev3.
    
    Authored-by: Ruifeng Zheng <ru...@apache.org>
    Signed-off-by: Ruifeng Zheng <ru...@apache.org>
    (cherry picked from commit 6d6e2dc77bfc96638751687e49da2fe1b3936131)
    Signed-off-by: Ruifeng Zheng <ru...@apache.org>
---
 .../scala/org/apache/spark/ml/feature/VectorIndexer.scala   | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala b/mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala
index 7fce6d51dcfc..57d28bba2eeb 100644
--- a/mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala
+++ b/mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala
@@ -326,10 +326,10 @@ class VectorIndexerModel private[ml] (
   }
 
   /**
-   * Pre-computed feature attributes, with some missing info.
-   * In transform(), set attribute name and other info, if available.
+   * Computes feature attributes with some missing information.
+   * In transform(), set attribute name and other information, if available.
    */
-  private val partialFeatureAttributes: Array[Attribute] = {
+  private def partialFeatureAttributes: Array[Attribute] = {
     val attrs = new Array[Attribute](numFeatures)
     var categoricalFeatureCount = 0 // validity check for numFeatures, categoryMaps
     var featureIndex = 0
@@ -358,8 +358,11 @@ class VectorIndexerModel private[ml] (
       }
       featureIndex += 1
     }
-    require(categoricalFeatureCount == categoryMaps.size, "VectorIndexerModel given categoryMaps" +
-      s" with keys outside expected range [0,...,numFeatures), where numFeatures=$numFeatures")
+    require(
+      categoricalFeatureCount == categoryMaps.size,
+      "VectorIndexerModel given categoryMaps" +
+        s" with keys outside expected range [0,...,numFeatures), " +
+        s"where numFeatures=$numFeatures")
     attrs
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org